Introduction
Model and problem statement
In this paper, we consider a stochastic, discrete-time model describing the spiking activity of a network of n n n interacting neurons partitioned into k k k communities. The model has two components, which can be informally described as follows.
The first is a directed weighted graph with n n n vertices, whose edges can be of k + 1 k+1 k + 1 types: k k k intra-community types, one for each community, and a single inter-community type. An edge is of intra-community type if it connects two neurons in the same community, and of inter-community type otherwise. Each edge in the graph carries a weight whose absolute value is the same for all edges of a given type; we refer to this common value as the intensity of the edge type. The sign of the weight, however, is independent of the edge type: edges with positive weights are called excitatory , while those with negative weights are called inhibitory . For most of our results, the graph structure and the edge signs are random. Specifically, edges are present independently of one another, with probabilities depending only on their types, as in a Stochastic Block Model [1 ] . Independently of the edge configuration, the edge signs are independent and identically distributed, with each sign being positive with a fixed probability. We write Θ \Theta Θ for the matrix indicating which edges are present in the graph and H H H for the matrix of signs. Hereafter, we call the pair ( Θ , H ) (\Theta,H) ( Θ , H ) the environment and denote by A = ( A i j ) i , j ∈ [ n ] A=(A_{ij})_{i,j\in[n]} A = ( A ij ) i , j ∈ [ n ] its associated interaction matrix . The entry A i j A_{ij} A ij represents the weight of the edge from neuron i i i to neuron j j j , scaled by n − 1 n^{-1} n − 1 , if the edge is present, and is 0 0 0 otherwise.
The second component is a sequence X = ( X t ) t ∈ N X=(X_t)_{t\in\mathbb{N}} X = ( X t ) t ∈ N of random vectors taking values in { 0 , 1 } n \{0,1\}^n { 0 , 1 } n , representing the time evolution of the spikes of the n n n neurons. Writing X i , t X_{i,t} X i , t to denote the i i i -th coordinate of the random vector X t X_t X t , we have X i , t = 1 X_{i,t}=1 X i , t = 1 if neuron i i i spikes at time t t t , and 0 0 0 otherwise. Conditionally on the realization of the environment ( Θ , H ) (\Theta,H) ( Θ , H ) , the sequence X X X forms a Markov chain. Specifically, given the environment and the entire configuration of spikes at time t − 1 t-1 t − 1 , each neuron i i i spikes independently of the others at the next time step, with probability
P ( Θ , H ) ( X i , t = 1 ∣ X t − 1 ) = λ + ( A T X t − 1 ) i , (1) \mathbb{P}_{(\Theta,H)}(X_{i,t}=1\mid X_{t-1})=\lambda+(A^{\mathsf{T}}X_{t-1})_i, \tag*{(1)} P ( Θ , H ) ( X i , t = 1 ∣ X t − 1 ) = λ + ( A T X t − 1 ) i , (1)
where A T A^{\mathsf{T}} A T denotes the transpose of the matrix A A A and λ ∈ [ 0 , 1 ] \lambda\in[0,1] λ ∈ [ 0 , 1 ] represents the spontaneous firing rate , which is the same for all neurons. The second term on the right-hand side of (1) is a weighted sum of the spikes emitted at time t − 1 t-1 t − 1 by the neurons that point to neuron i i i in the graph, where the weights are given by the corresponding entries of the interaction matrix A A A . Since the entries of A A A are of order n − 1 n^{-1} n − 1 , the interactions are individually weak.
Throughout our analysis, we work under conditions on the model parameters (see (14) and (15) ) ensuring that the right-hand side of (1) is always between 0 0 0 and 1 1 1 for a set of realizations of the environment ( Θ , H ) (\Theta,H) ( Θ , H ) whose probability approaches 1 1 1 as n → ∞ n\to\infty n → ∞ (this is formalized in Lemma 3.1). We refer to such realizations as admissible realizations . Conditionally on admissible realizations of the environment, the Markov chain X X X admits a unique invariant measure, and we always work with its stationary version. In the sequel, we denote by P ( Θ , H ) \mathbb{P}_{(\Theta,H)} P ( Θ , H ) the law of the stationary chain X X X given the environment ( Θ , H ) (\Theta,H) ( Θ , H ) , and by E ( Θ , H ) \mathbb{E}_{(\Theta,H)} E ( Θ , H ) the corresponding expectation.
In what follows, let E ( Θ , H ) [ X t ] \mathbb{E}_{(\Theta,H)}[X_t] E ( Θ , H ) [ X t ] be the column vector of the mean firing activity at time t t t and let Σ ( Θ , H ) ( t ) \Sigma^{(t)}_{(\Theta,H)} Σ ( Θ , H ) ( t ) denote the t t t -lagged firing covariance matrix that is defined as
Σ ( Θ , H ) ( t ) : = E ( Θ , H ) [ X t ( X 0 ) T ] − E ( Θ , H ) [ X t ] ( E ( Θ , H ) [ X 0 ] ) T , t ∈ N . \Sigma^{(t)}_{(\Theta,H)}:=\mathbb{E}_{(\Theta,H)}[X_t(X_0)^{\mathsf{T}}]-\mathbb{E}_{(\Theta,H)}[X_t](\mathbb{E}_{(\Theta,H)}[X_0])^{\mathsf{T}},\quad t\in\mathbb{N}. Σ ( Θ , H ) ( t ) := E ( Θ , H ) [ X t ( X 0 ) T ] − E ( Θ , H ) [ X t ] ( E ( Θ , H ) [ X 0 ] ) T , t ∈ N .
By stationarity, the mean vectors E ( Θ , H ) [ X t ] \mathbb{E}_{(\Theta,H)}[X_t] E ( Θ , H ) [ X t ] do not depend on t t t , and in the rest of the paper we refer to E ( Θ , H ) [ X 0 ] \mathbb{E}_{(\Theta,H)}[X_0] E ( Θ , H ) [ X 0 ] as the vector of mean firing rates. Note that both this vector and the lagged covariance matrices are random, depending on the realization of the environment. Also, recall that the law of this environment is determined by the partition of the population into the k k k communities, the edge probabilities, the intensities of the different edge types, and the probability that an edge is excitatory.
This leads to the following related problems, which we address in this paper.
Problem statement. As the size n n n of the neuronal network grows while the number k k k of communities is kept fixed, how does the community structure of the environment manifest itself in the mean firing rates and in the lagged firing covariance matrices? Conversely, to what extent can the partition of the population into communities be recovered from these quantities alone?
Overview of the main results
Throughout the paper, we denote by C = { C 1 , … , C k } \mathcal{C} = \{C_1,\ldots,C_k\} C = { C 1 , … , C k } the partition of the set [ n ] : = { 1 , … , n } [n] := \{1,\ldots,n\} [ n ] := { 1 , … , n } representing the k k k communities of the entire network of neurons, whose size is n n n . In our analysis, we assume that all communities have sizes proportional to the total number of neurons in the network, i.e., we suppose that ∣ C b ∣ = α b n ≥ 1 \lvert C_b\rvert= \alpha_b n \ge1 ∣ C b ∣ = α b n ≥ 1 for every b ∈ [ k ] b \in[k] b ∈ [ k ] , for some positive values α 1 , … , α k \alpha_1,\ldots,\alpha_k α 1 , … , α k satisfying α 1 + … + α k = 1 \alpha_1+\ldots+\alpha_k=1 α 1 + … + α k = 1 . In what follows, we write ∥ ⋅ ∥ ∞ \lVert\cdot\rVert_\infty ∥ ⋅ ∥ ∞ to denote the usual supremum norm in R n \mathbb{R}^n R n and K K K to indicate a positive constant independent of n n n .
Recall that E ( Θ , H ) [ X 0 ] \mathbb{E}_{(\Theta,H)}[X_0] E ( Θ , H ) [ X 0 ] denotes the random vector of mean firing rates, where the randomness comes from the random environment ( Θ , H ) (\Theta,H) ( Θ , H ) . In our first main result, Theorem 3.1, we show how the community structure of the environment is encoded in this random vector when n n n is large. Specifically, we show that with high probability
∥ E ( Θ , H ) [ X 0 ] − m ‾ ∥ ∞ ≤ K log ( n ) n , \left\lVert\mathbb{E}_{(\Theta,H)}[X_0]-\overline{m}\right\rVert_\infty\le K\sqrt{\frac{\log(n)}{n}}, E ( Θ , H ) [ X 0 ] − m ∞ ≤ K n log ( n ) ,
where m ‾ \overline{m} m is the n n n -dimensional vector defined as
m ‾ : = λ ∑ b ∈ [ k ] ζ b 1 C b , (2) \overline{m}:=\lambda\sum_{b\in[k]}\zeta_b\mathbf{1}_{C_b}, \tag*{(2)} m := λ b ∈ [ k ] ∑ ζ b 1 C b , (2)
where ζ = ( ζ 1 , … , ζ k ) \zeta=(\zeta_1,\ldots,\zeta_k) ζ = ( ζ 1 , … , ζ k ) is an explicitly known k k k -dimensional vector (see, (17) ) and 1 C b \mathbf{1}_{C_b} 1 C b denotes the n n n -dimensional column vector with entries in C b C_b C b equal to 1 1 1 and 0 0 0 otherwise. Note that all entries of m ‾ \overline{m} m corresponding to neurons in community C b C_b C b share the same value λ ζ b \lambda\zeta_b λ ζ b , that is, m ‾ \overline{m} m is constant within each community. The vector ζ \zeta ζ can be explicitly computed from the k × k k\times k k × k matrix W W W defined in (16) whose entry W a b W_{ab} W ab corresponds to the mean cumulative effect that community C b C_b C b exerts on community C a C_a C a .
The proof of Theorem 3.1 can be divided into three steps. In the first step, we prove that for all admissible realizations of the environment, we can write
E ( Θ , H ) [ X 0 ] = ( I n − A T ) − 1 λ 1 n , \mathbb{E}_{(\Theta,H)}[X_0]=(I_n-A^{\mathsf{T}})^{-1}\lambda\mathbf{1}_n, E ( Θ , H ) [ X 0 ] = ( I n − A T ) − 1 λ 1 n ,
where I n I_n I n is the n n n -dimensional identify matrix and 1 n \mathbf{1}_n 1 n is the n n n -dimensional column vector with all entries equal to 1 1 1 . This is a consequence of Proposition 2.1 stated in § 2.2 .
The second step consists in quantifying the error of approximating the random vector E ( Θ , H ) [ X 0 ] \mathbb{E}_{(\Theta,H)}[X_0] E ( Θ , H ) [ X 0 ] by the deterministic vector ( I n − E ( n ) [ A T ] ) − 1 λ 1 n (I_n-\mathbb{E}^{(n)}[A^{\mathsf{T}}])^{-1}\lambda\mathbf{1}_n ( I n − E ( n ) [ A T ] ) − 1 λ 1 n , where E ( n ) \mathbb{E}^{(n)} E ( n ) denotes the expectation taken with respect to the randomness of the environment ( Θ , H ) (\Theta,H) ( Θ , H ) . This follows from Lemma A.1 of Appendix § 5 and Hoeffding’s inequality. The final step is then to show that ( I n − E ( n ) [ A T ] ) − 1 λ 1 n = m ‾ (I_n-\mathbb{E}^{(n)}[A^{\mathsf{T}}])^{-1}\lambda\mathbf{1}_n=\overline{m} ( I n − E ( n ) [ A T ] ) − 1 λ 1 n = m inherits the block structure of the environment ( Θ , H ) (\Theta,H) ( Θ , H ) . Lemma D.2 of Appendix § 8 establishes the main result needed in the proof of this final step.
As a simple consequence of Theorem 3.1, we show that the vector of firing variances, defined as
v ( Θ , H ) ( X 0 ) : = ( E ( Θ , H ) [ X 1 , 0 ] ( 1 − E ( Θ , H ) [ X 1 , 0 ] ) , … , E ( Θ , H ) [ X n , 0 ] ( 1 − E ( Θ , H ) [ X n , 0 ] ) ) T , v_{(\Theta,H)}(X_0) := \left(\mathbb{E}_{(\Theta,H)}[X_{1,0}]\left(1-\mathbb{E}_{(\Theta,H)}[X_{1,0}]\right),\ldots,\mathbb{E}_{(\Theta,H)}[X_{n,0}]\left(1-\mathbb{E}_{(\Theta,H)}[X_{n,0}]\right)\right)^{\mathsf{T}}, v ( Θ , H ) ( X 0 ) := ( E ( Θ , H ) [ X 1 , 0 ] ( 1 − E ( Θ , H ) [ X 1 , 0 ] ) , … , E ( Θ , H ) [ X n , 0 ] ( 1 − E ( Θ , H ) [ X n , 0 ] ) ) T ,
satisfies with high probability
∥ v ( Θ , H ) ( X 0 ) − v ‾ ∥ ∞ ≤ K log ( n ) n , \left\|v_{(\Theta,H)}(X_0)-\overline{v}\right\|_{\infty}\leq K\sqrt{\frac{\log(n)}{n}}, v ( Θ , H ) ( X 0 ) − v ∞ ≤ K n log ( n ) ,
where the n n n -dimensional vector
v ‾ : = ( m ‾ 1 ( 1 − m ‾ 1 ) , … , m ‾ n ( 1 − m ‾ n ) ) T (3) \overline{v}:=\left(\overline{m}_1(1-\overline{m}_1),\ldots,\overline{m}_n(1-\overline{m}_n)\right)^{\mathsf{T}} \tag*{(3)} v := ( m 1 ( 1 − m 1 ) , … , m n ( 1 − m n ) ) T (3)
is also constant within each community (see Corollary 3.1).
Next, we describe how the community structure of the environment emerges in the lagged firing covariances Σ ( Θ , H ) ( t ) \Sigma^{(t)}_{(\Theta,H)} Σ ( Θ , H ) ( t ) as n → ∞ n\to\infty n → ∞ . We start describing what happens for the simultaneous firing covariance, that is for t = 0 t=0 t = 0 . In this case, since the diagonal terms of Σ ( Θ , H ) ( 0 ) \Sigma^{(0)}_{(\Theta,H)} Σ ( Θ , H ) ( 0 ) are already known (they are given by the vector of firing variances v ( Θ , H ) ( X 0 ) v_{(\Theta,H)}(X_0) v ( Θ , H ) ( X 0 ) ), we only need to focus on the behavior of its off-diagonal entries. The asymptotic behavior of these entries is described by means of a k × k k\times k k × k matrix Ψ \Psi Ψ that is the unique solution of the Stein matrix equation
Ψ − W Ψ W T = W D W T , \Psi-W\Psi W^{\mathsf{T}}=WDW^{\mathsf{T}}, Ψ − W Ψ W T = W D W T ,
where D D D is a diagonal matrix whose non-null entries are expressed in terms of the vector ζ \zeta ζ defined in (17) and the parameters α b \alpha_b α b (see (20) ). Specifically, introducing the n × n n\times n n × n matrix
Σ ‾ ( 0 ) : = 1 n ∑ a , b ∈ [ k ] Ψ a b 1 C a 1 C b T (4) \overline{\Sigma}^{(0)}:=\frac{1}{n}\sum_{a,b\in[k]}\Psi_{ab}\mathbf{1}_{C_a}\mathbf{1}_{C_b}^{\mathsf{T}} \tag*{(4)} Σ ( 0 ) := n 1 a , b ∈ [ k ] ∑ Ψ ab 1 C a 1 C b T (4)
in Theorem 3.2 we establish that with high probability
max i ≠ j ∣ Σ ( Θ , H ) , i j ( 0 ) − Σ ‾ i j ( 0 ) ∣ ≤ K log ( n ) n 3 . \max_{i\ne j}\left|\Sigma^{(0)}_{(\Theta,H),ij}-\overline{\Sigma}^{(0)}_{ij}\right|\leq K\sqrt{\frac{\log(n)}{n^3}}. i = j max Σ ( Θ , H ) , ij ( 0 ) − Σ ij ( 0 ) ≤ K n 3 log ( n ) .
We see immediately from (4) that the matrix Σ ‾ ( 0 ) \overline{\Sigma}^{(0)} Σ ( 0 ) has a block structure: the entries Σ ‾ i j ( 0 ) \overline{\Sigma}^{(0)}_{ij} Σ ij ( 0 ) are equal to n − 1 Ψ a b n^{-1}\Psi_{ab} n − 1 Ψ ab for all pairs ( i , j ) ∈ C a × C b (i,j)\in C_a\times C_b ( i , j ) ∈ C a × C b . In particular, this implies that asymptotically as n → ∞ n\to\infty n → ∞ the off-diagonal entries of simultaneous firing covariance are of order n − 1 n^{-1} n − 1 , whereas its diagonal entries are of order 1 1 1 (see Corollary 3.1).
At the high level, the proof of Theorem 3.2 is similar to that of Theorem 3.1, except that we do not work directly with the simultaneous firing covariance Σ ( Θ , H ) ( 0 ) \Sigma^{(0)}_{(\Theta,H)} Σ ( Θ , H ) ( 0 ) . Instead, we work with its vectorized form. This means that we work with the n 2 n^2 n 2 -dimensional column vector obtained by stacking the columns of Σ ( Θ , H ) ( 0 ) \Sigma^{(0)}_{(\Theta,H)} Σ ( Θ , H ) ( 0 ) one by one, denoted by vec ( Σ ( Θ , H ) ( 0 ) ) \operatorname{vec}(\Sigma^{(0)}_{(\Theta,H)}) vec ( Σ ( Θ , H ) ( 0 ) ) . Using this representation, to prove Theorem 3.2, we first show, by relying on Proposition 2.2 , that for all admissible realizations of the environment
vec ( Σ ( Θ , H ) ( 0 ) ) = ( I n 2 − L ) − 1 vec ( Diag ( v ( Θ , H ) ( X 0 ) ) ) , \operatorname{vec}\left(\Sigma^{(0)}_{(\Theta,H)}\right)=\left(I_{n^2}-L\right)^{-1}\operatorname{vec}\left(\operatorname{Diag}\left(v_{(\Theta,H)}(X_0)\right)\right), vec ( Σ ( Θ , H ) ( 0 ) ) = ( I n 2 − L ) − 1 vec ( Diag ( v ( Θ , H ) ( X 0 ) ) ) ,
where vec ( Diag ( v ( Θ , H ) ( X 0 ) ) ) \operatorname{vec}(\operatorname{Diag}(v_{(\Theta,H)}(X_0))) vec ( Diag ( v ( Θ , H ) ( X 0 ))) denotes the vectorized diagonal matrix whose non-null entries are given by the vector of firing variances v ( Θ , H ) ( X 0 ) v_{(\Theta,H)}(X_0) v ( Θ , H ) ( X 0 ) and L L L is a n 2 × n 2 n^2 \times n^2 n 2 × n 2 matrix whose entries are defined as:
L u v = { 0 if u = n ( i − 1 ) + i , for i ∈ [ n ] , ( A ⊤ ⊗ A ⊤ ) u v otherwise . L_{uv} =
\begin{cases}
0 & \text{if } u = n(i-1)+i,\text{ for } i \in[n],\\
(A^\top\otimes A^\top)_{uv} & \text{otherwise}.
\end{cases} L uv = { 0 ( A ⊤ ⊗ A ⊤ ) uv if u = n ( i − 1 ) + i , for i ∈ [ n ] , otherwise .
To conclude, we estimate the error of approximating the random vector vec ( Σ ‾ ( Θ , H ) ( 0 ) ) \operatorname{vec}(\overline{\Sigma}^{(0)}_{(\Theta,H)}) vec ( Σ ( Θ , H ) ( 0 ) ) by the deterministic vector ( I n 2 − E ( n ) [ L ] ) − 1 vec ( Diag ( v ‾ ) ) (I_{n^2}-\mathbb{E}^{(n)}[L])^{-1}\operatorname{vec}(\operatorname{Diag}(\overline{v})) ( I n 2 − E ( n ) [ L ] ) − 1 vec ( Diag ( v )) and then show in Proposition B.1 stated in Appendix B that this latter vector is, in turn, sufficiently close to the vectorized version of the matrix Σ ‾ ( 0 ) \overline{\Sigma}^{(0)} Σ ( 0 ) .
Unlike the previous approximations, the behavior of the t t t -lagged firing covariance matrices for every t ≥ 1 t \ge1 t ≥ 1 is described by a random matrix denoted by Σ ‾ ( Θ , H ) ( t ) \overline{\Sigma}^{(t)}_{(\Theta,H)} Σ ( Θ , H ) ( t ) , which is defined as
Σ ‾ ( Θ , H ) ( t ) : = ( A ( Θ , H ) ⊤ ) t diag ( v ‾ ) + 1 n ∑ a , b ∈ [ k ] ( W t Ψ ) a b 1 C a 1 C b ⊤ , (5) \overline{\Sigma}^{(t)}_{(\Theta,H)} := (A^\top_{(\Theta,H)})^t\operatorname{diag}(\overline{v})+\frac{1}{n}\sum_{a,b\in[k]}(W^t\Psi)_{ab}\mathbf{1}_{C_a}\mathbf{1}_{C_b}^{\top}, \tag*{(5)} Σ ( Θ , H ) ( t ) := ( A ( Θ , H ) ⊤ ) t diag ( v ) + n 1 a , b ∈ [ k ] ∑ ( W t Ψ ) ab 1 C a 1 C b ⊤ , (5)
with v ‾ \overline{v} v and W W W given by (3) and (16), respectively, and Ψ \Psi Ψ as above (see (19) for a rigorous definition of Ψ \Psi Ψ ). This is the content of Theorem 3.3, where we show that with high probability
max i , j ∣ Σ ‾ ( Θ , H ) , i j ( t ) − Σ ‾ i j ( t ) ∣ ≤ K log ( n ) n 3 . \max_{i,j}\left|\overline{\Sigma}^{(t)}_{(\Theta,H),ij}-\overline{\Sigma}^{(t)}_{ij}\right|\le K\sqrt{\frac{\log(n)}{n^3}}. i , j max Σ ( Θ , H ) , ij ( t ) − Σ ij ( t ) ≤ K n 3 log ( n ) .
The proof idea of Theorem 3.3 is as follows. We first use the identity Σ ‾ ( Θ , H ) ( t ) = ( A ( Θ , H ) ⊤ ) t Σ ‾ ( Θ , H ) ( 0 ) \overline{\Sigma}^{(t)}_{(\Theta,H)}=(A^\top_{(\Theta,H)})^t\overline{\Sigma}^{(0)}_{(\Theta,H)} Σ ( Θ , H ) ( t ) = ( A ( Θ , H ) ⊤ ) t Σ ( Θ , H ) ( 0 ) which holds for all admissible realization of the environment by Proposition 2.3. Then, we decompose Σ ‾ ( Θ , H ) ( 0 ) \overline{\Sigma}^{(0)}_{(\Theta,H)} Σ ( Θ , H ) ( 0 ) as the sum of its diagonal and off-diagonal terms and apply Corollary 3.1 and Theorem 3.2 to these terms, respectively. In the final step, we quantify the error of replacing ( A ( Θ , H ) ⊤ ) t (A^\top_{(\Theta,H)})^t ( A ( Θ , H ) ⊤ ) t by its expectation in front of the second term.
The above results describe how the first and second order statistics of the firing process X X X inherits the community structure of the environment, as the number of neurons n → ∞ n \to\infty n → ∞ . In Subsection 3.2, we further investigate the extent to which the community structure can be recovered from these limiting statistics by discussing conditions on the model parameters under which the communities are identifiable. A thorough investigation of community detection in the class of models considered here, including the development and analysis of detection procedures, is left for future work.
Related work
The neuron-firing model considered in this paper is a variant of the one recently studied in [4 ] (), Chevallier and Ost 2026). Unlike our model, in these papers the underlying random graph model is homogeneous, being distributed as a directed Erdős-Rényi model with n n n vertices and edge probability p ∈ [ 0 , 1 ] p \in[0,1] p ∈ [ 0 , 1 ] , a special case of the model considered here. In addition, all edges carry the same intensity, whereas here the intensity depends on the type of the edge. Similarly to our model, the weight of each edge has a sign, though the assignment rule is different. Specifically, the edge from neuron j j j to neuron i i i has positive sign if neuron j j j belongs to a set P + \mathcal{P}_{+} P + and negative otherwise. In their framework, the sets P + \mathcal{P}_{+} P + and P − : = ( P + ) c \mathcal{P}_{-}:=(\mathcal{P}_{+})^{c} P − := ( P + ) c are interpreted as the sets of excitatory and inhibitory neurons, respectively.
In [4 ] , the authors show that it is possible to estimate the edge probability p p p by observing T T T consecutive configurations of spikes of the n n n neurons, with rate n − 1 / 2 + n 1 / 2 / T + n^{-1/2}+n^{1/2}/T+ n − 1/2 + n 1/2 / T + ( log ( T ) / T ) 1 / 2 (\log(T)/T)^{1/2} ( log ( T ) / T ) 1/2 . In particular, in the asymptotic regime in which n n n grows and T T T scales with n n n , the proposed estimator is proved to be consistent provided that n 1 / 2 / T → 0 n^{1/2}/T \to0 n 1/2 / T → 0 as n → ∞ n \to\infty n → ∞ . In the same setting, the paper [5 ] addresses the question of discriminating the sets of excitatory neurons from the inhibitory ones. In their main result, the authors show that this can be done by using simple aggregated and spectral methods for which the proportion of misclassified neurons converges to 0 in probability, provided that n / T → 0 n/T \to0 n / T → 0 (up to log factors) as n → ∞ n \to\infty n → ∞ . Moreover, this condition is proved to be near-optimal in the minimax sense. The methods developed in both papers rely on first and second order statistics of the spiking activity generated by a homogeneous random graph. Here we go beyond the homogeneity assumption, by characterizing these statistics when the underlying random graph model has a block structure, in both its edge probabilities and its intensities.
The problems stated in § 1.1 concern the relation between the architecture of a neuronal network (which characterizes the neuronal interactions) and its firing patterns. In experimental data, we often have access to the firing patterns of large networks of neurons, while their architecture is unobserved. Estimating these architectures from data has attracted considerable attention in recent years. This has been done in the context of interacting chains with variable-length memory in [6 ] , where the architecture of the network is described in terms of interaction neighborhoods and explicit bounds on the probabilities of under- and overestimating these neighborhoods are provided. In a more general context of sparse space-time models, [11 ] obtains oracle inequalities for Lasso-type estimators of the model interactions, valid even when only part of the network is observed.
From a more applied perspective, a number of works approaches the same problem through generalized linear models (GLMs) fitted to recorded spike trains. For example, the work [9 ] applies a GLM to the cross-correlogram of spike train pairs, together with a thresholding procedure to determine which inferred connections are statistically reliable, and recovers both the weighted adjacencies of the neuronal network and the excitatory or inhibitory character of each neuron. To achieve optimal performance the parameters of the model have to be carefully tuned for each individual dataset. This obstacle is addressed in the subsequent work [7 ] , where a convolutional neural network trained on simulated data replaces the GLM fitting. This removes the need for fine tuning model parameters, but the interpretation of results is less clear. As another example, let us mention [10 ] that proposes a Bayesian framework to uncover latent structures in neural circuits, such as the spatial arrangement of neurons and their functional types, combining a GLM for spike generation with graph-theoretical priors governing the relationship between latent features and neural connectivity patterns.
The GLM-based methods above are assessed on synthetic and empirical data rather than being supported by theoretical guarantees. We note that the cross-correlograms on which several of them are based correspond, to some extent, to lagged covariances of fixed pair of neurons, viewed as a function of the lag. The asymptotic description of these covariances provided in Theorems 3.2 and 3.3 may, therefore, be an important step towards a rigorous account of what such methods can recover.
In the theoretical neuroscience literature, the relation between architecture and activity has also been studied in the opposite direction, as for example in the works [Ostoijc et al.(2009)] and [14 ] , where the goal is to quantify correlations between the activity of pairs of neurons for different stochastic models and network architectures. In [Ostoijc et al.(2009)], this is done for a pair of integrate-and-fire neurons, the surrounding network being represented by a stationary Gaussian input. In [14 ] , neurons are modeled by Hawkes processes, a class of interacting point processes widely used in neuroscience. For this model, the authors obtain closed expressions for the covariances integrated over the time lag, under the assumption that the network is regular, i.e., all neurons have the same in- and out-degrees. They also assess through simulations the accuracy of these expressions when the network is no longer regular.
We conclude this section by briefly mentioning that stochastic models having a latent block structure have been recently considered in [2 , 3 , 13 ] . In the works [2 ] and [3 ] , the observations are assumed to be i.i.d. copies of a random vector. In the former, the observed random vector is distributed as the Ising model without external field where the coupling weights have a block structure, while in the latter the structure is imposed on the covariance matrix of the observed vector. In [13 ] , the network and its communities are reconstructed jointly from time series by a Bayesian procedure, without rigorously discussing guarantees of recovery. In our setting, the observations form a single trajectory of a dependent process, whose dynamics is Markovian conditionally on the environment. Since the associated transition probabilities depend on the environment that has different communities, one expects these communities to leave a trace in the lagged covariances; our Theorems 3.2 and 3.3 make this precise.
Outline of the paper
The rest of the paper is organized as follows. In Section 2, we introduce the neuron-firing model with communities on a fixed interaction graph and fix most of the notation used throughout the paper. Section 2.2 collects three auxiliary results showing that the first and second order statistics of the firing activity can be computed exactly from the interaction matrix, for every choice of the interaction graph. In Section 3, we extend our neuron-firing model to a random environment distributed as a stochastic block model. Our main results are stated in Section 3.1. In Section 3.2, we derive from these results the consequences for the identifiability of the communities. All proofs are deferred to the appendices.
Neurons firing model with communities (NFC)
In Section 2.1, we introduce our neuron-firing model with communities where the interactions between neurons are fixed, followed by a few preliminary results on this model, presented in Section 2.2.
NFC on a fixed environment
Consider integers n ≥ k ≥ 1 n \ge k \ge1 n ≥ k ≥ 1 and let [ n ] [n] [ n ] denote the set of neurons and let C = { C 1 , … , C k } \mathcal{C} = \{C_1,\ldots,C_k\} C = { C 1 , … , C k } be a partition of [ n ] [n] [ n ] representing k k k different communities of neurons. Let θ = ( θ i j ) 1 ≤ i , j ≤ n \theta= (\theta_{ij})_{1 \le i,j \le n} θ = ( θ ij ) 1 ≤ i , j ≤ n be a { 0 , 1 } \{0,1\} { 0 , 1 } matrix encoding the pairwise (directed) interaction between neurons, where θ i j = 0 \theta_{ij}=0 θ ij = 0 means that neuron i i i does not interact directly with neuron j j j . Note that the matrix θ \theta θ need not be symmetric which allows to have, e.g., θ i j = 0 \theta_{ij}=0 θ ij = 0 and θ j i = 1 \theta_{ji}=1 θ j i = 1 ; that is the matrix θ \theta θ can be interpreted as a directed graph. There are k + 1 k+1 k + 1 types of edges: k k k intra-community and 1 1 1 inter-community, depending on whether they connect neurons in a same community among the k k k possible ones or not, respectively. Given two neurons i , j i,j i , j , we shall write i ∼ b j i \overset{b}{\sim} j i ∼ b j to denote that i i i and j j j both belong to the community C b C_b C b and i ≁ j i \not\sim j i ∼ j to denote that they belong to different communities. As mentioned in the introduction, the intensity of the interaction may differ depending on the edge type. Each interaction in θ \theta θ be inhibitory or excitatory and this is encoded in a matrix η = ( η i j ) 1 ≤ i , j ≤ n \eta=(\eta_{ij})_{1\le i,j\le n} η = ( η ij ) 1 ≤ i , j ≤ n , with { − 1 , 1 } \{-1,1\} { − 1 , 1 } entries. Let G = ( C , θ , η ) G=(\mathcal{C},\theta,\eta) G = ( C , θ , η ) be a directed weighted graph with n n n vertices and k k k communities summarizing the network structure introduced above. Henceforth, we shall refer to G G G as the interaction graph .
The neurons’ activity is modeled via a system of n n n interacting { 0 , 1 } \{0,1\} { 0 , 1 } -valued random variables { X i , t , i ∈ [ n ] , t ∈ { 0 } ∪ Z + } \{X_{i,t},i\in[n],t\in\{0\}\cup\mathbb{Z}_{+}\} { X i , t , i ∈ [ n ] , t ∈ { 0 } ∪ Z + } , representing whether a neuron fires or not at a given time; X i , t = 1 X_{i,t}=1 X i , t = 1 means that neuron i i i fires at time t t t and X i , t = 0 X_{i,t}=0 X i , t = 0 that it does not fire. These random variables are coupled via the interaction graph G = ( C , θ , η ) G=(\mathcal{C},\theta,\eta) G = ( C , θ , η ) as detailed below. Denoting by X t = ( X 1 , t , … , X n , t ) X_t=(X_{1,t},\ldots,X_{n,t}) X t = ( X 1 , t , … , X n , t ) the configuration of the system of n n n neurons at time t t t , the process X = ( X t ) t ≥ 0 X=(X_t)_{t\ge0} X = ( X t ) t ≥ 0 evolves as a discrete time Markov chain on the state space { 0 , 1 } n \{0,1\}^n { 0 , 1 } n in which the conditional distribution of X t X_t X t given that X t − 1 = x X_{t-1}=x X t − 1 = x , x ∈ { 0 , 1 } n x\in\{0,1\}^n x ∈ { 0 , 1 } n , is that of n n n independent Bernoulli random variables with parameters ϕ G , i ( x ) \phi_{G,i}(x) ϕ G , i ( x ) , i ∈ [ n ] i\in[n] i ∈ [ n ] , i.e., for all x , y ∈ { 0 , 1 } n x,y\in\{0,1\}^n x , y ∈ { 0 , 1 } n and t ≥ 1 t\ge1 t ≥ 1 ,
P G ( X t = y ∣ X t − 1 = x ) = ∏ i ∈ [ n ] ( ϕ G , i ( x ) ) y i ( 1 − ϕ G , i ( x ) ) 1 − y i , (6) \mathbb{P}_G(X_t=y\mid X_{t-1}=x)=\prod_{i\in[n]}(\phi_{G,i}(x))^{y_i}(1-\phi_{G,i}(x))^{1-y_i}, \tag*{(6)} P G ( X t = y ∣ X t − 1 = x ) = i ∈ [ n ] ∏ ( ϕ G , i ( x ) ) y i ( 1 − ϕ G , i ( x ) ) 1 − y i , (6)
where, given x = ( x 1 , … , x N ) ∈ { 0 , 1 } n x=(x_1,\ldots,x_N)\in\{0,1\}^n x = ( x 1 , … , x N ) ∈ { 0 , 1 } n ,
ϕ G , i ( x ) : = λ + ∑ b ∈ [ k ] 1 { i ∈ C b } ( μ b n ∑ j : j ∼ b i θ j i η j i x j + μ ≁ n ∑ j : j ≁ i θ j i η j i x j ) . (7) \phi_{G,i}(x):=\lambda+\sum_{b\in[k]}\mathbf{1}_{\{i\in C_b\}}\left(\frac{\mu_b}{n}\sum_{j:j\overset{b}{\sim}i}\theta_{ji}\eta_{ji}x_j+\frac{\mu_{\not\sim}}{n}\sum_{j:j\not\sim i}\theta_{ji}\eta_{ji}x_j\right). \tag*{(7)} ϕ G , i ( x ) := λ + b ∈ [ k ] ∑ 1 { i ∈ C b } n μ b j : j ∼ b i ∑ θ j i η j i x j + n μ ∼ j : j ∼ i ∑ θ j i η j i x j . (7)
Equation (7) may be interpreted as the probability of neuron i i i firing at a given time given the system’s configuration at the preceding time. The first inner summation covers neurons in the same community of i i i while the second covers neurons in the other communities. The parameter λ ∈ [ 0 , 1 ] \lambda\in[0,1] λ ∈ [ 0 , 1 ] represents the spontaneous activity. The parameters ( μ b ) b ∈ [ k ] (\mu_b)_{b\in[k]} ( μ b ) b ∈ [ k ] and μ ≁ \mu_{\not\sim} μ ∼ , all positive, codify the intensity of the interaction between neurons in the same community and in different ones, respectively. A neuron j j j interacting directly with i i i (i.e., θ j i = 1 \theta_{ji}=1 θ j i = 1 ) may either excite (η j i = 1 \eta_{ji}=1 η j i = 1 ) or inhibit (η j i = − 1 \eta_{ji}=-1 η j i = − 1 ) neuron i i i ’s activity, increasing or decreasing its firing probability by an amount of μ b / n \mu_b/n μ b / n (resp., μ ≁ / n \mu_{\not\sim}/n μ ∼ / n ) if j ∼ b i j\overset{b}{\sim}i j ∼ b i (resp., j ≁ i j\not\sim i j ∼ i ).
It will be convenient to write (7) in a matrix form. Thus, given the set of neurons [ n ] [n] [ n ] , the k k k communities C \mathcal{C} C and the interaction graph G = ( C , θ , η ) G=(\mathcal{C},\theta,\eta) G = ( C , θ , η ) , we introduce the corresponding interaction matrix A = ( A i j ) i , j ∈ [ n ] A=(A_{ij})_{i,j\in[n]} A = ( A ij ) i , j ∈ [ n ] whose entries are defined as (to avoid clutter in the notation we omit the dependence on G G G ),
A i j : = ∑ b ∈ [ k ] μ b n θ i j η i j 1 { i ∼ b j } + μ ≁ n θ i j η i j 1 { i ≁ j } . (8) A_{ij}:=\sum_{b\in[k]}\frac{\mu_b}{n}\theta_{ij}\eta_{ij}\mathbf{1}_{\{i\overset{b}{\sim}j\}}+\frac{\mu_{\not\sim}}{n}\theta_{ij}\eta_{ij}\mathbf{1}_{\{i\not\sim j\}}. \tag*{(8)} A ij := b ∈ [ k ] ∑ n μ b θ ij η ij 1 { i ∼ b j } + n μ ∼ θ ij η ij 1 { i ∼ j } . (8)
Once defined the interaction matrix A A A , for every x ∈ { 0 , 1 } n x\in\{0,1\}^n x ∈ { 0 , 1 } n and every i ∈ [ n ] i\in[n] i ∈ [ n ] , the firing probability ϕ G , i ( x ) \phi_{G,i}(x) ϕ G , i ( x ) can be rewritten as
ϕ G , i ( x ) = λ + ( A T x ) i . \phi_{G,i}(x)=\lambda+(A^\mathsf{T}x)_i. ϕ G , i ( x ) = λ + ( A T x ) i .
Note that, for (6) to be well defined, we should have that ϕ G , i ( x ) ∈ [ 0 , 1 ] \phi_{G,i}(x)\in[0,1] ϕ G , i ( x ) ∈ [ 0 , 1 ] for every i ∈ [ n ] i\in[n] i ∈ [ n ] and every x ∈ { 0 , 1 } n x\in\{0,1\}^n x ∈ { 0 , 1 } n . This induces some restriction on the model parameters λ \lambda λ , ( μ b ) b ∈ [ k ] (\mu_b)_{b\in[k]} ( μ b ) b ∈ [ k ] , μ ≁ \mu_{\not\sim} μ ∼ and the interaction graph G G G . A sufficient condition is:
∥ A T ∥ ∞ < λ ∧ ( 1 − λ ) , (9) \lVert A^\mathsf{T}\rVert_{\infty}<\lambda\wedge(1-\lambda), \tag*{(9)} ∥ A T ∥ ∞ < λ ∧ ( 1 − λ ) , (9)
where ∥ A T ∥ ∞ : = max i ∑ j ∈ [ n ] ∣ A i j T ∣ \lVert A^\mathsf{T}\rVert_{\infty}:=\max_{i}\sum_{j\in[n]}|A_{ij}^{\mathsf{T}}| ∥ A T ∥ ∞ := max i ∑ j ∈ [ n ] ∣ A ij T ∣ and, hereafter, u ∧ v u\wedge v u ∧ v (resp., u ∨ v u\vee v u ∨ v ) denotes the minimum (resp., maximum) value between the real numbers u u u and $v.
Note that ∥ A ⊤ ∥ ∞ = 1 n max b ∈ [ k ] μ b Δ b + μ ≁ Δ ≁ n \lVert A^{\top}\rVert_{\infty} = \frac{1}{n}\max_{b\in[k]}\mu_b\Delta_b+\mu_{\not\sim}\frac{\Delta_{\not\sim}}{n} ∥ A ⊤ ∥ ∞ = n 1 max b ∈ [ k ] μ b Δ b + μ ∼ n Δ ∼ , where Δ b = max i ∈ C b ∑ j : j ∼ i θ j i \Delta_b=\max_{i\in C_b}\sum_{j:j\sim i}\theta_{ji} Δ b = max i ∈ C b ∑ j : j ∼ i θ j i is the maximal intra degree within community C b C_b C b and Δ ≁ = max i ∑ j : j ≁ i θ j i \Delta_{\not\sim}=\max_i\sum_{j:j\not\sim i}\theta_{ji} Δ ∼ = max i ∑ j : j ∼ i θ j i is the maximal inter degree. Note also that, for (9) to be satisfied, it must necessarily be the case that ∥ A ⊤ ∥ ∞ < 1 / 2 \lVert A^{\top}\rVert_{\infty}<1/2 ∥ A ⊤ ∥ ∞ < 1/2 .
Throughout the paper we will work with a stationary version of the chain X X X , which in particular implies that P G ( X t = x ) = P G ( X 0 = x ) \mathbb{P}_{G}(X_t=x)=\mathbb{P}_{G}(X_0=x) P G ( X t = x ) = P G ( X 0 = x ) for all t t t and every x ∈ { 0 , 1 } n x\in\{0,1\}^{n} x ∈ { 0 , 1 } n , as well as E G [ X i , t X j , t ] = E G [ X i , 0 X j , 0 ] \mathbb{E}_{G}[X_{i,t}X_{j,t}]=\mathbb{E}_{G}[X_{i,0}X_{j,0}] E G [ X i , t X j , t ] = E G [ X i , 0 X j , 0 ] for all t t t and i , j ∈ [ n ] i,j\in[n] i , j ∈ [ n ] . In passing, let us point out that if (9) holds, then 0 < φ i , G ( x ) < 1 0<\varphi_{i,G}(x)<1 0 < φ i , G ( x ) < 1 for every i ∈ [ n ] i\in[n] i ∈ [ n ] and every x ∈ { 0 , 1 } n x\in\{0,1\}^{n} x ∈ { 0 , 1 } n . The latter implies that the Markov chain is irreducible and aperiodic which, in turn, assures the existence of a unique stationary measure.
General notation. Before concluding this section, let us introduce some general notation used in the sequel. We write 1 n \mathbf{1}_n 1 n to indicate the column vector of length n n n with all entries equal to 1. More generally, For any subset S ⊂ [ n ] S\subset[n] S ⊂ [ n ] , we denote 1 S \mathbf{1}_S 1 S the column vector of length n n n with entries in S S S equal to 1 and 0 otherwise. For a n n n -dimensional vector w w w , we denote ∥ w ∥ ∞ : = max i ∈ [ n ] ∣ w i ∣ \lVert w\rVert_{\infty}:=\max_{i\in[n]}|w_i| ∥ w ∥ ∞ := max i ∈ [ n ] ∣ w i ∣ to indicate its supremum norm. For a n 2 n^2 n 2 -dimensional column vector w w w , we will often write w i j w_{ij} w ij to indicate its entry w u w_u w u , where u = n ( j − 1 ) + i u=n(j-1)+i u = n ( j − 1 ) + i with i , j ∈ [ n ] i,j\in[n] i , j ∈ [ n ] . Given n n n -dimensional column vectors w w w and v v v , we write w ⊙ v w\odot v w ⊙ v to denote their Hadamard product. This is an n n n -dimensional vector whose entries ( w ⊙ v ) i : = w i v i (w\odot v)_i:=w_iv_i ( w ⊙ v ) i := w i v i , for all i ∈ [ n ] i\in[n] i ∈ [ n ] .
We denote I n I_n I n the n n n -dimensional identity matrix. Given an n n n -dimensional vector w w w , we denote by diag ( w ) \operatorname{diag}(w) diag ( w ) the n × n n\times n n × n diagonal matrix with diagonal entries given by diag ( w ) i i = w i \operatorname{diag}(w)_{ii}=w_i diag ( w ) ii = w i , i ∈ [ n ] i\in[n] i ∈ [ n ] . If w ˉ \bar{w} w ˉ is another n n n -dimensional vector, we denote w ( w ˉ ) ⊤ w(\bar{w})^{\top} w ( w ˉ ) ⊤ the n × n n\times n n × n matrix whose entries are given by w ( w ˉ ) i j ⊤ = w i w ˉ j w(\bar{w})^{\top}_{ij}=w_i\bar{w}_j w ( w ˉ ) ij ⊤ = w i w ˉ j , i , j ∈ [ n ] i,j\in[n] i , j ∈ [ n ] . Given an n × n n\times n n × n matrix B B B we denote by d 0 ( B ) d_0(B) d 0 ( B ) the matrix obtained by setting to zero all diagonal elements of B B B and keeping the off-diagonal elements unaltered. We denote by Diag ( B ) = B − d 0 ( B ) \operatorname{Diag}(B)=B-d_0(B) Diag ( B ) = B − d 0 ( B ) , i.e., the diagonal matrix obtained by setting to zero all off-diagonal elements of B B B and keeping the diagonal elements of B B B unaltered. We also denote by vec ( B ) \operatorname{vec}(B) vec ( B ) its vectorization, that is the n 2 n^2 n 2 -dimensional column vector obtained by stacking the columns of B B B one by one. Specifically, for u ∈ { 1 , … , n 2 } u\in\{1,\ldots,n^2\} u ∈ { 1 , … , n 2 } , vec ( B ) u = B i j \operatorname{vec}(B)_u=B_{ij} vec ( B ) u = B ij , with i , j ∈ [ n ] i,j\in[n] i , j ∈ [ n ] uniquely determined by u = n ( j − 1 ) + i u=n(j-1)+i u = n ( j − 1 ) + i . With slight abuse of notation, we shall often write vec ( B ) i j \operatorname{vec}(B)_{ij} vec ( B ) ij instead of vec ( B ) u \operatorname{vec}(B)_u vec ( B ) u to refer to the entry u = n ( j − 1 ) + i u=n(j-1)+i u = n ( j − 1 ) + i of the vector vec ( B ) \operatorname{vec}(B) vec ( B ) . Also, when B B B is such that I n − B I_n-B I n − B is invertible, we denote G ( B ) = ( I n − B ) − 1 \mathcal{G}(B)=(I_n-B)^{-1} G ( B ) = ( I n − B ) − 1 and G ≥ 1 ( B ) = ( I n − B ) − 1 − I n \mathcal{G}_{\geq1}(B)=(I_n-B)^{-1}-I_n G ≥ 1 ( B ) = ( I n − B ) − 1 − I n . If I n − B I_n-B I n − B is invertible it holds that G ( B ) = ∑ u = 0 ∞ B u \mathcal{G}(B)=\sum_{u=0}^{\infty}B^u G ( B ) = ∑ u = 0 ∞ B u and G ≥ 1 ( B ) = ∑ u = 1 ∞ B u \mathcal{G}_{\geq1}(B)=\sum_{u=1}^{\infty}B^u G ≥ 1 ( B ) = ∑ u = 1 ∞ B u (see, e.g., [8 ), Theorem 6.2.8] ). Moreover, given two n × n n\times n n × n matrices B B B and C C C , we denote by B ⊗ C B\otimes C B ⊗ C their Kronecker product, that is the n 2 × n 2 n^2\times n^2 n 2 × n 2 matrix whose entries are given by ( B ⊗ C ) u v = B i j C ℓ h (B\otimes C)_{uv}=B_{ij}C_{\ell h} ( B ⊗ C ) uv = B ij C ℓ h , where h , i , j , ℓ ∈ [ n ] h,i,j,\ell\in[n] h , i , j , ℓ ∈ [ n ] are uniquely determined by u = n ( j − 1 ) + i u=n(j-1)+i u = n ( j − 1 ) + i and v = n ( ℓ − 1 ) + h v=n(\ell-1)+h v = n ( ℓ − 1 ) + h . Similar as above, we may also write ( B ⊗ C ) i j , h ℓ (B\otimes C)_{ij,h\ell} ( B ⊗ C ) ij , h ℓ instead of ( B ⊗ C ) u v (B\otimes C)_{uv} ( B ⊗ C ) uv to refer to the entry ( u , v ) = ( n ( j − 1 ) + i , n ( ℓ − 1 ) + h ) (u,v)=(n(j-1)+i,n(\ell-1)+h) ( u , v ) = ( n ( j − 1 ) + i , n ( ℓ − 1 ) + h ) of the matrix B ⊗ C B\otimes C B ⊗ C . Let us recall for later use that ( B ⊗ C ) ( D ⊗ E ) = ( B D ⊗ C E ) (B\otimes C)(D\otimes E)=(BD\otimes CE) ( B ⊗ C ) ( D ⊗ E ) = ( B D ⊗ C E ) for all n × n n\times n n × n matrices B , C , D B,C,D B , C , D and E E E . Applying this identity iteratively we deduce that ( B ⊗ C ) u = ( B u ) ⊗ ( C u ) (B\otimes C)^u=(B^u)\otimes(C^u) ( B ⊗ C ) u = ( B u ) ⊗ ( C u ) for all u ≥ 0 u\geq0 u ≥ 0 . Recall also that the Kronecker product and the vectorization are related by the following property: given three matrices B , C , D B,C,D B , C , D it holds that vec ( B D C ) = ( C ⊤ ⊗ B ) vec ( D ) \operatorname{vec}(BDC)=(C^{\top}\otimes B)\operatorname{vec}(D) vec ( B D C ) = ( C ⊤ ⊗ B ) vec ( D ) . Finally, the max norm of a matrix B B B is denoted by ∥ B ∥ max : = max i , j ∣ B i j ∣ \lVert B\rVert_{\max}:=\max_{i,j}|B_{ij}| ∥ B ∥ m a x := max i , j ∣ B ij ∣ , whereas, as already mentioned, the sup norm ∥ B ∥ ∞ : = max i ∑ j ∈ [ n ] ∣ B i j ∣ \lVert B\rVert_{\infty}:=\max_i\sum_{j\in[n]}|B_{ij}| ∥ B ∥ ∞ := max i ∑ j ∈ [ n ] ∣ B ij ∣ .
Auxiliary results
In this section, we collect three auxiliary results that collectively show that for the Markov chain model defined in the previous section, first and second order statistics can be explicitly computed from the interaction matrix A A A , for every choice of the interaction graph G G G . Their proofs are presented in Appendix G.
In the first auxiliary result, we show that the vector of the mean firing activity E G [ X 0 ] \mathbb{E}_{G}[X_{0}] E G [ X 0 ] can be explicitly computed from the interaction matrix A A A and the parameter λ \lambda λ .
Proposition 2.1 (Firing mean). Let [ n ] [n] [ n ] be the set of neurons, C = { C 1 , … , C k } \mathcal{C} = \{C_{1}, \ldots, C_{k}\} C = { C 1 , … , C k } the k k k communities, G = ( C , θ , η ) G = (\mathcal{C}, \theta, \eta) G = ( C , θ , η ) the interaction graph and λ ∈ [ 0 , 1 ] \lambda\in[0,1] λ ∈ [ 0 , 1 ] the spontaneous activity. Assume that the corresponding interaction matrix A A A and λ \lambda λ satisfy (9). Then, the firing mean E G [ X 0 ] \mathbb{E}_{G}[X_{0}] E G [ X 0 ] is given by
E G [ X 0 ] = G ( A T ) λ 1 n . \mathbb{E}_{G}[X_{0}] = \mathcal{G}(A^{\mathsf{T}})\lambda\mathbf{1}_{n}. E G [ X 0 ] = G ( A T ) λ 1 n .
In what follows, we write Σ G ( t ) \Sigma_{G}^{(t)} Σ G ( t ) to denote the t t t -lagged firing covariance matrix that is defined as
Σ G ( t ) : = E G [ X t ( X 0 ) T ] − E G [ X t ] ( E G [ X 0 ] ) T , t ∈ N . (10) \Sigma_{G}^{(t)} := \mathbb{E}_{G}[X_{t}(X_{0})^{\mathsf{T}}] - \mathbb{E}_{G}[X_{t}](\mathbb{E}_{G}[X_{0}])^{\mathsf{T}}, \ t \in\mathbb{N}. \tag*{(10)} Σ G ( t ) := E G [ X t ( X 0 ) T ] − E G [ X t ] ( E G [ X 0 ] ) T , t ∈ N . (10)
Along the text, we often refer to the 0-lagged firing covariance matrix Σ G ( 0 ) \Sigma_{G}^{(0)} Σ G ( 0 ) as the synchronous firing covariance matrix.
The second auxiliary result tells us that the synchronous firing covariance matrix Σ G ( 0 ) \Sigma_{G}^{(0)} Σ G ( 0 ) can be computed explicitly from the interaction matrix A A A and the vector of firing variances, defined as
v G ( X 0 ) : = E G [ X 0 ] ⊙ ( 1 n − E G [ X 0 ] ) . (11) v_{G}(X_{0}) := \mathbb{E}_{G}[X_{0}] \odot(\mathbf{1}_{n} - \mathbb{E}_{G}[X_{0}]). \tag*{(11)} v G ( X 0 ) := E G [ X 0 ] ⊙ ( 1 n − E G [ X 0 ]) . (11)
Observe that the vector v G ( X 0 ) v_{G}(X_{0}) v G ( X 0 ) can be computed from Proposition 2.1.
Proposition 2.2. Assume that the interaction matrix A A A and the parameter λ \lambda λ satisfy (9). Then, the synchronous firing covariance matrix Σ G ( 0 ) \Sigma_{G}^{(0)} Σ G ( 0 ) can be written in a vectorized form as
vec ( Σ G ( 0 ) ) = G ( L ) vec ( diag ( v G ( X 0 ) ) ) . (12) \operatorname{vec}(\Sigma_{G}^{(0)}) = \mathcal{G}(L)\operatorname{vec}(\operatorname{diag}(v_{G}(X_{0}))). \tag*{(12)} vec ( Σ G ( 0 ) ) = G ( L ) vec ( diag ( v G ( X 0 ))) . (12) where v G ( X 0 ) v_{G}(X_{0}) v G ( X 0 ) is the vector of the firing variances defined in (11) and L L L is a n 2 × n 2 n^{2} \times n^{2} n 2 × n 2 matrix whose entries are defined as:
L u v = { 0 if u = n ( i − 1 ) + i , for i ∈ [ n ] , ( A T ⊗ A T ) u v otherwise . L_{uv} =
\begin{cases}
0 & \text{if } u = n(i-1) + i,\ \text{for } i \in[n],\\
(A^{\mathsf{T}} \otimes A^{\mathsf{T}})_{uv} & \text{otherwise}.
\end{cases} L uv = { 0 ( A T ⊗ A T ) uv if u = n ( i − 1 ) + i , for i ∈ [ n ] , otherwise .
Our final auxiliary result relates the synchronous firing covariance matrix with the t t t -lagged firing covariance matrices Σ G ( t ) \Sigma_{G}^{(t)} Σ G ( t ) . Specifically, we have the following.
Proposition 2.3 (Lagged firing covariances). For all t ≥ 1 t \ge1 t ≥ 1 , the t t t -lagged and the synchronous firing covariances are related by
Σ G ( t ) = ( A T ) t Σ G ( 0 ) . \Sigma_{G}^{(t)} = (A^{\mathsf{T}})^{t}\Sigma_{G}^{(0)}. Σ G ( t ) = ( A T ) t Σ G ( 0 ) .
As already anticipated at the beginning of the section, the results presented in Propositions 2.1, 2.2 and 2.3 allow us to compute explicitly the first and second order statistics of the Markov chain model defined through equations (6) and (7) from the interaction matrix A A A , for every choice of the interaction graph G G G . The main goal of the next sections is to provide approximations as n → ∞ n \to\infty n → ∞ for these first and second order statistics, when the interaction graph G G G is random and distributed as a Stochastic Block Model with k k k communities.
NFC on random environment
We begin this section by extending the NFC to a random environment, i.e., when the underlying interaction graph is random. Given [ n ] [n] [ n ] the set of neurons and C = { C 1 , … , C k } \mathcal{C} = \{C_1,\ldots,C_k\} C = { C 1 , … , C k } the k k k communities, the matrix encoding the pairwise interaction between neurons will be distributed similarly as a Stochastic Block Model (SBM) with the k k k communities [1 ] . We denote this matrix by Θ \Theta Θ (to differentiate from the constant case θ \theta θ in § 2.1 ). Specifically, this means that ( Θ i j ) i , j ∈ [ n ] (\Theta_{ij})_{i,j\in[n]} ( Θ ij ) i , j ∈ [ n ] are independent Bernoulli random variables with parameter ( p b ) b ∈ [ k ] (p_b)_{b\in[k]} ( p b ) b ∈ [ k ] or q q q depending on whether the corresponding edge is intra or inter, respectively. In particular, Θ i j ∼ Ber ( p b ) \Theta_{ij}\sim\operatorname{Ber}(p_b) Θ ij ∼ Ber ( p b ) if i ∼ b j i\overset{b}{\sim}j i ∼ b j for some b ∈ [ k ] b\in[k] b ∈ [ k ] and Θ i j ∼ Ber ( q ) \Theta_{ij}\sim\operatorname{Ber}(q) Θ ij ∼ Ber ( q ) if i ≁ j i\not\sim j i ∼ j . We also assume that the matrix governing whether an edge is inhibitory/excitatory, here denoted by H H H (to differentiate from the constant case η \eta η in § 2.1 ), has i.i.d. components, independent from Θ \Theta Θ , distributed as
H i j = { + 1 with probability β − 1 with probability ( 1 − β ) H_{ij} =
\begin{cases}
+1 & \text{with probability }\beta\\
-1 & \text{with probability }(1-\beta)
\end{cases} H ij = { + 1 − 1 with probability β with probability ( 1 − β )
where β ∈ [ 0 , 1 ] ∖ { 1 / 2 } \beta\in[0,1]\setminus\{1/2\} β ∈ [ 0 , 1 ] ∖ { 1/2 } . Recall that we also assume that the k k k communities have sizes proportional to the total number of neurons, that is, ∣ C b ∣ = α b n ≥ 1 |C_b|=\alpha_b n\ge1 ∣ C b ∣ = α b n ≥ 1 for every b ∈ [ k ] b\in[k] b ∈ [ k ] with α 1 + … + α k = 1 \alpha_1+\ldots+\alpha_k=1 α 1 + … + α k = 1 and α min : = min b ∈ [ k ] α b > 0 \alpha_{\min}:=\min_{b\in[k]}\alpha_b>0 α m i n := min b ∈ [ k ] α b > 0 . Thus, besides the parameters λ , ( μ b ) b ∈ [ k ] , μ ≁ , n \lambda,(\mu_b)_{b\in[k]},\mu_{\not\sim},n λ , ( μ b ) b ∈ [ k ] , μ ∼ , n , the NFC on this random environment also depends on p = ( p b ) b ∈ [ k ] , q , β p=(p_b)_{b\in[k]},q,\beta p = ( p b ) b ∈ [ k ] , q , β and ( α b ) b ∈ [ k ] (\alpha_b)_{b\in[k]} ( α b ) b ∈ [ k ] . Since in our setting, all parameters but n n n are kept fixed, we shall denote by P \mathbb{P} P the distribution of the interaction graph and by E ( n ) \mathbb{E}^{(n)} E ( n ) the corresponding expectation, to keep the notation lighter. Moreover, given θ ∈ { 0 , 1 } n × n \theta\in\{0,1\}^{n\times n} θ ∈ { 0 , 1 } n × n and η ∈ { − 1 , + 1 } n × n \eta\in\{-1,+1\}^{n\times n} η ∈ { − 1 , + 1 } n × n , we shall denote by P ( θ , η ) \mathbb{P}_{(\theta,\eta)} P ( θ , η ) the stationary distribution of the process X = ( X t ) t ≥ 0 X=(X_t)_{t\ge0} X = ( X t ) t ≥ 0 and by ϕ ( θ , η ) , i \phi_{(\theta,\eta),i} ϕ ( θ , η ) , i , with i ∈ [ n ] i\in[n] i ∈ [ n ] , the firing probability of i i i , conditioned on Θ = θ \Theta=\theta Θ = θ and H = η H=\eta H = η , i.e., conditioned on a realization of the interaction graph. We shall instead denote by P ( Θ , H ) \mathbb{P}_{(\Theta,H)} P ( Θ , H ) the (random) distribution of X t X_t X t , when we conditioned on the random matrices Θ \Theta Θ and H H H , rather than on specific realizations. The corresponding expectation will be denoted by E ( Θ , H ) \mathbb{E}_{(\Theta,H)} E ( Θ , H ) .
In this random scenario, the corresponding interaction matrix (see (8)) is a random matrix. We shall denote A ( θ , η ) A_{(\theta,\eta)} A ( θ , η ) to indicate the realization of this interaction matrix associated to the realization of the interaction graph Θ = θ \Theta=\theta Θ = θ and H = η H=\eta H = η . To avoid clutter, when the realization of the interaction graph is not fixed and there is no risk of confusion, we will write A A A to denote the corresponding random interaction matrix. We adopt a similar notation for the matrix L L L (see, Proposition 2.2) computed from the interaction matrix A A A .
Let Ω a d m \Omega^{adm} Ω a d m be the set of all realizations of the random environment for which (9) holds, i.e.,
Ω a d m = { ( θ , η ) ∈ { 0 , 1 } n × n × { − 1 , 1 } n × n : ∥ A ( θ , η ) T ∥ ∞ < λ ∧ ( 1 − λ ) } . (13) \Omega^{adm}=\left\{(\theta,\eta)\in\{0,1\}^{n\times n}\times\{-1,1\}^{n\times n}:\left\|A^T_{(\theta,\eta)}\right\|_\infty<\lambda\wedge(1-\lambda)\right\}. \tag*{(13)} Ω a d m = { ( θ , η ) ∈ { 0 , 1 } n × n × { − 1 , 1 } n × n : A ( θ , η ) T ∞ < λ ∧ ( 1 − λ ) } . (13)
Note that, on the event Ω a d m \Omega^{adm} Ω a d m , we have that ∥ A ( θ , η ) T ∥ ∞ < 1 / 2 \left\|A^T_{(\theta,\eta)}\right\|_\infty<1/2 A ( θ , η ) T ∞ < 1/2 . In particular, the model is well-defined for all pairs ( θ , η ) ∈ Ω a d m (\theta,\eta)\in\Omega^{adm} ( θ , η ) ∈ Ω a d m in the sense that for these pairs the firing probability ϕ ( θ , η ) , i ( x ) ∈ ( 0 , 1 ) \phi_{(\theta,\eta),i}(x)\in(0,1) ϕ ( θ , η ) , i ( x ) ∈ ( 0 , 1 ) for all x ∈ { 0 , 1 } n x\in\{0,1\}^n x ∈ { 0 , 1 } n and i ∈ [ n ] i\in[n] i ∈ [ n ] . In order to have a well-defined model also for the pairs ( θ , η ) ∉ Ω a d m (\theta,\eta)\notin\Omega^{adm} ( θ , η ) ∈ / Ω a d m , we consider for such pairs the truncated firing probability ( ϕ ( θ , η ) , i ( x ) ) + ∧ 1 (\phi_{(\theta,\eta),i}(x))^+\wedge1 ( ϕ ( θ , η ) , i ( x ) ) + ∧ 1 , where ( ⋅ ) + (\cdot)^+ ( ⋅ ) + denotes the positive part.
In what follows, we define
M : = max b ∈ [ k ] { μ b α b p b + μ ≁ ( 1 − α b ) q } . (14) M:=\max_{b\in[k]}\{\mu_b\alpha_b p_b+\mu_{\not\sim}(1-\alpha_b)q\}. \tag*{(14)} M := b ∈ [ k ] max { μ b α b p b + μ ∼ ( 1 − α b ) q } . (14)
and suppose that the model parameters λ \lambda λ , ( α b ) b ∈ [ k ] (\alpha_b)_{b\in[k]} ( α b ) b ∈ [ k ] , ( μ b ) b ∈ [ k ] (\mu_b)_{b\in[k]} ( μ b ) b ∈ [ k ] , μ ≁ \mu_{\not\sim} μ ∼ , ( p b ) b ∈ [ k ] (p_b)_{b\in[k]} ( p b ) b ∈ [ k ] and q q q are chosen in such a way that M M M satisfies
M < λ ∧ ( 1 − λ ) . (15) M < \lambda\wedge(1-\lambda). \tag*{(15)} M < λ ∧ ( 1 − λ ) . (15)
Under Condition (15), we can prove the following result.
Lemma 3.1. Suppose that (15) is satisfied. Let Ω a d m \Omega^{adm} Ω a d m be the set of admissible realizations defined in (13). Then, there exists a positive constant C C C depending on the parameters λ \lambda λ , ( α b ) b ∈ [ k ] (\alpha_b)_{b\in[k]} ( α b ) b ∈ [ k ] , ( μ b ) b ∈ [ k ] (\mu_b)_{b\in[k]} ( μ b ) b ∈ [ k ] , μ ≁ \mu_{\not\sim} μ ∼ , ( p b ) b ∈ [ k ] (p_b)_{b\in[k]} ( p b ) b ∈ [ k ] and q q q such that P ( n ) ( ( Θ , H ) ∈ Ω a d m ) ≥ 1 − 2 n e − C n \mathbb{P}^{(n)}((\Theta,H)\in\Omega^{adm})\ge1-2ne^{-Cn} P ( n ) (( Θ , H ) ∈ Ω a d m ) ≥ 1 − 2 n e − C n .
Lemma 3.1 ensures that, whenever Condition (15) holds, the random environment ( Θ , H ) (\Theta,H) ( Θ , H ) belongs to the set of admissible realizations Ω a d m \Omega^{adm} Ω a d m with probability converging to 1 1 1 exponentially fast as n → ∞ n\to\infty n → ∞ . Hence, whenever convenient, we restrict our analysis in what follows to environments lying in Ω a d m \Omega^{adm} Ω a d m . The proof of Lemma 3.1 is provided in Appendix E.
Main results
Before stating our first main result, let us introduce a couple of definitions. Let W ∈ R k × k W\in\mathbb{R}^{k\times k} W ∈ R k × k be the matrix with entries given by
W a b : = ( 2 β − 1 ) μ b α b p b 1 { a = b } + μ ≁ α b q 1 { b ≠ a } , a , b ∈ [ k ] . (16) W_{ab}:=(2\beta-1)\mu_b\alpha_b p_b\mathbf{1}_{\{a=b\}}+\mu_{\not\sim}\alpha_bq\mathbf{1}_{\{b\ne a\}},\qquad a,b\in[k]. \tag*{(16)} W ab := ( 2 β − 1 ) μ b α b p b 1 { a = b } + μ ∼ α b q 1 { b = a } , a , b ∈ [ k ] . (16)
It can be shown that, given i ∈ C a i\in C_a i ∈ C a , then W a b = E ( n ) [ ( A ⊤ 1 C b ) i ] W_{ab}=\mathbb{E}^{(n)}[(A^\top\mathbf{1}_{C_b})_i] W ab = E ( n ) [( A ⊤ 1 C b ) i ] , i.e., W a b W_{ab} W ab corresponds to the mean cumulative effect that community b b b exerts on neuron i i i belonging to community C a C_a C a . Note that, if (15) holds, then ∥ W ∥ ∞ = max a ∈ [ k ] ∑ b ∈ [ k ] ∣ W a b ∣ ≤ M < ( 1 − λ ) < 1 \lVert W\rVert_\infty=\max_{a\in[k]}\sum_{b\in[k]}|W_{ab}|\le M<(1-\lambda)<1 ∥ W ∥ ∞ = max a ∈ [ k ] ∑ b ∈ [ k ] ∣ W ab ∣ ≤ M < ( 1 − λ ) < 1 and the matrix ( I k − W ) (I_k-W) ( I k − W ) is invertible, as the spectral radius ρ ( W ) \rho(W) ρ ( W ) of the matrix W W W satisfies ρ ( W ) ≤ ∥ W ∥ ∞ \rho(W)\le\lVert W\rVert_\infty ρ ( W ) ≤ ∥ W ∥ ∞ . Thus, whenever (15) holds, we denote by ζ \zeta ζ the vector of length k k k defined as
ζ : = G ( W ) 1 k . (17) \zeta:=\mathcal{G}(W)\mathbf{1}_k. \tag*{(17)} ζ := G ( W ) 1 k . (17)
Given the k k k -dimensional vector ζ \zeta ζ , we then consider the n n n -dimensional vector m ‾ \overline{m} m defined in (2). Note that m ‾ i = λ ζ b \overline{m}_i=\lambda\zeta_b m i = λ ζ b for all entries i ∈ C b i\in C_b i ∈ C b . Using that M < ( 1 − λ ) M<(1-\lambda) M < ( 1 − λ ) , one can check that the vector m ‾ \overline{m} m satisfies ∥ m ‾ ∥ ∞ < 1 \lVert\overline{m}\rVert_\infty<1 ∥ m ∥ ∞ < 1 . As we will see in our first main result below, this vector can be seen as an approximation for the random vector of the firing mean E ( Θ , H ) [ X 0 ] \mathbb{E}_{(\Theta,H)}[X_0] E ( Θ , H ) [ X 0 ] , as the size of network n → ∞ n\to\infty n → ∞ . Moreover, this approximation holds with high probability and its error is of order log ( n ) / n \sqrt{\log(n)/n} log ( n ) / n .
Theorem 3.1. Suppose (15) is granted and let m ‾ \overline{m} m be the vector defined in (2). Then, for every δ ∈ ( 0 , 1 ] \delta\in(0,1] δ ∈ ( 0 , 1 ] and every n ≥ k n\ge k n ≥ k , it holds that
P ( n ) ( ∥ E ( Θ , H ) [ X 0 ] − m ‾ ∥ ∞ ≥ 2 k λ ∥ ζ ∥ ∞ R 2 α min n log ( 2 k n δ ) ) ≤ δ + 2 n e − C n , \mathbb{P}^{(n)}\left(\left\lVert\mathbb{E}_{(\Theta,H)}[X_0]-\overline{m}\right\rVert_\infty\ge2k\lambda\lVert\zeta\rVert_\infty R\sqrt{\frac{2\alpha_{\min}}{n}\log\left(\frac{2kn}{\delta}\right)}\right)\le\delta+2ne^{-Cn}, P ( n ) ( E ( Θ , H ) [ X 0 ] − m ∞ ≥ 2 k λ ∥ ζ ∥ ∞ R n 2 α m i n log ( δ 2 k n ) ) ≤ δ + 2 n e − C n , where R : = max b ∈ [ k ] ∣ μ b ∣ ∨ ∣ μ ≁ ∣ R:=\max_{b\in[k]}|\mu_b|\vee|\mu_{\not\sim}| R := max b ∈ [ k ] ∣ μ b ∣ ∨ ∣ μ ∼ ∣ and C C C is the positive constant given by Lemma 3.1.
The proof of Theorem 3.1 will be given in Appendix A.
Using Theorem 3.1, we can show that the random vector of the firing variances v Θ , H ( X 0 ) = E ( Θ , H ) [ X 0 ] ⊙ ( 1 n − E ( Θ , H ) [ X 0 ] ) v_{\Theta,H}(X_0)=\mathbb{E}_{(\Theta,H)}[X_0]\odot(\mathbf{1}_n-\mathbb{E}_{(\Theta,H)}[X_0]) v Θ , H ( X 0 ) = E ( Θ , H ) [ X 0 ] ⊙ ( 1 n − E ( Θ , H ) [ X 0 ]) can be approximated, with high probability, by the vector v ‾ \overline{v} v defined in (3), up to an error term that is of order log ( n ) / n \sqrt{\log(n)/n} log ( n ) / n . This is the content of the next result, which is also proved in Appendix A.
Corollary 3.1. Suppose (15) is granted and let v ˉ \bar{v} v ˉ be the vector defined in (3) . Then, for every δ ∈ ( 0 , 1 ] \delta\in(0,1] δ ∈ ( 0 , 1 ] and every n ≥ k n \ge k n ≥ k , it holds that
P ( n ) ( ∥ v ( Θ , H ) ( X 0 ) − v ˉ ∥ ∞ ≥ 2 k λ ∥ ζ ∥ ∞ R 2 α min n log ( 2 k n δ ) ) ≤ δ + 2 n e − C n , \mathbb{P}^{(n)}\left(\left\|v_{(\Theta,H)}(X_{0})-\bar{v}\right\|_{\infty} \ge2k\lambda\|\zeta\|_{\infty}R\sqrt{\frac{2\alpha_{\min}}{n}\log\left(\frac{2kn}{\delta}\right)}\right)\le\delta+2ne^{-Cn}, P ( n ) ( v ( Θ , H ) ( X 0 ) − v ˉ ∞ ≥ 2 k λ ∥ ζ ∥ ∞ R n 2 α m i n log ( δ 2 k n ) ) ≤ δ + 2 n e − C n , where R R R and C C C are positive constants defined in Theorem 3.1 and Lemma 3.1 respectively.
Next, we study the asymptotic behavior of the synchronous covariance matrix Σ ( Θ , H ) ( 0 ) \Sigma^{(0)}_{(\Theta,H)} Σ ( Θ , H ) ( 0 ) . With high probability, its diagonal terms are uniformly close to the vector v ˉ \bar{v} v ˉ , up to an error term of order log ( n ) / n \sqrt{\log(n)/n} log ( n ) / n . This follows from Corollary 3.1 and the fact that the diagonal of Σ ( Θ , H ) ( 0 ) \Sigma^{(0)}_{(\Theta,H)} Σ ( Θ , H ) ( 0 ) is given by the vector v ( Θ , H ) ( X 0 ) v_{(\Theta,H)}(X_{0}) v ( Θ , H ) ( X 0 ) . Hence, it remains to understand the asymptotic behavior of the off-diagonal entries of synchronous covariance matrix, that is, of the matrix d 0 ( Σ ( Θ , H ) ( 0 ) ) d_{0}(\Sigma^{(0)}_{(\Theta,H)}) d 0 ( Σ ( Θ , H ) ( 0 ) ) .
In what follows, let ξ \xi ξ be the k k k -dimensional vector
ξ = ( ξ 1 , … , ξ k ) , ξ b : = λ ζ b ( 1 − λ ζ b ) , b ∈ [ k ] , (18) \xi=(\xi_{1},\ldots,\xi_{k}),\qquad\xi_{b}:=\lambda\zeta_{b}(1-\lambda\zeta_{b}),\qquad b\in[k], \tag*{(18)} ξ = ( ξ 1 , … , ξ k ) , ξ b := λ ζ b ( 1 − λ ζ b ) , b ∈ [ k ] , (18)
where ζ \zeta ζ is the vector defined in (17) . Let also α − 1 : = ( α 1 − 1 , … , α k − 1 ) \alpha^{-1}:=(\alpha_{1}^{-1},\ldots,\alpha_{k}^{-1}) α − 1 := ( α 1 − 1 , … , α k − 1 ) . Given the vectors ξ \xi ξ and α − 1 \alpha^{-1} α − 1 , we then consider the k × k k\times k k × k matrix Ψ \Psi Ψ , defined as
Ψ : = ∑ u = 1 ∞ W u diag ( ξ ⊙ α − 1 ) ( W T ) u . (19) \Psi:=\sum_{u=1}^{\infty}W^{u}\operatorname{diag}(\xi\mathbin{\odot}\alpha^{-1})(W^{\mathsf{T}})^{u}. \tag*{(19)} Ψ := u = 1 ∑ ∞ W u diag ( ξ ⊙ α − 1 ) ( W T ) u . (19)
Recall that under Condition (15), we have ρ ( W ) ≤ ∥ W ∥ ∞ < 1 \rho(W)\le\left\lVert W\right\rVert_{\infty}<1 ρ ( W ) ≤ ∥ W ∥ ∞ < 1 , where ρ ( W ) \rho(W) ρ ( W ) denotes the spectral radius of the matrix W W W . Hence, Lemma H.1 ensures that the matrix Ψ \Psi Ψ is well-defined and is the unique solution of the Stein equation
Ψ − W Ψ W T = W diag ( ξ ⊙ α − 1 ) W T . (20) \Psi-W\Psi W^{\mathsf{T}}=W\operatorname{diag}(\xi\mathbin{\odot}\alpha^{-1})W^{\mathsf{T}}. \tag*{(20)} Ψ − W Ψ W T = W diag ( ξ ⊙ α − 1 ) W T . (20)
Note that the matrix Ψ \Psi Ψ is symmetric (this can be verified directly from (19) ).
Now, recall the deterministic matrix Σ ‾ ( 0 ) \overline{\Sigma}^{(0)} Σ ( 0 ) defined in (4) , whose entry Σ ‾ i j ( 0 ) \overline{\Sigma}^{(0)}_{ij} Σ ij ( 0 ) is given by n − 1 Ψ a b n^{-1}\Psi_{ab} n − 1 Ψ ab for all i ∈ C a i\in C_{a} i ∈ C a and j ∈ C b j\in C_{b} j ∈ C b . Note that since the matrix Ψ \Psi Ψ is symmetric, the matrix Σ ‾ ( 0 ) \overline{\Sigma}^{(0)} Σ ( 0 ) is also symmetric. In our next main result, we show that, with high probability, the entries of Σ ( Θ , H ) ( 0 ) \Sigma^{(0)}_{(\Theta,H)} Σ ( Θ , H ) ( 0 ) and Σ ‾ ( 0 ) \overline{\Sigma}^{(0)} Σ ( 0 ) become uniformly close as the size of the network n → ∞ n\to\infty n → ∞ .
Theorem 3.2. Suppose (15) is granted and let Σ ‾ ( 0 ) \overline{\Sigma}^{(0)} Σ ( 0 ) be the matrix defined in (4) . Then there exists a positive constant K K K independent of n n n such that for every δ ∈ ( 0 , 1 ] \delta\in(0,1] δ ∈ ( 0 , 1 ] and every n ≥ k n\ge k n ≥ k ,
P ( n ) ( ∥ d 0 ( Σ ( Θ , H ) ( 0 ) ) − d 0 ( Σ ‾ ( 0 ) ) ∥ max ≥ K log ( n / δ ) n 3 ) ≤ δ + 2 n e − C n , \mathbb{P}^{(n)}\left(\left\lVert d_{0}\left(\Sigma^{(0)}_{(\Theta,H)}\right)-d_{0}\left(\overline{\Sigma}^{(0)}\right)\right\rVert_{\max}\ge K\sqrt{\frac{\log(n/\delta)}{n^{3}}}\right)\le\delta+2ne^{-Cn}, P ( n ) ( d 0 ( Σ ( Θ , H ) ( 0 ) ) − d 0 ( Σ ( 0 ) ) m a x ≥ K n 3 log ( n / δ ) ) ≤ δ + 2 n e − C n , where C C C is the positive constant given by Lemma 3.1.
Note that, by Corollary 3.1, the diagonal entries of the synchronous covariance matrix are of order 1 1 1 asymptotically as n → ∞ n\to\infty n → ∞ , with high probability. As a consequence of Theorem 3.2, we see that its off-diagonal entries have a different order of magnitude, scaling as n − 1 n^{-1} n − 1 . Theorem 3.2 also tells us that the synchronous covariance matrix inherits the block structure present in the underlying random graph Θ \Theta Θ . The proof of Theorem 3.2 is presented in Appendix § 6 .
Finally, we focus on the asymptotic behavior of the t t t -lagged covariance matrices Σ ( Θ , H ) ( t ) \Sigma^{(t)}_{(\Theta,H)} Σ ( Θ , H ) ( t ) , for t ≥ 1 t \ge1 t ≥ 1 . To describe their asymptotic behavior, we consider the random matrix Σ ‾ ( Θ , H ) ( t ) \overline{\Sigma}^{(t)}_{(\Theta,H)} Σ ( Θ , H ) ( t ) defined in (5).
Theorem 3.3. Suppose (15) is granted and let Σ ‾ ( Θ , H ) ( t ) \overline{\Sigma}^{(t)}_{(\Theta,H)} Σ ( Θ , H ) ( t ) be the random matrix defined in (5), for every integer t ≥ 1 t \ge1 t ≥ 1 . Then, there exists a positive constant K K K independent of n n n and t t t , such that for every δ ∈ ( 0 , 1 ] \delta\in(0,1] δ ∈ ( 0 , 1 ] , every n ≥ k n \ge k n ≥ k and every integer t ≥ 1 t \ge1 t ≥ 1 ,
P ( n ) ( ∥ Σ ( Θ , H ) ( t ) − Σ ‾ ( Θ , H ) ( t ) ∥ max ≥ K log ( n / δ ) n 3 ) ≤ δ + 2 n e − C n , \mathbb{P}^{(n)}\left(\left\lVert\Sigma^{(t)}_{(\Theta,H)}-\overline{\Sigma}^{(t)}_{(\Theta,H)}\right\rVert_{\max} \ge K\sqrt{\frac{\log(n/\delta)}{n^3}}\right) \le\delta+2ne^{-Cn}, P ( n ) ( Σ ( Θ , H ) ( t ) − Σ ( Θ , H ) ( t ) m a x ≥ K n 3 log ( n / δ ) ) ≤ δ + 2 n e − C n , where C C C is the positive constant given by Lemma 3.1.
Unlike Theorems 3.1 and 3.2, note that in Theorem 3.3 the approximating matrix Σ ‾ ( Θ , H ) ( t ) \overline{\Sigma}^{(t)}_{(\Theta,H)} Σ ( Θ , H ) ( t ) is itself random, as it depends on the t t t -th power of the interaction matrix transpose A ( Θ , H ) T A^{\mathsf{T}}_{(\Theta,H)} A ( Θ , H ) T . This follows from the identity Σ ( Θ , H ) ( t ) = ( A ( Θ , H ) T ) t Σ ( Θ , H ) ( 0 ) \Sigma^{(t)}_{(\Theta,H)}=(A^{\mathsf{T}}_{(\Theta,H)})^t\Sigma^{(0)}_{(\Theta,H)} Σ ( Θ , H ) ( t ) = ( A ( Θ , H ) T ) t Σ ( Θ , H ) ( 0 ) (Proposition 2.3 ) and the fact that the diagonal and off-diagonal entries of the synchronous covariance matrix have different scales. To better understand this, write Σ ( Θ , H ) ( 0 ) = Diag ( v ( Θ , H ) ( X 0 ) ) + d 0 ( Σ ( Θ , H ) ( 0 ) ) \Sigma^{(0)}_{(\Theta,H)}=\operatorname{Diag}(v_{(\Theta,H)}(X_0))+d_0(\Sigma^{(0)}_{(\Theta,H)}) Σ ( Θ , H ) ( 0 ) = Diag ( v ( Θ , H ) ( X 0 )) + d 0 ( Σ ( Θ , H ) ( 0 ) ) and note that, on one hand, the off-diagonal part is of order n − 1 n^{-1} n − 1 by Theorem 3.2, so that ( A T ) t (A^{\mathsf{T}})^t ( A T ) t acting on it may be replaced by its mean at negligible cost. By doing so, we obtain the deterministic block matrix 1 n ∑ a , b ( W t Ψ ) a b 1 C a 1 C b T \frac{1}{n}\sum_{a,b}(W^t\Psi)_{ab}\mathbf{1}_{C_a}\mathbf{1}_{C_b}^{\mathsf{T}} n 1 ∑ a , b ( W t Ψ ) ab 1 C a 1 C b T . On the other hand, the diagonal part is of order 1 1 1 (see Corollary 3.1), and the same replacement in ( A T ) t diag ( v ( Θ , H ) ( X 0 ) ) (A^{\mathsf{T}})^t\operatorname{diag}(v_{(\Theta,H)}(X_0)) ( A T ) t diag ( v ( Θ , H ) ( X 0 )) would lead to an error of order log ( n ) / n \sqrt{\log(n)/n} log ( n ) / n , much larger than the log ( n ) / n 3 \sqrt{\log(n)/n^3} log ( n ) / n 3 rate of Theorem 3.3. The proof of Theorem 3.3 is given in Appendix § 7 .
Consequences for community identifiability
The results of Section 3.1 describe how the community structure of the interaction network is encoded in the first and second order statistics of the firing process, as the size of the network n → ∞ n \to\infty n → ∞ . In this section, we focus on the converse direction, describing how the communities can be identified from (the limit of) these statistics.While implementing community detection algorithms is beyond the scope of this work, these latter results provide the theoretical foundation for their future development within this framework.
First, we consider the limit of the mean firing vector E ( Θ , H ) [ X 0 ] \mathbb{E}_{(\Theta,H)}[X_0] E ( Θ , H ) [ X 0 ] given by the n n n -dimensional vector m ‾ \overline{m} m defined in (2). Recall that the vector m ‾ \overline{m} m is defined in terms of the k k k -dimensional vector ζ \zeta ζ given by (17). Clearly, in the case where all entries of ζ \zeta ζ are distinct, we can correctly identify the k k k communities of the model from the vector m ‾ \overline{m} m , up to permutation. On the other extreme, when all entries of ζ \zeta ζ are equal, the vector m ‾ \overline{m} m carries no information about the model communities. In the lemma below, we identify a necessary and sufficient condition, depending only on the parameters ( α b ) b ∈ [ k ] (\alpha_b)_{b\in[k]} ( α b ) b ∈ [ k ] , ( μ b ) b ∈ [ k ] (\mu_b)_{b\in[k]} ( μ b ) b ∈ [ k ] , ( p b ) b ∈ [ k ] (p_b)_{b\in[k]} ( p b ) b ∈ [ k ] , μ α \mu_{\alpha} μ α and q q q , under which all entries of ζ \zeta ζ have the same value. Its proof is presented in Appendix § 10 .
Lemma 3.2. Suppose condition (15) is granted and consider ζ = G ( W ) 1 k \zeta=\mathcal{G}(W)\mathbf{1}_k ζ = G ( W ) 1 k . Then, all entries of ζ \zeta ζ are equal if and only if all entries of W 1 k W\mathbf{1}_k W 1 k are equal. In this case, for b ∈ [ k ] b\in[k] b ∈ [ k ] , we have
ζ b = 1 1 − ( W 1 k ) 1 = 1 1 − ( 2 β − 1 ) ( μ 1 α 1 p 1 + μ α q ( 1 − α 1 ) ) . \zeta_b=\frac{1}{1-(W\mathbf{1}_k)_1}=\frac{1}{1-(2\beta-1)(\mu_1\alpha_1p_1+\mu_{\alpha}q(1-\alpha_1))}. ζ b = 1 − ( W 1 k ) 1 1 = 1 − ( 2 β − 1 ) ( μ 1 α 1 p 1 + μ α q ( 1 − α 1 )) 1 .
Remark 3.1. One can easily check that the vector W 1 k W1_k W 1 k has all entries equal to each other if and only if α b ( μ b p b − μ ≁ q ) = α c ( μ c p c − μ ≁ q ) \alpha_b(\mu_bp_b-\mu_{\nsim}q)=\alpha_c(\mu_cp_c-\mu_{\nsim}q) α b ( μ b p b − μ ≁ q ) = α c ( μ c p c − μ ≁ q ) for all b , c ∈ [ k ] b,c\in[k] b , c ∈ [ k ] . Moreover, if the communities are balanced (i.e., α b = 1 / k \alpha_b=1/k α b = 1/ k for all b ∈ [ k ] b\in[k] b ∈ [ k ] ) the latter condition reduces to μ b p b = μ a p a \mu_bp_b=\mu_ap_a μ b p b = μ a p a for all b ∈ [ k ] b\in[k] b ∈ [ k ] .
Lemma 3.2 ensures that when W 1 k W1_k W 1 k has some entries with different values, we certainly must find a pair ( b , c ) ∈ [ k ] × [ k ] (b,c)\in[k]\times[k] ( b , c ) ∈ [ k ] × [ k ] , with b ≠ c b\ne c b = c , for which the values of ζ b \zeta_b ζ b and ζ c \zeta_c ζ c are different. In this case, by studying the coordinates of m ˉ \bar{m} m ˉ we can then partition the set of neurons into at least two different classes. Note that Lemma 3.2 does not exclude the possibility of still finding entries of ζ \zeta ζ with equal value. Of course, the communities of these coordinates cannot be distinguished by the vector of means.
In the remaining of this section, we focus on the case in which the value of all entries of the vector ζ \zeta ζ is the same. According to Lemma 3.2 and Remark 3.1, the entries of ζ \zeta ζ have the same value if and only if the parameters ( α b ) b ∈ [ k ] (\alpha_b)_{b\in[k]} ( α b ) b ∈ [ k ] , ( μ b ) b ∈ [ k ] (\mu_b)_{b\in[k]} ( μ b ) b ∈ [ k ] , ( p b ) b ∈ [ k ] (p_b)_{b\in[k]} ( p b ) b ∈ [ k ] , μ ≁ \mu_{\nsim} μ ≁ and q q q satisfy the following condition:
α 1 ( μ 1 p 1 − μ ≁ q ) = … = α k ( μ k p k − μ ≁ q ) = : κ . (21) \alpha_1(\mu_1p_1-\mu_{\nsim}q)=\ldots=\alpha_k(\mu_kp_k-\mu_{\nsim}q)=:\kappa. \tag*{(21)} α 1 ( μ 1 p 1 − μ ≁ q ) = … = α k ( μ k p k − μ ≁ q ) =: κ . (21)
Hence, we shall assume that κ ≠ 0 \kappa\ne0 κ = 0 in what follows. As mentioned above, in such a case the vector of asymptotic firing means m ˉ \bar{m} m ˉ carries no information about the communities. Interestingly, going to second order statistics such as the synchronous firing covariance we still find enough information for distinguishing the communities, up to permutation. This is a consequence of the proposition below. Before stating it, let us introduce some notation. For every integer t ≥ 0 t\ge0 t ≥ 0 , define the matrices
S ˉ ( t ) : = ( 2 β − 1 ) t κ t γ n ∑ a ∈ [ k ] α a − 1 1 C a 1 C a T and B ˉ ( t ) : = ( 2 β − 1 ) t ( γ ( M t − κ t ) + ρ ≁ M t ) n 1 n 1 n T , (22) \bar{S}^{(t)}:=\frac{(2\beta-1)^t\kappa^t\gamma}{n}\sum_{a\in[k]}\alpha_a^{-1}\mathbf{1}_{C_a}\mathbf{1}_{C_a}^{\mathsf{T}}
\quad\text{and}\quad
\bar{B}^{(t)}:=\frac{(2\beta-1)^t\left(\gamma(M^t-\kappa^t)+\rho_{\nsim}M^t\right)}{n}\mathbf{1}_n\mathbf{1}_n^{\mathsf{T}}, \tag*{(22)} S ˉ ( t ) := n ( 2 β − 1 ) t κ t γ a ∈ [ k ] ∑ α a − 1 1 C a 1 C a T and B ˉ ( t ) := n ( 2 β − 1 ) t ( γ ( M t − κ t ) + ρ ≁ M t ) 1 n 1 n T , (22)
where the real values γ \gamma γ and ρ ≁ \rho_{\nsim} ρ ≁ are given by
γ : = ( ( 2 β − 1 ) κ ) 2 ( 1 − ( 2 β − 1 ) 2 κ 2 ) − 1 and ρ ≁ = γ ( ( M / κ ) 2 − 1 ) ( 1 − ( 2 β − 1 ) 2 M 2 ) − 1 , (23) \gamma:=\left((2\beta-1)\kappa\right)^2\left(1-(2\beta-1)^2\kappa^2\right)^{-1}
\quad\text{and}\quad
\rho_{\nsim}=\gamma\left((M/\kappa)^2-1\right)\left(1-(2\beta-1)^2M^2\right)^{-1}, \tag*{(23)} γ := ( ( 2 β − 1 ) κ ) 2 ( 1 − ( 2 β − 1 ) 2 κ 2 ) − 1 and ρ ≁ = γ ( ( M / κ ) 2 − 1 ) ( 1 − ( 2 β − 1 ) 2 M 2 ) − 1 , (23)
with M M M defined in (14). Note that the matrix S ˉ ( t ) \bar{S}^{(t)} S ˉ ( t ) has a community structure that resembles that of the underlying random graph Θ \Theta Θ , whereas the matrix B ˉ ( t ) \bar{B}^{(t)} B ˉ ( t ) have equal entries. Moreover, the entries of both matrices scale as n − 1 n^{-1} n − 1 .
Proposition 3.1. Suppose (15) and (21) are granted. For every integer t ≥ 0 t\ge0 t ≥ 0 , let S ˉ ( t ) \bar{S}^{(t)} S ˉ ( t ) and B ˉ ( t ) \bar{B}^{(t)} B ˉ ( t ) be the matrices defined in (22). Then, the matrix Σ ˉ ( 0 ) \bar{\Sigma}^{(0)} Σ ˉ ( 0 ) defined in (4) can be decomposed as
Σ ˉ ( 0 ) = ξ 1 ( S ˉ ( 0 ) + B ˉ ( 0 ) ) . \bar{\Sigma}^{(0)}=\xi_1\left(\bar{S}^{(0)}+\bar{B}^{(0)}\right). Σ ˉ ( 0 ) = ξ 1 ( S ˉ ( 0 ) + B ˉ ( 0 ) ) . with ξ 1 \xi_1 ξ 1 as defined in (18). Moreover, for every integer t ≥ 1 t\ge1 t ≥ 1 , the random matrix Σ ˉ Θ , H ( t ) \bar{\Sigma}^{(t)}_{\Theta,H} Σ ˉ Θ , H ( t ) defined in (5) can be decomposed as
Σ ˉ Θ , H ( t ) = ξ 1 ( ( A Θ , H T ) t + S ˉ ( t ) + B ˉ ( t ) ) . \bar{\Sigma}^{(t)}_{\Theta,H}=\xi_1\left((A_{\Theta,H}^{\mathsf{T}})^t+\bar{S}^{(t)}+\bar{B}^{(t)}\right). Σ ˉ Θ , H ( t ) = ξ 1 ( ( A Θ , H T ) t + S ˉ ( t ) + B ˉ ( t ) ) .
Proposition 3.1 is proved in Appendix F. Note that this proposition tells us that, under Condition (21), the synchronous firing covariance is (asymptotically as n → ∞ n\to\infty n → ∞ ) smaller between neurons in different communities (n − 1 ξ ρ ≁ n^{-1}\xi\rho_{\nsim} n − 1 ξ ρ ≁ ) than that between neurons in the same community C b C_b C b (n − 1 ξ ( γ α b − 1 + ρ ≁ ) n^{-1}\xi(\gamma\alpha_b^{-1}+\rho_{\nsim}) n − 1 ξ ( γ α b − 1 + ρ ≁ ) ) for all b ∈ [ k ] b\in[k] b ∈ [ k ] . As a consequence, the synchronous firing covariance carries enough information asymptotically for discriminating the communities C 1 , … , C k C_1,\ldots,C_k C 1 , … , C k up to permutation. If, additionally, the sizes of the communities α 1 , … , α k \alpha_1,\ldots,\alpha_k α 1 , … , α k are known, it also allows us to exactly find all the communities.
Proposition 3.1 also tells us that the t t t -lagged firing covariance matrix can be decomposed (modulo the factor ξ 1 \xi_1 ξ 1 ) into the sum of three matrices: the t t t -th power of the interaction matrix A T A^{\mathsf{T}} A T , the signal matrix S ‾ ( t ) \overline{S}^{(t)} S ( t ) and the bias matrix B ‾ ( t ) \overline{B}^{(t)} B ( t ) . Note that interaction matrix A T A^{\mathsf{T}} A T is random, whereas the matrices S ‾ ( t ) \overline{S}^{(t)} S ( t ) and B ‾ ( t ) \overline{B}^{(t)} B ( t ) are deterministic. Note also that, unlike the bias matrix B ‾ ( t ) \overline{B}^{(t)} B ( t ) , in which all entries are equal (ξ 1 n − 1 ( 2 β − 1 ) t ( γ ( M t − κ t ) + ρ ≁ M t ) \xi_1 n^{-1}(2\beta-1)^t(\gamma(M^t-\kappa^t)+\rho_{\not\sim}M^t) ξ 1 n − 1 ( 2 β − 1 ) t ( γ ( M t − κ t ) + ρ ∼ M t ) ), the signal matrix S ‾ ( t ) \overline{S}^{(t)} S ( t ) has a community structure that is inherited from the underlying random graph Θ \Theta Θ . Specifically, the value of the entry ( i , j ) ∈ C a × C b (i,j)\in C_a\times C_b ( i , j ) ∈ C a × C b of the matrix S ‾ ( t ) \overline{S}^{(t)} S ( t ) is ξ 1 n − 1 ( 2 β − 1 ) t κ t γ α a − 1 \xi_1 n^{-1}(2\beta-1)^t\kappa^t\gamma\alpha_a^{-1} ξ 1 n − 1 ( 2 β − 1 ) t κ t γ α a − 1 , when a = b a=b a = b ; whereas when a ≠ b a\ne b a = b , the value is 0 0 0 .
In classical community detection problems, the goal is to find the communities from a realization of the adjacency matrix of the underlying random graph model. In our framework, the adjacency matrix counterpart is the interaction matrix A T A^{\mathsf{T}} A T . Since this matrix is not observed in our setting, it cannot be directly used for finding the communities. As Theorem 3.3 and Proposition 3.1 show, the interaction matrix A T A^{\mathsf{T}} A T is closely related to the the one-lag matrix Σ ‾ ( Θ , H ) ( 1 ) \overline{\Sigma}^{(1)}_{(\Theta,H)} Σ ( Θ , H ) ( 1 ) , asymptotically as n → ∞ n\to\infty n → ∞ . Moreover, the one-lag covariance matrix inherits the community structure of the random graph Θ \Theta Θ not only through the interaction matrix A T A^{\mathsf{T}} A T but also through the signal matrix S ‾ ( 1 ) \overline{S}^{(1)} S ( 1 ) . This suggests that detecting the communities from the one-lag matrix Σ ‾ ( Θ , H ) \overline{\Sigma}_{(\Theta,H)} Σ ( Θ , H ) may be easier than when relying solely on A T A^{\mathsf{T}} A T . A rigorous proof of this conjecture is beyond the scope of the paper.
Acknowledgements
G.I.’s research was partially supported by FAPERJ (grant E-26/210.516/2024) and by the National Council for Scientific and Technological Development - CNPq (grant 300150/2026-1). G.O. was partially supported by the Serrapilheira Institute (grant number Serra – 2211-42049), FAPERJ (grant E-26/204.532/2024) and CNPq (grant 305591/2026-6).
Asymptotic approximation for the mean and variance
In this section, we prove Theorem 3.1 and Corollary 3.1. We start proving Theorem 3.1. In the proof, we rely on the following lemma which is proven in Appendix § 9 .
Lemma A.1. Suppose (15) is granted and let ζ \zeta ζ and m ‾ \overline{m} m be the vectors defined in (17) and (2), respectively. Then, on the event { ( Θ , H ) ∈ Ω a d m } \{(\Theta,H)\in\Omega^{adm}\} {( Θ , H ) ∈ Ω a d m } , it holds that
E ( Θ , H ) [ X 0 ] = m ‾ + λ ∑ b ∈ [ k ] ζ b G ( A T ) ϵ ( b ) , \mathbb{E}_{(\Theta,H)}[X_0]=\overline{m}+\lambda\sum_{b\in[k]}\zeta_b\mathcal{G}(A^{\mathsf{T}})\epsilon^{(b)}, E ( Θ , H ) [ X 0 ] = m + λ b ∈ [ k ] ∑ ζ b G ( A T ) ϵ ( b ) , where ϵ ( b ) : = ( A T − E p , q , β [ A T ] ) 1 C b \epsilon^{(b)}:=(A^{\mathsf{T}}-\mathbb{E}_{p,q,\beta}[A^{\mathsf{T}}])\mathbf{1}_{C_b} ϵ ( b ) := ( A T − E p , q , β [ A T ]) 1 C b for all b ∈ [ k ] b\in[k] b ∈ [ k ] .
By Lemma 3.1, the proof of Theorem 3.1 can be reduced to the case the event { ( Θ , H ) ∈ Ω a d m } \{(\Theta,H)\in\Omega^{adm}\} {( Θ , H ) ∈ Ω a d m } holds. Hence, given Lemma A.1, the main step in the proof of Theorem 3.1 consists in showing that the supremum norm of all random vectors ϵ ( b ) \epsilon^{(b)} ϵ ( b ) , b ∈ [ k ] b\in[k] b ∈ [ k ] is O ( log ( n / δ ) / n ) O(\sqrt{\log(n/\delta)/n}) O ( log ( n / δ ) / n ) with probability at least 1 − δ 1-\delta 1 − δ . Once this established, we then show that the same must hold for the vectors G ( A T ) ϵ ( b ) \mathcal{G}(A^{\mathsf{T}})\epsilon^{(b)} G ( A T ) ϵ ( b ) on Ω a d m \Omega^{adm} Ω a d m .
Proof on Theorem 3.1. Throughout the proof, we denote E = { ( Θ , H ) ∈ Ω a d m } \mathcal{E}=\{(\Theta,H)\in\Omega^{adm}\} E = {( Θ , H ) ∈ Ω a d m } . With this notation, observe that for all τ > 0 \tau>0 τ > 0 we can write,
P ( n ) ( ∥ E ( Θ , H ) − m ‾ ∥ ∞ ≥ τ ) ≤ P ( n ) ( E c ) + P ( n ) ( E ∩ { ∥ E ( Θ , H ) − m ‾ ∥ ∞ ≥ τ } ) . (24) \mathbb{P}^{(n)}\left(\left\|\mathbb{E}_{(\Theta,H)}-\overline{m}\right\|_{\infty}\geq\tau\right)\leq\mathbb{P}^{(n)}(\mathcal{E}^{c})+\mathbb{P}^{(n)}\left(\mathcal{E}\cap\left\{\left\|\mathbb{E}_{(\Theta,H)}-\overline{m}\right\|_{\infty}\geq\tau\right\}\right). \tag*{(24)} P ( n ) ( E ( Θ , H ) − m ∞ ≥ τ ) ≤ P ( n ) ( E c ) + P ( n ) ( E ∩ { E ( Θ , H ) − m ∞ ≥ τ } ) . (24)
By Lemma 3.1, the first term on the right-hand side of (24) is bounded from above by 2 n e − C n 2ne^{-Cn} 2 n e − C n . Hence, to conclude the proof, it suffices to show that the second term on the right of (24) is smaller than δ \delta δ , for τ = 2 k λ ∥ ζ ∥ ∞ R ( 2 α min ) / n log ( ( 2 k n ) / δ ) \tau=2k\lambda\|\zeta\|_{\infty}R\sqrt{(2\alpha_{\min})/n\log((2kn)/\delta)} τ = 2 k λ ∥ ζ ∥ ∞ R ( 2 α m i n ) / n log (( 2 k n ) / δ ) . This can be done as follows. By defining w i j = θ i j η i j − ( 2 β − 1 ) p b w_{ij}=\theta_{ij}\eta_{ij}-(2\beta-1)p_b w ij = θ ij η ij − ( 2 β − 1 ) p b if i ∼ b j i\overset{b}{\sim}j i ∼ b j for some b ∈ [ k ] b\in[k] b ∈ [ k ] , and w i j = θ i j η i j − ( 2 β − 1 ) q w_{ij}=\theta_{ij}\eta_{ij}-(2\beta-1)q w ij = θ ij η ij − ( 2 β − 1 ) q if i ≁ j i\not\sim j i ∼ j , we obtain that, for every i ∈ [ n ] i\in[n] i ∈ [ n ] ,
ϵ i ( b ) = ∑ j ∈ C b w j i n ( μ b 1 { i ∈ C b } + μ ≁ 1 { i ∉ C b } ) . \epsilon_i^{(b)}=\sum_{j\in C_b}\frac{w_{ji}}{n}\left(\mu_b\mathbf{1}_{\{i\in C_b\}}+\mu_{\not\sim}\mathbf{1}_{\{i\notin C_b\}}\right). ϵ i ( b ) = j ∈ C b ∑ n w j i ( μ b 1 { i ∈ C b } + μ ∼ 1 { i ∈ / C b } ) .
Note that, for every fixed i ∈ [ n ] i\in[n] i ∈ [ n ] , the random variables ( w j i ) j ∈ C b (w_{ji})_{j\in C_b} ( w j i ) j ∈ C b are independent, centered, and ∣ w j i ( μ b 1 { i ∈ C b } + μ ≁ 1 { i ∉ C b } ) ∣ ≤ 2 max a ∈ [ k ] ∣ μ a ∣ ∨ ∣ μ ≁ ∣ |w_{ji}(\mu_b\mathbf{1}_{\{i\in C_b\}}+\mu_{\not\sim}\mathbf{1}_{\{i\notin C_b\}})|\leq2\max_{a\in[k]}|\mu_a|\vee|\mu_{\not\sim}| ∣ w j i ( μ b 1 { i ∈ C b } + μ ∼ 1 { i ∈ / C b } ) ∣ ≤ 2 max a ∈ [ k ] ∣ μ a ∣ ∨ ∣ μ ∼ ∣ . Therefore, by union bound and Hoeffding’s inequality we obtain that, for all z > 0 z>0 z > 0 ,
P ( n ) ( max b ∈ [ k ] ∥ ϵ ( b ) ∥ ∞ > z ) ≤ ∑ b ∈ [ k ] P ( n ) ( ∥ ϵ ( b ) ∥ ∞ > z ) ≤ ∑ b ∈ [ k ] ∑ i ∈ [ n ] P ( n ) ( ∣ ϵ i ( b ) ∣ > z ) ≤ ∑ ℓ ∈ [ b ] ∑ i ∈ [ n ] P ( n ) ( ∣ ∑ j ∈ C b w j i ( μ ℓ 1 { i ∈ C b } + μ ≁ 1 { i ∉ C b } ) ∣ > n z ) ≤ ∑ b ∈ [ k ] ∑ i ∈ [ n ] 2 exp { − 2 z 2 n 2 4 R 2 ∣ C b ∣ } ≤ 2 n k exp { − z 2 n 2 R 2 α min } , \begin{aligned}
\mathbb{P}^{(n)}\left(\max_{b\in[k]}\|\epsilon^{(b)}\|_{\infty}>z\right)
&\leq\sum_{b\in[k]}\mathbb{P}^{(n)}\left(\|\epsilon^{(b)}\|_{\infty}>z\right)\\
&\leq\sum_{b\in[k]}\sum_{i\in[n]}\mathbb{P}^{(n)}\left(|\epsilon_i^{(b)}|>z\right)\\
&\leq\sum_{\ell\in[b]}\sum_{i\in[n]}\mathbb{P}^{(n)}\left(\left|\sum_{j\in C_b}w_{ji}\left(\mu_\ell\mathbf{1}_{\{i\in C_b\}}+\mu_{\not\sim}\mathbf{1}_{\{i\notin C_b\}}\right)\right|>nz\right)\\
&\leq\sum_{b\in[k]}\sum_{i\in[n]}2\exp\left\{-\frac{2z^2n^2}{4R^2|C_b|}\right\}
\leq2nk\exp\left\{-\frac{z^2n}{2R^2\alpha_{\min}}\right\},
\end{aligned} P ( n ) ( b ∈ [ k ] max ∥ ϵ ( b ) ∥ ∞ > z ) ≤ b ∈ [ k ] ∑ P ( n ) ( ∥ ϵ ( b ) ∥ ∞ > z ) ≤ b ∈ [ k ] ∑ i ∈ [ n ] ∑ P ( n ) ( ∣ ϵ i ( b ) ∣ > z ) ≤ ℓ ∈ [ b ] ∑ i ∈ [ n ] ∑ P ( n ) j ∈ C b ∑ w j i ( μ ℓ 1 { i ∈ C b } + μ ∼ 1 { i ∈ / C b } ) > n z ≤ b ∈ [ k ] ∑ i ∈ [ n ] ∑ 2 exp { − 4 R 2 ∣ C b ∣ 2 z 2 n 2 } ≤ 2 nk exp { − 2 R 2 α m i n z 2 n } ,
where R = max b ∈ [ k ] ∣ μ b ∣ ∨ ∣ μ ≁ ∣ R=\max_{b\in[k]}|\mu_b|\vee|\mu_{\not\sim}| R = max b ∈ [ k ] ∣ μ b ∣ ∨ ∣ μ ∼ ∣ and α min = min b ∈ [ k ] α b > 0 \alpha_{\min}=\min_{b\in[k]}\alpha_b>0 α m i n = min b ∈ [ k ] α b > 0 . In particular, for every δ ∈ ( 0 , 1 ] \delta\in(0,1] δ ∈ ( 0 , 1 ] , by taking z = R 2 α min n log ( 2 n k δ ) z=R\sqrt{\frac{2\alpha_{\min}}{n}\log\left(\frac{2nk}{\delta}\right)} z = R n 2 α m i n log ( δ 2 nk ) , we conclude that
P ( n ) ( max b ∈ [ k ] ∥ ϵ ( b ) ∥ ∞ ≤ R 2 α min n log ( 2 n k δ ) ) ≥ 1 − δ . (25) \mathbb{P}^{(n)}\left(\max_{b\in[k]}\|\epsilon^{(b)}\|_{\infty}\leq R\sqrt{\frac{2\alpha_{\min}}{n}\log\left(\frac{2nk}{\delta}\right)}\right)\geq1-\delta. \tag*{(25)} P ( n ) ( b ∈ [ k ] max ∥ ϵ ( b ) ∥ ∞ ≤ R n 2 α m i n log ( δ 2 nk ) ) ≥ 1 − δ . (25)
By Lemma A.1, on the event E \mathcal{E} E , we have that
E ( Θ , H ) [ X 0 ] − m ‾ = λ ∑ b ∈ [ k ] ζ b G ( A ⊤ ) ϵ ( b ) . \mathbb{E}_{(\Theta,H)}[X_0]-\overline{m}=\lambda\sum_{b\in[k]}\zeta_b\mathcal{G}(A^{\top})\epsilon^{(b)}. E ( Θ , H ) [ X 0 ] − m = λ b ∈ [ k ] ∑ ζ b G ( A ⊤ ) ϵ ( b ) .
Thus, by the triangle inequality, on the event E \mathcal{E} E , which assure that ∥ A ⊤ ∥ ∞ < 1 / 2 \|A^{\top}\|_{\infty}<1/2 ∥ A ⊤ ∥ ∞ < 1/2 , we obtain
∥ E ( Θ , H ) [ X 0 ] − m ‾ ∥ ∞ ≤ λ ∑ b ∈ [ k ] ∥ ζ b G ( A ⊤ ) ϵ ( b ) ∥ ∞ ≤ λ ∥ ζ ∥ ∞ ∥ ∑ b ∈ [ k ] G ( A ⊤ ) ϵ ( b ) ∥ ∞ ≤ λ ∥ ζ ∥ ∞ ∑ b ∈ [ k ] ∑ u = 0 ∞ ∥ ( A ⊤ ) u ϵ ( b ) ∥ ∞ ≤ λ ∥ ζ ∥ ∞ ∑ b ∈ [ k ] ∑ u = 0 ∞ ∥ ( A ⊤ ) u ∥ ∞ ∥ ϵ ( b ) ∥ ∞ ≤ λ ∥ ζ ∥ ∞ ∑ b ∈ [ k ] ∥ ϵ ( b ) ∥ ∞ ∑ u = 0 ∞ ∥ A ⊤ ∥ ∞ u ≤ 2 λ ∥ ζ ∥ ∞ k max b ∈ [ k ] ∥ ϵ ( b ) ∥ ∞ . (26) \begin{aligned}
\left\|\mathbb{E}_{(\Theta,H)}[X_0]-\overline{m}\right\|_{\infty}
&\leq\lambda\sum_{b\in[k]}\|\zeta_b\mathcal{G}(A^{\top})\epsilon^{(b)}\|_{\infty}
\leq\lambda\|\zeta\|_{\infty}\left\|\sum_{b\in[k]}\mathcal{G}(A^{\top})\epsilon^{(b)}\right\|_{\infty}\\
&\leq\lambda\|\zeta\|_{\infty}\sum_{b\in[k]}\sum_{u=0}^{\infty}\|(A^{\top})^u\epsilon^{(b)}\|_{\infty}
\leq\lambda\|\zeta\|_{\infty}\sum_{b\in[k]}\sum_{u=0}^{\infty}\|(A^{\top})^u\|_{\infty}\|\epsilon^{(b)}\|_{\infty}\\
&\leq\lambda\|\zeta\|_{\infty}\sum_{b\in[k]}\|\epsilon^{(b)}\|_{\infty}\sum_{u=0}^{\infty}\|A^{\top}\|_{\infty}^{u}
\leq2\lambda\|\zeta\|_{\infty}k\max_{b\in[k]}\|\epsilon^{(b)}\|_{\infty}. \tag*{(26)}
\end{aligned} E ( Θ , H ) [ X 0 ] − m ∞ ≤ λ b ∈ [ k ] ∑ ∥ ζ b G ( A ⊤ ) ϵ ( b ) ∥ ∞ ≤ λ ∥ ζ ∥ ∞ b ∈ [ k ] ∑ G ( A ⊤ ) ϵ ( b ) ∞ ≤ λ ∥ ζ ∥ ∞ b ∈ [ k ] ∑ u = 0 ∑ ∞ ∥ ( A ⊤ ) u ϵ ( b ) ∥ ∞ ≤ λ ∥ ζ ∥ ∞ b ∈ [ k ] ∑ u = 0 ∑ ∞ ∥ ( A ⊤ ) u ∥ ∞ ∥ ϵ ( b ) ∥ ∞ ≤ λ ∥ ζ ∥ ∞ b ∈ [ k ] ∑ ∥ ϵ ( b ) ∥ ∞ u = 0 ∑ ∞ ∥ A ⊤ ∥ ∞ u ≤ 2 λ ∥ ζ ∥ ∞ k b ∈ [ k ] max ∥ ϵ ( b ) ∥ ∞ . (26)
where in (*) we used the sub-multiplicativity of the matrix norm ∥ ⋅ ∥ ∞ \lVert\cdot\rVert_{\infty} ∥ ⋅ ∥ ∞ . Combining the above bound with (25) , we deduce that the second term on the right of (24) is smaller than δ \delta δ , for
τ = 2 k λ ∥ ζ ∥ ∞ R 2 α min n log ( 2 k n δ ) , \tau= 2k\lambda\lVert\zeta\rVert_{\infty}R\sqrt{\frac{2\alpha_{\min}}{n}\log\left(\frac{2kn}{\delta}\right)}, τ = 2 k λ ∥ ζ ∥ ∞ R n 2 α m i n log ( δ 2 k n ) ,
concluding the proof.
Given Theorem 3.1, the proof of Corollary 3.1 is straightforward.
Proof of Corollary 3.1. First, use that
∥ v ( Θ , H ) ( X 0 ) − m ‾ ⊙ ( 1 n − m ‾ ) ∥ ∞ = max i ∈ [ n ] ∣ ( E ( Θ , H ) [ X i , 0 ] − m ‾ i ) ( 1 − m ‾ i − E ( Θ , H ) [ X i , 0 ] ) ∣ ≤ ∥ E ( Θ , H ) [ X 0 ] − m ‾ ∥ ∞ , \lVert v_{(\Theta,H)}(X_0)-\overline{m}\odot(\mathbf{1}_n-\overline{m})\rVert_{\infty}
=\max_{i\in[n]}\left|(\mathbb{E}_{(\Theta,H)}[X_{i,0}]-\overline{m}_i)(1-\overline{m}_i-\mathbb{E}_{(\Theta,H)}[X_{i,0}])\right|
\leq\lVert\mathbb{E}_{(\Theta,H)}[X_0]-\overline{m}\rVert_{\infty}, ∥ v ( Θ , H ) ( X 0 ) − m ⊙ ( 1 n − m ) ∥ ∞ = i ∈ [ n ] max ( E ( Θ , H ) [ X i , 0 ] − m i ) ( 1 − m i − E ( Θ , H ) [ X i , 0 ]) ≤ ∥ E ( Θ , H ) [ X 0 ] − m ∥ ∞ ,
and then use Theorem 3.1 to conclude.
To conclude this section, we now prove Lemma A.1 . The proof relies on the so-called resolvent identity , which ensures that
B − 1 − C − 1 = B − 1 ( C − B ) C − 1 , (27) B^{-1}-C^{-1}=B^{-1}(C-B)C^{-1}, \tag*{(27)} B − 1 − C − 1 = B − 1 ( C − B ) C − 1 , (27)
for all invertible n × n n\times n n × n matrices B B B and C C C .
Proof of Lemma A.1 . First, observe that the matrix ( I n − A T ) (I_n-A^{\mathsf{T}}) ( I n − A T ) is invertible on Ω a d m \Omega^{adm} Ω a d m and that the matrix ( I n − E ( n ) [ A T ] ) (I_n-\mathbb{E}^{(n)}[A^{\mathsf{T}}]) ( I n − E ( n ) [ A T ]) is also invertible since (15) is granted. Applying the resolvent identity (27) with B = ( I n − A T ) B=(I_n-A^{\mathsf{T}}) B = ( I n − A T ) and C = ( I n − E ( n ) [ A T ] ) C=(I_n-\mathbb{E}^{(n)}[A^{\mathsf{T}}]) C = ( I n − E ( n ) [ A T ]) so that B − 1 = G ( A T ) B^{-1}=\mathcal{G}(A^{\mathsf{T}}) B − 1 = G ( A T ) , C − 1 = G ( E ( n ) [ A T ] ) C^{-1}=\mathcal{G}(\mathbb{E}^{(n)}[A^{\mathsf{T}}]) C − 1 = G ( E ( n ) [ A T ]) and C − B = A T − E ( n ) [ A T ] C-B=A^{\mathsf{T}}-\mathbb{E}^{(n)}[A^{\mathsf{T}}] C − B = A T − E ( n ) [ A T ] , we then deduce that on Ω a d m \Omega^{adm} Ω a d m ,
( G ( A T ) − G ( E ( n ) [ A T ] ) ) 1 n = G ( A T ) ( A T − E ( n ) [ A T ] ) G ( E ( n ) [ A T ] ) 1 n . (28) \left(\mathcal{G}(A^{\mathsf{T}})-\mathcal{G}(\mathbb{E}^{(n)}[A^{\mathsf{T}}])\right)\mathbf{1}_n
=\mathcal{G}(A^{\mathsf{T}})\left(A^{\mathsf{T}}-\mathbb{E}^{(n)}[A^{\mathsf{T}}]\right)\mathcal{G}(\mathbb{E}^{(n)}[A^{\mathsf{T}}])\mathbf{1}_n. \tag*{(28)} ( G ( A T ) − G ( E ( n ) [ A T ]) ) 1 n = G ( A T ) ( A T − E ( n ) [ A T ] ) G ( E ( n ) [ A T ]) 1 n . (28)
Claim. The matrix G ( E ( n ) [ A T ] ) \mathcal{G}(\mathbb{E}^{(n)}[A^{\mathsf{T}}]) G ( E ( n ) [ A T ]) satisfies
G ( E ( n ) [ A T ] ) 1 n = ∑ b ∈ [ k ] ζ b 1 C b . \mathcal{G}(\mathbb{E}^{(n)}[A^{\mathsf{T}}])\mathbf{1}_n
=\sum_{b\in[k]}\zeta_b\mathbf{1}_{C_b}. G ( E ( n ) [ A T ]) 1 n = b ∈ [ k ] ∑ ζ b 1 C b .
Combining (28) and the above Claim, we have that on Ω a d m \Omega^{adm} Ω a d m ,
G ( A T ) 1 n = ∑ b ∈ [ k ] ζ b 1 C b + G ( A T ) ( A T − E ( n ) [ A T ] ) ( ∑ b ∈ [ k ] ζ b 1 C b ) = ∑ b ∈ [ k ] ζ b 1 C b + ∑ b ∈ [ k ] ζ b G ( A T ) ( A T − E ( n ) [ A T ] ) 1 C b = ∑ b ∈ [ k ] ζ b 1 C b + ∑ b ∈ [ k ] ζ b G ( A T ) ϵ ( b ) . (29) \begin{aligned}
\mathcal{G}(A^{\mathsf{T}})\mathbf{1}_n
&=\sum_{b\in[k]}\zeta_b\mathbf{1}_{C_b}
+\mathcal{G}(A^{\mathsf{T}})\left(A^{\mathsf{T}}-\mathbb{E}^{(n)}[A^{\mathsf{T}}]\right)\left(\sum_{b\in[k]}\zeta_b\mathbf{1}_{C_b}\right)\\
&=\sum_{b\in[k]}\zeta_b\mathbf{1}_{C_b}
+\sum_{b\in[k]}\zeta_b\mathcal{G}(A^{\mathsf{T}})\left(A^{\mathsf{T}}-\mathbb{E}^{(n)}[A^{\mathsf{T}}]\right)\mathbf{1}_{C_b}\\
&=\sum_{b\in[k]}\zeta_b\mathbf{1}_{C_b}
+\sum_{b\in[k]}\zeta_b\mathcal{G}(A^{\mathsf{T}})\epsilon^{(b)}. \tag*{(29)}
\end{aligned} G ( A T ) 1 n = b ∈ [ k ] ∑ ζ b 1 C b + G ( A T ) ( A T − E ( n ) [ A T ] ) b ∈ [ k ] ∑ ζ b 1 C b = b ∈ [ k ] ∑ ζ b 1 C b + b ∈ [ k ] ∑ ζ b G ( A T ) ( A T − E ( n ) [ A T ] ) 1 C b = b ∈ [ k ] ∑ ζ b 1 C b + b ∈ [ k ] ∑ ζ b G ( A T ) ϵ ( b ) . (29)
The third equality follows directly from the definition of the “error” vectors ϵ ( b ) \epsilon^{(b)} ϵ ( b ) . By Proposition 2.1 , (2) and (29) , the proof follows. Hence, it only remains to prove the claim. This is done below.
Proof of the claim. First, observe that ( E ( n ) [ A T ] ) u 1 C a = ∑ c ∈ [ k ] ( W u ) c a 1 C c (\mathbb{E}^{(n)}[A^{\mathsf{T}}])^u\mathbf{1}_{C_a}=\sum_{c\in[k]}(W^u)_{ca}\mathbf{1}_{C_c} ( E ( n ) [ A T ] ) u 1 C a = ∑ c ∈ [ k ] ( W u ) c a 1 C c , for every u ≥ 1 u\geq1 u ≥ 1 and a ∈ [ k ] a\in[k] a ∈ [ k ] , by Lemma D.2. Next, sum over a ∈ [ k ] a\in[k] a ∈ [ k ] and use that ∑ a ∈ [ k ] 1 C a = 1 n \sum_{a\in[k]}\mathbf{1}_{C_a}=\mathbf{1}_n ∑ a ∈ [ k ] 1 C a = 1 n , to obtain for every integer u ≥ 1 u\geq1 u ≥ 1 ,
( E ( n ) [ A T ] ) u 1 n = ∑ c ∈ [ k ] ( W u 1 k ) c 1 C c . (\mathbb{E}^{(n)}[A^{\mathsf{T}}])^u\mathbf{1}_n
=\sum_{c\in[k]}(W^u\mathbf{1}_k)_c\mathbf{1}_{C_c}. ( E ( n ) [ A T ] ) u 1 n = c ∈ [ k ] ∑ ( W u 1 k ) c 1 C c .
Since this identity also holds for u = 0 u=0 u = 0 (both sides are equal to 1 n \mathbf{1}_n 1 n ), and ∑ u = 0 ∞ ( W u 1 k ) c = ζ c \sum_{u=0}^{\infty}(W^u\mathbf{1}_k)_c=\zeta_c ∑ u = 0 ∞ ( W u 1 k ) c = ζ c for all c ∈ [ k ] c\in[k] c ∈ [ k ] by (17), we deduce that
G ( E ( n ) [ A T ] ) 1 n = ∑ u = 0 ∞ ( E ( n ) [ A T ] ) u 1 n = ∑ u = 0 ∞ ∑ c ∈ [ k ] ( W u 1 k ) c 1 C c = ∑ c ∈ [ k ] ( ∑ u = 0 ∞ ( W u 1 k ) c ) 1 C c = ∑ c ∈ [ k ] ζ c 1 C c , \begin{aligned}
\mathcal{G}(\mathbb{E}^{(n)}[A^{\mathsf{T}}])\mathbf{1}_n=\sum_{u=0}^{\infty}(\mathbb{E}^{(n)}[A^{\mathsf{T}}])^u\mathbf{1}_n=\sum_{u=0}^{\infty}\sum_{c\in[k]}(W^u\mathbf{1}_k)_c\mathbf{1}_{C_c} \\
&=\sum_{c\in[k]}\left(\sum_{u=0}^{\infty}(W^u\mathbf{1}_k)_c\right)\mathbf{1}_{C_c}=\sum_{c\in[k]}\zeta_c\mathbf{1}_{C_c},
\end{aligned} G ( E ( n ) [ A T ]) 1 n = u = 0 ∑ ∞ ( E ( n ) [ A T ] ) u 1 n = u = 0 ∑ ∞ c ∈ [ k ] ∑ ( W u 1 k ) c 1 C c = c ∈ [ k ] ∑ ( u = 0 ∑ ∞ ( W u 1 k ) c ) 1 C c = c ∈ [ k ] ∑ ζ c 1 C c ,
concluding the proof.
Asymptotic approximation for the synchronous covariance matrix
This section is devoted to the proof of Theorem 3.2. Recall that the n 2 × n 2 n^2\times n^2 n 2 × n 2 matrix L L L is defined as
L u v = { 0 if u = n ( i − 1 ) + i , for i ∈ [ n ] , ( A T ⊗ A T ) u v otherwise , (30) L_{uv}=
\begin{cases}
0 & \text{if } u=n(i-1)+i,\ \text{for } i\in[n],\\
(A^{\mathsf{T}}\otimes A^{\mathsf{T}})_{uv} & \text{otherwise},
\end{cases} \tag*{(30)} L uv = { 0 ( A T ⊗ A T ) uv if u = n ( i − 1 ) + i , for i ∈ [ n ] , otherwise , (30)
or, equivalently,
L i j , h ℓ = 1 { i ≠ j } A ℓ j T A i h T = 1 { i ≠ j } A ℓ j A h i , for all i , j , ℓ , h ∈ [ n ] . L_{ij,h\ell}=\mathbb{1}_{\{i\ne j\}}A_{\ell j}^{\mathsf{T}}A_{ih}^{\mathsf{T}}=\mathbb{1}_{\{i\ne j\}}A_{\ell j}A_{hi},\qquad\text{for all }i,j,\ell,h\in[n]. L ij , h ℓ = 1 { i = j } A ℓ j T A ih T = 1 { i = j } A ℓ j A hi , for all i , j , ℓ , h ∈ [ n ] .
Hereafter, for a , b ∈ [ k ] a,b\in[k] a , b ∈ [ k ] , we denote
v ( a , b ) : = vec ( d 0 ( 1 C a 1 C b T ) ) . (31) v^{(a,b)}:=\operatorname{vec}\left(d_0(\mathbf{1}_{C_a}\mathbf{1}_{C_b}^{\mathsf{T}})\right). \tag*{(31)} v ( a , b ) := vec ( d 0 ( 1 C a 1 C b T ) ) . (31)
Note that v i j ( a , b ) = ( d 0 ( 1 C a 1 C b T ) ) i j = 1 { i ≠ j } ( 1 C a 1 C b T ) i j v_{ij}^{(a,b)}=(d_0(\mathbf{1}_{C_a}\mathbf{1}_{C_b}^{\mathsf{T}}))_{ij}=\mathbb{1}_{\{i\ne j\}}(\mathbf{1}_{C_a}\mathbf{1}_{C_b}^{\mathsf{T}})_{ij} v ij ( a , b ) = ( d 0 ( 1 C a 1 C b T ) ) ij = 1 { i = j } ( 1 C a 1 C b T ) ij , for all a , b ∈ [ k ] a,b\in[k] a , b ∈ [ k ] and i , j ∈ [ n ] i,j\in[n] i , j ∈ [ n ] . In the proof of Theorem 3.2, we will use the following result.
Proposition B.1. Suppose (15) is granted. Let v ‾ \overline{v} v be the vector defined in (3) and let Σ ‾ ( 0 ) \overline{\Sigma}^{(0)} Σ ( 0 ) and L L L be the matrices defined in (4) and (30), respectively. Then, for all n ≥ k n\ge k n ≥ k ,
G ≥ 1 ( E ( n ) [ L ] ) vec ( diag ( v ‾ ) ) = vec ( d 0 ( Σ ‾ ( 0 ) ) ) + 1 n 2 ∑ a ∈ [ k ] ∑ c ∈ [ k ] E a c v ( a , c ) , \mathcal{G}_{\ge1}(\mathbb{E}^{(n)}[L])\operatorname{vec}(\operatorname{diag}(\overline{v}))=\operatorname{vec}\left(d_0(\overline{\Sigma}^{(0)})\right)+\frac{1}{n^2}\sum_{a\in[k]}\sum_{c\in[k]}E_{ac}v^{(a,c)}, G ≥ 1 ( E ( n ) [ L ]) vec ( diag ( v )) = vec ( d 0 ( Σ ( 0 ) ) ) + n 2 1 a ∈ [ k ] ∑ c ∈ [ k ] ∑ E a c v ( a , c ) , where the k × k k\times k k × k matrix E = ( E a c ) a , c ∈ [ k ] E=(E_{ac})_{a,c\in[k]} E = ( E a c ) a , c ∈ [ k ] satisfies ∥ E ∥ max ≤ K \lVert E\rVert_{\max}\le K ∥ E ∥ m a x ≤ K , for some positive constant K K K which is independent of n n n .
Before proving Proposition B.1, we will present the proof of Theorem 3.2.
Proof of Theorem 3.2. We begin as in the proof of Theorem 3.1. Denote E = { ( Θ , H ) ∈ Ω a d m } \mathcal{E}=\{(\Theta,H)\in\Omega^{ad m}\} E = {( Θ , H ) ∈ Ω a d m } and note that for all τ > 0 \tau>0 τ > 0 we can write,
P ( n ) ( ∥ d 0 ( Σ ‾ ( Θ , H ) ( 0 ) ) − d 0 ( Σ ‾ ( 0 ) ) ∥ max ≥ τ ) ≤ P ( n ) ( E c ) + P ( n ) ( E ∩ { ∥ d 0 ( Σ ‾ ( Θ , H ) ( 0 ) ) − d 0 ( Σ ‾ ( 0 ) ) ∥ max ≥ τ } ) . (32) \mathbb{P}^{(n)}\left(\left\lVert d_0\left(\overline{\Sigma}^{(0)}_{(\Theta,H)}\right)-d_0\left(\overline{\Sigma}^{(0)}\right)\right\rVert_{\max}\ge\tau\right)\le\mathbb{P}^{(n)}(\mathcal{E}^{c})
+\mathbb{P}^{(n)}\left(\mathcal{E}\cap\left\{\left\lVert d_0\left(\overline{\Sigma}^{(0)}_{(\Theta,H)}\right)-d_0\left(\overline{\Sigma}^{(0)}\right)\right\rVert_{\max}\ge\tau\right\}\right). \tag*{(32)} P ( n ) ( d 0 ( Σ ( Θ , H ) ( 0 ) ) − d 0 ( Σ ( 0 ) ) m a x ≥ τ ) ≤ P ( n ) ( E c ) + P ( n ) ( E ∩ { d 0 ( Σ ( Θ , H ) ( 0 ) ) − d 0 ( Σ ( 0 ) ) m a x ≥ τ } ) . (32)
By Lemma 3.1, the first term on the right-hand side of (32) is bounded from above by 2 n e − C n 2ne^{-C n} 2 n e − C n . Therefore, to conclude the proof, it remains to show that the second term on the right of (32) is smaller than δ \delta δ for τ = K log ( n / δ ) n 3 \tau= K\sqrt{\frac{\log(n/\delta)}{n^3}} τ = K n 3 l o g ( n / δ ) , where K K K is a positive constant independent of n n n . This is done below.
In the rest of the proof, K 1 K_1 K 1 denotes a positive constant independent of n n n that can change from line to line. We start observing that from Proposition 2.2, on the event E \mathcal{E} E , we can write vec ( Σ ( Θ , H ) ( 0 ) ) = G ( L ) vec ( diag ( v Θ , H ( X 0 ) ) ) \operatorname{vec}(\Sigma^{(0)}_{(\Theta,H)})=\mathcal{G}(L)\operatorname{vec}(\operatorname{diag}(v_{\Theta,H}(X_0))) vec ( Σ ( Θ , H ) ( 0 ) ) = G ( L ) vec ( diag ( v Θ , H ( X 0 ))) . Moreover, using that
vec ( d 0 ( Σ ( Θ , H ) ( 0 ) ) ) = vec ( Σ ( Θ , H ) ( 0 ) − diag ( v Θ , H ( X 0 ) ) ) = vec ( Σ ( Θ , H ) ( 0 ) ) − vec ( diag ( v Θ , H ( X 0 ) ) ) , \operatorname{vec}\left(d_0\left(\Sigma^{(0)}_{(\Theta,H)}\right)\right)=\operatorname{vec}\left(\Sigma^{(0)}_{(\Theta,H)}-\operatorname{diag}(v_{\Theta,H}(X_0))\right)
=\operatorname{vec}\left(\Sigma^{(0)}_{(\Theta,H)}\right)-\operatorname{vec}\left(\operatorname{diag}(v_{\Theta,H}(X_0))\right), vec ( d 0 ( Σ ( Θ , H ) ( 0 ) ) ) = vec ( Σ ( Θ , H ) ( 0 ) − diag ( v Θ , H ( X 0 )) ) = vec ( Σ ( Θ , H ) ( 0 ) ) − vec ( diag ( v Θ , H ( X 0 )) ) ,
we obtain that, on the event E \mathcal{E} E ,
vec ( d 0 ( Σ ( Θ , H ) ( 0 ) ) ) = G ( L ) vec ( diag ( v Θ , H ( X 0 ) ) ) − vec ( diag ( v Θ , H ( X 0 ) ) ) = G ≥ 1 ( L ) vec ( diag ( v Θ , H ( X 0 ) ) ) . \operatorname{vec}\left(d_0\left(\Sigma^{(0)}_{(\Theta,H)}\right)\right)=\mathcal{G}(L)\operatorname{vec}\left(\operatorname{diag}(v_{\Theta,H}(X_0))\right)-\operatorname{vec}\left(\operatorname{diag}(v_{\Theta,H}(X_0))\right)
=\mathcal{G}_{\geq1}(L)\operatorname{vec}\left(\operatorname{diag}(v_{\Theta,H}(X_0))\right). vec ( d 0 ( Σ ( Θ , H ) ( 0 ) ) ) = G ( L ) vec ( diag ( v Θ , H ( X 0 )) ) − vec ( diag ( v Θ , H ( X 0 )) ) = G ≥ 1 ( L ) vec ( diag ( v Θ , H ( X 0 )) ) .
Next, using the definition of L L L , the definition of the vector v ˉ \bar v v ˉ (recall (3)) and the triangle inequality, we have that
∥ L ( vec ( diag ( v Θ , H ( X 0 ) ) ) − vec ( v ˉ ) ) ∥ ∞ = max i , j ∈ [ n 2 ] ∣ ∑ ( h , ℓ ) ∈ [ n 2 ] L i j , h ℓ ( vec ( diag ( v Θ , H ( X 0 ) ) ) − vec ( diag ( v ˉ ) ) ) h ℓ ∣ = max i , j ∈ [ n 2 ] ∣ 1 { i ≠ j } ∑ ℓ ∈ [ n 2 ] A i ℓ A ℓ j ( vec ( diag ( v Θ , H ( X 0 ) ) ) − vec ( diag ( v ˉ ) ) ) ℓ ℓ ∣ = max i , j ∈ [ n 2 ] ∣ 1 { i ≠ j } ∑ ℓ ∈ [ n 2 ] ∑ b ∈ [ k ] ∑ c ∈ C b A i ℓ A ℓ j ( E ( Θ , H ) [ X ℓ , 0 ] ( 1 − E ( Θ , H ) [ X ℓ , 0 ] ) − λ C b ( 1 − λ C b ) ) ∣ ≤ ∥ E ( Θ , H ) [ X 0 ] − m ˉ ∥ ∞ R n ∥ A T ∥ ∞ , \begin{aligned}
\left\|L\left(\operatorname{vec}\left(\operatorname{diag}(v_{\Theta,H}(X_0))\right)-\operatorname{vec}(\bar v)\right)\right\|_\infty
&= \max_{i,j\in[n^2]}\left|\sum_{(h,\ell)\in[n^2]}L_{ij,h\ell}\left(\operatorname{vec}\left(\operatorname{diag}(v_{\Theta,H}(X_0))\right)-\operatorname{vec}(\operatorname{diag}(\bar v))\right)_{h\ell}\right|\\
&= \max_{i,j\in[n^2]}\left|\mathbf{1}_{\{i\ne j\}}\sum_{\ell\in[n^2]}A_{i\ell}A_{\ell j}\left(\operatorname{vec}\left(\operatorname{diag}(v_{\Theta,H}(X_0))\right)-\operatorname{vec}(\operatorname{diag}(\bar v))\right)_{\ell\ell}\right|\\
&= \max_{i,j\in[n^2]}\left|\mathbf{1}_{\{i\ne j\}}\sum_{\ell\in[n^2]}\sum_{b\in[k]}\sum_{c\in\mathcal{C}_b}A_{i\ell}A_{\ell j}\left(\mathbb{E}_{(\Theta,H)}[X_{\ell,0}](1-\mathbb{E}_{(\Theta,H)}[X_{\ell,0}])-\lambda_{\mathcal{C}_b}(1-\lambda_{\mathcal{C}_b})\right)\right|\\
&\leq\left\|\mathbb{E}_{(\Theta,H)}[X_0]-\bar m\right\|_\infty\frac{R}{n}\|A^\mathsf{T}\|_\infty,
\end{aligned} ∥ L ( vec ( diag ( v Θ , H ( X 0 )) ) − vec ( v ˉ ) ) ∥ ∞ = i , j ∈ [ n 2 ] max ( h , ℓ ) ∈ [ n 2 ] ∑ L ij , h ℓ ( vec ( diag ( v Θ , H ( X 0 )) ) − vec ( diag ( v ˉ )) ) h ℓ = i , j ∈ [ n 2 ] max 1 { i = j } ℓ ∈ [ n 2 ] ∑ A i ℓ A ℓ j ( vec ( diag ( v Θ , H ( X 0 )) ) − vec ( diag ( v ˉ )) ) ℓℓ = i , j ∈ [ n 2 ] max 1 { i = j } ℓ ∈ [ n 2 ] ∑ b ∈ [ k ] ∑ c ∈ C b ∑ A i ℓ A ℓ j ( E ( Θ , H ) [ X ℓ , 0 ] ( 1 − E ( Θ , H ) [ X ℓ , 0 ]) − λ C b ( 1 − λ C b ) ) ≤ E ( Θ , H ) [ X 0 ] − m ˉ ∞ n R ∥ A T ∥ ∞ ,
where R = max b ∈ [ k ] ∣ μ b ∣ ∨ ∣ μ b ′ ∣ R=\max_{b\in[k]}|\mu_b|\vee|\mu_{b'}| R = max b ∈ [ k ] ∣ μ b ∣ ∨ ∣ μ b ′ ∣ . Combining the preceding inequality with inequalities (26) and (25), we obtain that for every δ ∈ ( 0 , 1 ] \delta\in(0,1] δ ∈ ( 0 , 1 ] ,
P ( n ) ( E ∩ { ∥ L ( vec ( diag ( v Θ , H ( X 0 ) ) ) − vec ( v ˉ ) ) ∥ ∞ ≤ K 1 n − 3 log ( n / δ ) } ) ≥ 1 − δ . (33) \mathbb{P}^{(n)}\left(\mathcal{E}\cap\left\{\left\|L\left(\operatorname{vec}\left(\operatorname{diag}(v_{\Theta,H}(X_0))\right)-\operatorname{vec}(\bar v)\right)\right\|_\infty\leq K_1\sqrt{n^{-3}\log(n/\delta)}\right\}\right)\geq1-\delta. \tag*{(33)} P ( n ) ( E ∩ { ∥ L ( vec ( diag ( v Θ , H ( X 0 )) ) − vec ( v ˉ ) ) ∥ ∞ ≤ K 1 n − 3 log ( n / δ ) } ) ≥ 1 − δ . (33)
Now, since ∥ L ∥ ∞ ≤ ( ∥ A T ∥ ∞ ) 2 \|L\|_\infty\leq\left(\|A^\mathsf{T}\|_\infty\right)^2 ∥ L ∥ ∞ ≤ ( ∥ A T ∥ ∞ ) 2 and ∥ A T ∥ ∞ ≤ 1 / 2 \|A^\mathsf{T}\|_\infty\leq1/2 ∥ A T ∥ ∞ ≤ 1/2 on the event E \mathcal{E} E , we also have that
∥ G ≥ 1 ( L ) vec ( diag ( v Θ , H ( X 0 ) ) ) − G ≥ 1 ( L ) vec ( v ˉ ) ∥ ∞ ≤ ∥ ∑ u = 0 ∞ L u ∥ ∞ × \left\|\mathcal{G}_{\geq1}(L)\operatorname{vec}\left(\operatorname{diag}(v_{\Theta,H}(X_0))\right)-\mathcal{G}_{\geq1}(L)\operatorname{vec}(\bar v)\right\|_\infty\leq\left\|\sum_{u=0}^{\infty}L^u\right\|_\infty\times ∥ G ≥ 1 ( L ) vec ( diag ( v Θ , H ( X 0 )) ) − G ≥ 1 ( L ) vec ( v ˉ ) ∥ ∞ ≤ u = 0 ∑ ∞ L u ∞ ×
∥ L ( vec ( diag ( v Θ , H ( X 0 ) ) ) − vec ( v ˉ ) ) ∥ ∞ ≤ 4 3 ∥ L ( vec ( diag ( v Θ , H ( X 0 ) ) ) − vec ( v ˉ ) ) ∥ ∞ . \left\|L\left(\operatorname{vec}\left(\operatorname{diag}(v_{\Theta,H}(X_0))\right)-\operatorname{vec}(\bar v)\right)\right\|_\infty\leq\frac{4}{3}\left\|L\left(\operatorname{vec}\left(\operatorname{diag}(v_{\Theta,H}(X_0))\right)-\operatorname{vec}(\bar v)\right)\right\|_\infty. ∥ L ( vec ( diag ( v Θ , H ( X 0 )) ) − vec ( v ˉ ) ) ∥ ∞ ≤ 3 4 ∥ L ( vec ( diag ( v Θ , H ( X 0 )) ) − vec ( v ˉ ) ) ∥ ∞ .
Combining the above inequality with (33), it then follows that
P ( n ) ( E ∩ { ∥ G ≥ 1 ( L ) ( vec ( diag ( v Θ , H ( X 0 ) ) ) − vec ( v ˉ ) ) ∥ ∞ ≤ K 1 n − 3 log ( n / δ ) } ) ≥ 1 − δ . (34) \mathbb{P}^{(n)}\left(\mathcal{E}\cap\left\{\left\|\mathcal{G}_{\geq1}(L)\left(\operatorname{vec}\left(\operatorname{diag}(v_{\Theta,H}(X_0))\right)-\operatorname{vec}(\bar v)\right)\right\|_\infty\leq K_1\sqrt{n^{-3}\log(n/\delta)}\right\}\right)\geq1-\delta. \tag*{(34)} P ( n ) ( E ∩ { ∥ G ≥ 1 ( L ) ( vec ( diag ( v Θ , H ( X 0 )) ) − vec ( v ˉ ) ) ∥ ∞ ≤ K 1 n − 3 log ( n / δ ) } ) ≥ 1 − δ . (34)
Having established (34), we now conclude. First, observe that by applying the resolvent identity with B = I n 2 − L B=I_{n^2}-L B = I n 2 − L and C = I n 2 − E ( n ) [ L ] C=I_{n^2}-\mathbb{E}^{(n)}[L] C = I n 2 − E ( n ) [ L ] , we have
G ≥ 1 ( L ) vec ( diag ( v ‾ ) ) = G ≥ 1 ( E ( n ) [ L ] ) vec ( diag ( v ‾ ) ) + G ( L ) ( L − E ( n ) ( L ) ) G ( E ( n ) [ L ] ) vec ( diag ( v ‾ ) ) . \mathcal{G}_{\geq1}(L)\operatorname{vec}(\operatorname{diag}(\overline{v}))=\mathcal{G}_{\geq1}(\mathbb{E}^{(n)}[L])\operatorname{vec}(\operatorname{diag}(\overline{v}))+\mathcal{G}(L)(L-\mathbb{E}^{(n)}(L))\mathcal{G}(\mathbb{E}^{(n)}[L])\operatorname{vec}(\operatorname{diag}(\overline{v})). G ≥ 1 ( L ) vec ( diag ( v )) = G ≥ 1 ( E ( n ) [ L ]) vec ( diag ( v )) + G ( L ) ( L − E ( n ) ( L )) G ( E ( n ) [ L ]) vec ( diag ( v )) .
Hence, from above identity, (34) and Proposition B.1 it suffices to show that for every δ ∈ ( 0 , 1 ] \delta\in(0,1] δ ∈ ( 0 , 1 ] , on the event E \mathcal{E} E
∥ 1 n 2 ∑ a , c ∈ [ k ] E a c v ( a , c ) + G ( L ) ( L − E ( n ) ( L ) ) ( vec ( diag ( v ‾ ) ) + 1 n ∑ a , c ∈ [ k ] Ψ a c v ( a , c ) ) + 1 n 2 ∑ a , c ∈ [ k ] E a c v ( a , c ) ∥ ∞ ≤ K 1 log ( n / δ ) n 3 , \left\|\frac{1}{n^2}\sum_{a,c\in[k]}E_{ac}v^{(a,c)}+\mathcal{G}(L)(L-\mathbb{E}^{(n)}(L))\left(\operatorname{vec}(\operatorname{diag}(\overline{v}))+\frac{1}{n}\sum_{a,c\in[k]}\Psi_{ac}v^{(a,c)}\right)+\frac{1}{n^2}\sum_{a,c\in[k]}E_{ac}v^{(a,c)}\right\|_{\infty}
\leq K_1\sqrt{\frac{\log(n/\delta)}{n^3}}, n 2 1 a , c ∈ [ k ] ∑ E a c v ( a , c ) + G ( L ) ( L − E ( n ) ( L )) vec ( diag ( v )) + n 1 a , c ∈ [ k ] ∑ Ψ a c v ( a , c ) + n 2 1 a , c ∈ [ k ] ∑ E a c v ( a , c ) ∞ ≤ K 1 n 3 log ( n / δ ) ,
with probability at least 1 − δ 1-\delta 1 − δ . Recalling that ∥ ∥ ∥ E ∥ ∥ ∥ max ≤ K 1 \lVert\!\lVert\!\lVert E\rVert\!\rVert\!\rVert_{\max}\leq K_1 ∥ ∥ ∥ E ∥ ∥ ∥ m a x ≤ K 1 the first term on the RHS above can be absorbed into the error term. Hence, to conclude the proof it suffices to show that for every δ ∈ ( 0 , 1 ] \delta\in(0,1] δ ∈ ( 0 , 1 ] ,
P ( n ) ( E ∩ { max a , b ∈ [ k ] ∥ G ( L ) ϵ ( a , b ) ∥ ∞ ≤ K 1 log ( n / δ ) n } ) ≥ 1 − δ , (35) \mathbb{P}^{(n)}\left(\mathcal{E}\cap\left\{\max_{a,b\in[k]}\left\|\mathcal{G}(L)\epsilon^{(a,b)}\right\|_{\infty}\leq K_1\sqrt{\frac{\log(n/\delta)}{n}}\right\}\right)\geq1-\delta, \tag*{(35)} P ( n ) ( E ∩ { a , b ∈ [ k ] max G ( L ) ϵ ( a , b ) ∞ ≤ K 1 n log ( n / δ ) } ) ≥ 1 − δ , (35)
where ϵ ( a , b ) = ( L − E ( n ) [ L ] ) v ( a , b ) \epsilon^{(a,b)}=(L-\mathbb{E}^{(n)}[L])v^{(a,b)} ϵ ( a , b ) = ( L − E ( n ) [ L ]) v ( a , b ) with v ( a , b ) v^{(a,b)} v ( a , b ) as defined in (31) and that
P ( n ) ( E ∩ { ∥ G ( L ) ( L − E ( n ) ( L ) ) vec ( diag ( v ‾ ) ) ∥ ∞ ≤ K 1 log ( n / δ ) n 3 } ) ≥ 1 − δ . (36) \mathbb{P}^{(n)}\left(\mathcal{E}\cap\left\{\left\|\mathcal{G}(L)(L-\mathbb{E}^{(n)}(L))\operatorname{vec}(\operatorname{diag}(\overline{v}))\right\|_{\infty}\leq K_1\sqrt{\frac{\log(n/\delta)}{n^3}}\right\}\right)\geq1-\delta. \tag*{(36)} P ( n ) ( E ∩ { G ( L ) ( L − E ( n ) ( L )) vec ( diag ( v )) ∞ ≤ K 1 n 3 log ( n / δ ) } ) ≥ 1 − δ . (36)
To see that (35) holds, we use Hoeffding’s inequality and argue similarly as in the proof of Theorem 3.1, to first show that
max a , b ∈ [ k ] ∥ ϵ ( a , b ) ∥ ∞ ≤ K 1 log ( n / δ ) n . \max_{a,b\in[k]}\left\|\epsilon^{(a,b)}\right\|_{\infty}\leq K_1\sqrt{\frac{\log(n/\delta)}{n}}. a , b ∈ [ k ] max ϵ ( a , b ) ∞ ≤ K 1 n log ( n / δ ) .
Then combine the above inequality with the fact that, on the event E \mathcal{E} E , it holds that ∥ G ( L ) ∥ ∞ ≤ ∑ u = 0 ∞ ( ∥ L ∥ ∞ ) u ≤ 4 / 3 \lVert\mathcal{G}(L)\rVert_{\infty}\leq\sum_{u=0}^{\infty}(\lVert L\rVert_{\infty})^u\leq4/3 ∥ G ( L ) ∥ ∞ ≤ ∑ u = 0 ∞ (∥ L ∥ ∞ ) u ≤ 4/3 .
As for the inequality (36), we argue as follows. First, observe that
( ( L − E ( n ) ( L ) ) vec ( diag ( v ‾ ) ) ) i j = 1 { i ≠ j } ∑ a ∈ [ k ] ξ a ∑ ℓ ∈ C a ( A ℓ i A ℓ j − E ( n ) [ A ℓ i A ℓ j ] ) = 1 { i ≠ j } ∑ a ∈ [ k ] ξ a n − 2 ∑ ℓ ∈ C a ( Θ ℓ i H ℓ i Θ ℓ j H ℓ j − E ( n ) [ Θ ℓ i H ℓ i Θ ℓ j H ℓ j ] ) . \begin{aligned}
\left((L-\mathbb{E}^{(n)}(L))\operatorname{vec}(\operatorname{diag}(\overline{v}))\right)_{ij}
=\mathbf{1}_{\{i\neq j\}}\sum_{a\in[k]}\xi_a\sum_{\ell\in C_a}\left(A_{\ell i}A_{\ell j}-\mathbb{E}^{(n)}[A_{\ell i}A_{\ell j}]\right) \\
&=\mathbf{1}_{\{i\neq j\}}\sum_{a\in[k]}\xi_a n^{-2}\sum_{\ell\in C_a}\left(\Theta_{\ell i}H_{\ell i}\Theta_{\ell j}H_{\ell j}-\mathbb{E}^{(n)}[\Theta_{\ell i}H_{\ell i}\Theta_{\ell j}H_{\ell j}]\right).
\end{aligned} ( ( L − E ( n ) ( L )) vec ( diag ( v )) ) ij = 1 { i = j } a ∈ [ k ] ∑ ξ a ℓ ∈ C a ∑ ( A ℓ i A ℓ j − E ( n ) [ A ℓ i A ℓ j ] ) = 1 { i = j } a ∈ [ k ] ∑ ξ a n − 2 ℓ ∈ C a ∑ ( Θ ℓ i H ℓ i Θ ℓ j H ℓ j − E ( n ) [ Θ ℓ i H ℓ i Θ ℓ j H ℓ j ] ) .
Using Hoeffding’s inequality, we can show that with probability at least 1 − δ 1-\delta 1 − δ ,
max i , j ∈ [ n ] i ≠ j ∣ 1 { i ≠ j } ∑ a ∈ [ k ] ξ a n − 1 ∑ ℓ ∈ C a ( Θ ℓ i H ℓ i Θ ℓ j H ℓ j − E ( n ) [ Θ ℓ i H ℓ i Θ ℓ j H ℓ j ] ) ∣ ≤ K 1 log ( n / δ ) n , \max_{\substack{i,j\in[n]\\i\neq j}}\left|\mathbf{1}_{\{i\neq j\}}\sum_{a\in[k]}\xi_a n^{-1}\sum_{\ell\in C_a}\left(\Theta_{\ell i}H_{\ell i}\Theta_{\ell j}H_{\ell j}-\mathbb{E}^{(n)}[\Theta_{\ell i}H_{\ell i}\Theta_{\ell j}H_{\ell j}]\right)\right|\leq K_1\sqrt{\frac{\log(n/\delta)}{n}}, i , j ∈ [ n ] i = j max 1 { i = j } a ∈ [ k ] ∑ ξ a n − 1 ℓ ∈ C a ∑ ( Θ ℓ i H ℓ i Θ ℓ j H ℓ j − E ( n ) [ Θ ℓ i H ℓ i Θ ℓ j H ℓ j ] ) ≤ K 1 n log ( n / δ ) ,
so that
∥ ( L − E ( n ) ( L ) ) vec ( diag ( v ‾ ) ) ∥ ∞ ≤ K 1 log ( n / δ ) n 3 , \left\|(L-\mathbb{E}^{(n)}(L))\operatorname{vec}(\operatorname{diag}(\overline{v}))\right\|_{\infty}\leq K_1\sqrt{\frac{\log(n/\delta)}{n^3}}, ( L − E ( n ) ( L )) vec ( diag ( v )) ∞ ≤ K 1 n 3 log ( n / δ ) ,
with probability at least 1 − δ 1-\delta 1 − δ . To conclude, we then proceed as before using that, on the event E \mathcal{E} E , it holds that ∥ G ( L ) ∥ ∞ ≤ ∑ u = 0 ∞ ( ∥ L ∥ ∞ ) u ≤ 4 / 3 \lVert\mathcal{G}(L)\rVert_{\infty}\leq\sum_{u=0}^{\infty}(\lVert L\rVert_{\infty})^u\leq4/3 ∥ G ( L ) ∥ ∞ ≤ ∑ u = 0 ∞ (∥ L ∥ ∞ ) u ≤ 4/3 .
We now prove Proposition B.1.
Proof of Proposition B.1. Since vec ( diag ( v ‾ ) ) = vec ( diag ( ∑ b ∈ [ k ] ξ b 1 C b ) ) = ∑ b ∈ [ k ] ξ b vec ( diag ( 1 C b ) ) \operatorname{vec}(\operatorname{diag}(\overline{v}))=\operatorname{vec}\left(\operatorname{diag}\left(\sum_{b\in[k]}\xi_b\mathbf{1}_{C_b}\right)\right)=\sum_{b\in[k]}\xi_b\operatorname{vec}(\operatorname{diag}(\mathbf{1}_{C_b})) vec ( diag ( v )) = vec ( diag ( ∑ b ∈ [ k ] ξ b 1 C b ) ) = ∑ b ∈ [ k ] ξ b vec ( diag ( 1 C b )) and G ≥ 1 ( E ( n ) [ L ] ) = ∑ u ≥ 1 ( E ( n ) [ L ] ) u \mathcal{G}_{\geq1}(\mathbb{E}^{(n)}[L])=\sum_{u\geq1}(\mathbb{E}^{(n)}[L])^u G ≥ 1 ( E ( n ) [ L ]) = ∑ u ≥ 1 ( E ( n ) [ L ] ) u , it follows from item (iii) of Lemma D.3 that,
G ≥ 1 ( E ( n ) [ L ] ) vec ( diag ( v ‾ ) ) = 1 n ∑ u = 1 ∞ ∑ a , c ∈ [ k ] ( H ( u ) ) a c v ( a , c ) = 1 n ∑ a , c ∈ [ k ] ( ∑ u = 1 ∞ H ( u ) ) a c v ( a , c ) . (37) \mathcal{G}_{\geq1}(\mathbb{E}^{(n)}[L])\operatorname{vec}(\operatorname{diag}(\overline{v}))=\frac{1}{n}\sum_{u=1}^{\infty}\sum_{a,c\in[k]} \left(H^{(u)}\right)_{ac}v^{(a,c)}
=\frac{1}{n}\sum_{a,c\in[k]}\left(\sum_{u=1}^{\infty}H^{(u)}\right)_{ac}v^{(a,c)}. \tag*{(37)} G ≥ 1 ( E ( n ) [ L ]) vec ( diag ( v )) = n 1 u = 1 ∑ ∞ a , c ∈ [ k ] ∑ ( H ( u ) ) a c v ( a , c ) = n 1 a , c ∈ [ k ] ∑ ( u = 1 ∑ ∞ H ( u ) ) a c v ( a , c ) . (37)
Set Φ : = ∑ u = 1 ∞ H ( u ) \Phi:=\sum_{u=1}^{\infty}H^{(u)} Φ := ∑ u = 1 ∞ H ( u ) . We first check that Φ \Phi Φ is well-defined. Since n α a ≥ 1 n\alpha_a\geq1 n α a ≥ 1 for all a ∈ [ k ] a\in[k] a ∈ [ k ] , we have that 0 ≤ 1 − 1 n α a ≤ 1 0\leq1-\frac{1}{n\alpha_a}\leq1 0 ≤ 1 − n α a 1 ≤ 1 for all a ∈ [ k ] a\in[k] a ∈ [ k ] . From this it follows that
∥ H ( u ) − 1 n Diag ( H ( u ) ) diag ( α − 1 ) ∥ max = ( max a ≠ b ∣ H a b ( u ) ∣ ) ∨ ( max a ∣ H a a ( u ) ( 1 − 1 n α a ) ∣ ) ≤ ∥ H ( u ) ∥ max . \left\|H^{(u)}-\frac{1}{n}\operatorname{Diag}\left(H^{(u)}\right)\operatorname{diag}\left(\alpha^{-1}\right)\right\|_{\max}
=\left(\max_{a\ne b}\left|H^{(u)}_{ab}\right|\right)\vee\left(\max_a\left|H^{(u)}_{aa}\left(1-\frac{1}{n\alpha_a}\right)\right|\right)
\leq\left\|H^{(u)}\right\|_{\max}. H ( u ) − n 1 Diag ( H ( u ) ) diag ( α − 1 ) m a x = ( a = b max H ab ( u ) ) ∨ ( a max H aa ( u ) ( 1 − n α a 1 ) ) ≤ H ( u ) m a x .
Using the inequalities ∥ B C ∥ max ≤ ∥ B ∥ ∞ ∥ C ∥ max \left\|BC\right\|_{\max}\leq\left\|B\right\|_{\infty}\left\|C\right\|_{\max} ∥ B C ∥ m a x ≤ ∥ B ∥ ∞ ∥ C ∥ m a x and ∥ B C ∥ max ≤ ∥ B ∥ max ∥ C T ∥ ∞ \left\|BC\right\|_{\max}\leq\left\|B\right\|_{\max}\left\|C^{\mathsf{T}}\right\|_{\infty} ∥ B C ∥ m a x ≤ ∥ B ∥ m a x C T ∞ together with ∥ W ∥ ∞ ≤ M \left\|W\right\|_{\infty}\leq M ∥ W ∥ ∞ ≤ M and (47), we then obtain
∥ H ( u + 1 ) ∥ max ≤ ∥ W ∥ ∞ ∥ H ( u ) − 1 n Diag ( H ( u ) ) diag ( α − 1 ) ∥ max ∥ W ∥ ∞ ≤ M 2 ∥ H ( u ) ∥ max . \left\|H^{(u+1)}\right\|_{\max}\leq\left\|W\right\|_{\infty}\left\|H^{(u)}-\frac{1}{n}\operatorname{Diag}\left(H^{(u)}\right)\operatorname{diag}\left(\alpha^{-1}\right)\right\|_{\max}\left\|W\right\|_{\infty}\leq M^2\left\|H^{(u)}\right\|_{\max}. H ( u + 1 ) m a x ≤ ∥ W ∥ ∞ H ( u ) − n 1 Diag ( H ( u ) ) diag ( α − 1 ) m a x ∥ W ∥ ∞ ≤ M 2 H ( u ) m a x .
Iterating the above inequality, we deduce that ∥ H ( u + 1 ) ∥ max ≤ M 2 u ∥ H ( 1 ) ∥ max \left\|H^{(u+1)}\right\|_{\max}\leq M^{2u}\left\|H^{(1)}\right\|_{\max} H ( u + 1 ) m a x ≤ M 2 u H ( 1 ) m a x . Now, since M < 1 / 2 M<1/2 M < 1/2 under Condition (15) and ∥ H ( 1 ) ∥ max = max a , b ∣ ∑ c W a c ξ c W c b α b − 1 ∣ < ∞ \left\|H^{(1)}\right\|_{\max}=\max_{a,b}\left|\sum_cW_{ac}\xi_cW_{cb}\alpha_b^{-1}\right|<\infty H ( 1 ) m a x = max a , b ∑ c W a c ξ c W c b α b − 1 < ∞ , it follows then that
∥ Φ ∥ max ≤ ∑ u = 1 ∞ ∥ H ( u ) ∥ max ≤ ( ∑ u = 0 ∞ M 2 u ) ∥ H ( 1 ) ∥ max < ∞ , \left\|\Phi\right\|_{\max}\leq\sum_{u=1}^{\infty}\left\|H^{(u)}\right\|_{\max}\leq\left(\sum_{u=0}^{\infty}M^{2u}\right)\left\|H^{(1)}\right\|_{\max}<\infty, ∥ Φ ∥ m a x ≤ u = 1 ∑ ∞ H ( u ) m a x ≤ ( u = 0 ∑ ∞ M 2 u ) H ( 1 ) m a x < ∞ ,
proving that Φ \Phi Φ is well-defined.
Summing the recursion (47) over u ≥ 1 u\geq1 u ≥ 1 and using the definition of the matrix Φ \Phi Φ , we see that
Φ = H ( 1 ) + W ( Φ − 1 n Diag ( Φ ) diag ( α − 1 ) ) W T . (38) \Phi=H^{(1)}+W\left(\Phi-\frac{1}{n}\operatorname{Diag}(\Phi)\operatorname{diag}\left(\alpha^{-1}\right)\right)W^{\mathsf{T}}. \tag*{(38)} Φ = H ( 1 ) + W ( Φ − n 1 Diag ( Φ ) diag ( α − 1 ) ) W T . (38)
Next, we solve (38) in its vectorized form. Starting from (38) and using the linearity of the vec \operatorname{vec} vec operation together with the identity vec ( W B W T ) = ( W ⊗ W ) vec ( B ) \operatorname{vec}(WBW^{\mathsf{T}})=(W\otimes W)\operatorname{vec}(B) vec ( W B W T ) = ( W ⊗ W ) vec ( B ) , we have that
vec ( Φ ) = vec ( H ( 1 ) ) + ( W ⊗ W ) [ vec ( Φ ) − 1 n vec ( Diag ( Φ ) diag ( α − 1 ) ) ] . \operatorname{vec}(\Phi)=\operatorname{vec}\left(H^{(1)}\right)+(W\otimes W)\left[\operatorname{vec}(\Phi)-\frac{1}{n}\operatorname{vec}\left(\operatorname{Diag}(\Phi)\operatorname{diag}\left(\alpha^{-1}\right)\right)\right]. vec ( Φ ) = vec ( H ( 1 ) ) + ( W ⊗ W ) [ vec ( Φ ) − n 1 vec ( Diag ( Φ ) diag ( α − 1 ) ) ] .
Let Q ∈ R k 2 × k 2 Q\in\mathbb{R}^{k^2\times k^2} Q ∈ R k 2 × k 2 be the matrix with entries Q a b , c d = 1 { a = b = c = d } α a − 1 Q_{ab,cd}=\mathbf{1}_{\{a=b=c=d\}}\alpha_a^{-1} Q ab , c d = 1 { a = b = c = d } α a − 1 , for all a , b , c , d ∈ [ k ] a,b,c,d\in[k] a , b , c , d ∈ [ k ] and note that for all a , b ∈ [ k ] a,b\in[k] a , b ∈ [ k ] ,
( Q vec ( Φ ) ) a b = 1 { a = b } α a − 1 vec ( Φ ) a a = vec ( Diag ( Φ ) diag ( α − 1 ) ) a b , \left(Q\operatorname{vec}(\Phi)\right)_{ab}=\mathbf{1}_{\{a=b\}}\alpha_a^{-1}\operatorname{vec}(\Phi)_{aa}=\operatorname{vec}\left(\operatorname{Diag}(\Phi)\operatorname{diag}\left(\alpha^{-1}\right)\right)_{ab}, ( Q vec ( Φ ) ) ab = 1 { a = b } α a − 1 vec ( Φ ) aa = vec ( Diag ( Φ ) diag ( α − 1 ) ) ab ,
i.e., vec ( Diag ( Φ ) diag ( α − 1 ) ) = Q vec ( Φ ) \operatorname{vec}\left(\operatorname{Diag}(\Phi)\operatorname{diag}\left(\alpha^{-1}\right)\right)=Q\operatorname{vec}(\Phi) vec ( Diag ( Φ ) diag ( α − 1 ) ) = Q vec ( Φ ) . Consequently, it follows that
( I k 2 − ( W ⊗ W ) ( I k 2 − 1 n Q ) ) vec ( Φ ) = vec ( H ( 1 ) ) . \left(I_{k^2}-(W\otimes W)\left(I_{k^2}-\frac{1}{n}Q\right)\right)\operatorname{vec}(\Phi)=\operatorname{vec}\left(H^{(1)}\right). ( I k 2 − ( W ⊗ W ) ( I k 2 − n 1 Q ) ) vec ( Φ ) = vec ( H ( 1 ) ) .
Observe that the matrix I k 2 − ( W ⊗ W ) ( I k 2 − 1 n Q ) I_{k^2} - (W \otimes W)(I_{k^2} - \frac{1}{n}Q) I k 2 − ( W ⊗ W ) ( I k 2 − n 1 Q ) is invertible. Indeed,
ρ ( ( W ⊗ W ) ( I k 2 − 1 n Q ) ) ≤ ∥ ( W ⊗ W ) ( I k 2 − 1 n Q ) ∥ ∞ ≤ ∥ W ⊗ W ∥ ∞ ∥ I k 2 − 1 n Q ∥ ∞ = ( max a , b ∣ ∑ d W b d ∣ ∑ c W a c ) ( 1 ∨ max a ∣ 1 − 1 n α a ∣ ) ≤ ∥ W ∥ ∞ 2 ≤ M 2 < 1 / 4. \begin{aligned}
\rho\left((W \otimes W)\left(I_{k^2} - \frac{1}{n}Q\right)\right) \le\left\lVert(W \otimes W)\left(I_{k^2} - \frac{1}{n}Q\right)\right\rVert_\infty\\
&\le\lVert W \otimes W\rVert_\infty\left\lVert I_{k^2} - \frac{1}{n}Q\right\rVert_\infty\\
&= \left(\max_{a,b} \left|\sum_d W_{bd}\right| \sum_c W_{ac}\right)\left(1 \vee\max_a \left|1 - \frac{1}{n\alpha_a}\right|\right) \\
&\le\lVert W\rVert_\infty^2 \le M^2 < 1/4.
\end{aligned} ρ ( ( W ⊗ W ) ( I k 2 − n 1 Q ) ) ≤ ( W ⊗ W ) ( I k 2 − n 1 Q ) ∞ ≤ ∥ W ⊗ W ∥ ∞ I k 2 − n 1 Q ∞ = ( a , b max d ∑ W b d c ∑ W a c ) ( 1 ∨ a max 1 − n α a 1 ) ≤ ∥ W ∥ ∞ 2 ≤ M 2 < 1/4.
where last inequality we have used Condition (15).
As a consequence, we deduce that
vec ( Φ ) = ( I k 2 − ( W ⊗ W ) ( I k 2 − 1 n Q ) ) − 1 vec ( H ( 1 ) ) . \operatorname{vec}(\Phi) = \left(I_{k^2} - (W \otimes W)\left(I_{k^2} - \frac{1}{n}Q\right)\right)^{-1}\operatorname{vec}(H^{(1)}). vec ( Φ ) = ( I k 2 − ( W ⊗ W ) ( I k 2 − n 1 Q ) ) − 1 vec ( H ( 1 ) ) .
Now, using the resolvent identity (27) with B = I k 2 − ( W ⊗ W ) ( I k 2 − 1 n Q ) B = I_{k^2} - (W \otimes W)\left(I_{k^2} - \frac{1}{n}Q\right) B = I k 2 − ( W ⊗ W ) ( I k 2 − n 1 Q ) and C = I k 2 − ( W ⊗ W ) C = I_{k^2} - (W \otimes W) C = I k 2 − ( W ⊗ W ) , we see that
( I k 2 − ( W ⊗ W ) ( I k 2 − 1 n Q ) ) − 1 = ( I k 2 − W ⊗ W ) − 1 \left(I_{k^2} - (W \otimes W)\left(I_{k^2} - \frac{1}{n}Q\right)\right)^{-1} = \left(I_{k^2} - W \otimes W\right)^{-1} ( I k 2 − ( W ⊗ W ) ( I k 2 − n 1 Q ) ) − 1 = ( I k 2 − W ⊗ W ) − 1
− 1 n ( I k 2 − ( W ⊗ W ) ( I k 2 − 1 n Q ) ) − 1 ( W ⊗ W ) Q ( I k 2 − W ⊗ W ) − 1 , - \frac{1}{n}\left(I_{k^2} - (W \otimes W)\left(I_{k^2} - \frac{1}{n}Q\right)\right)^{-1}(W \otimes W)Q\left(I_{k^2} - W \otimes W\right)^{-1}, − n 1 ( I k 2 − ( W ⊗ W ) ( I k 2 − n 1 Q ) ) − 1 ( W ⊗ W ) Q ( I k 2 − W ⊗ W ) − 1 ,
which, in turn, allow us to conclude that
vec ( Φ ) = vec ( Ψ ) + 1 n vec ( E ) , (39) \operatorname{vec}(\Phi) = \operatorname{vec}(\Psi) + \frac{1}{n}\operatorname{vec}(E), \tag*{(39)} vec ( Φ ) = vec ( Ψ ) + n 1 vec ( E ) , (39)
where Ψ : = vec − 1 ( ( I k 2 − W ⊗ W ) − 1 vec ( H ( 1 ) ) ) \Psi:= \operatorname{vec}^{-1}\left(\left(I_{k^2} - W \otimes W\right)^{-1}\operatorname{vec}(H^{(1)})\right) Ψ := vec − 1 ( ( I k 2 − W ⊗ W ) − 1 vec ( H ( 1 ) ) ) and
E : = vec − 1 ( − ( I k 2 − ( W ⊗ W ) ( I k 2 − 1 n Q ) ) − 1 ( W ⊗ W ) Q ( I k 2 − W ⊗ W ) − 1 vec ( H ( 1 ) ) ) . E := \operatorname{vec}^{-1}\left(-\left(I_{k^2} - (W \otimes W)\left(I_{k^2} - \frac{1}{n}Q\right)\right)^{-1}(W \otimes W)Q\left(I_{k^2} - W \otimes W\right)^{-1}\operatorname{vec}(H^{(1)})\right). E := vec − 1 ( − ( I k 2 − ( W ⊗ W ) ( I k 2 − n 1 Q ) ) − 1 ( W ⊗ W ) Q ( I k 2 − W ⊗ W ) − 1 vec ( H ( 1 ) ) ) .
Let us check that Ψ \Psi Ψ satisfies (19) . Using first the identity ( B ⊗ C ) u = B u ⊗ C u (B \otimes C)^u = B^u \otimes C^u ( B ⊗ C ) u = B u ⊗ C u which is valid for all u ≥ 0 u \ge0 u ≥ 0 , and then the identity vec ( B D C ) = ( C T ⊗ B ) vec ( D ) \operatorname{vec}(BDC) = (C^{\mathsf{T}} \otimes B)\operatorname{vec}(D) vec ( B D C ) = ( C T ⊗ B ) vec ( D ) , we have
vec ( Ψ ) = ( I k 2 − W ⊗ W ) − 1 vec ( H ( 1 ) ) = ∑ u = 0 ∞ ( W ⊗ W ) u vec ( H ( 1 ) ) = ∑ u = 0 ∞ ( W u ⊗ W u ) vec ( H ( 1 ) ) = ∑ u = 0 ∞ vec ( W u H ( 1 ) ( W T ) u ) = vec ( ∑ u = 0 ∞ W u H ( 1 ) ( W T ) u ) . \begin{aligned}
\operatorname{vec}(\Psi) = (I_{k^2} - W \otimes W)^{-1}\operatorname{vec}(H^{(1)}) = \sum_{u=0}^{\infty}(W \otimes W)^u\operatorname{vec}(H^{(1)}) = \sum_{u=0}^{\infty}(W^u \otimes W^u)\operatorname{vec}(H^{(1)}) \\
&= \sum_{u=0}^{\infty}\operatorname{vec}\left(W^u H^{(1)}(W^{\mathsf{T}})^u\right) = \operatorname{vec}\left(\sum_{u=0}^{\infty}W^u H^{(1)}(W^{\mathsf{T}})^u\right).
\end{aligned} vec ( Ψ ) = ( I k 2 − W ⊗ W ) − 1 vec ( H ( 1 ) ) = u = 0 ∑ ∞ ( W ⊗ W ) u vec ( H ( 1 ) ) = u = 0 ∑ ∞ ( W u ⊗ W u ) vec ( H ( 1 ) ) = u = 0 ∑ ∞ vec ( W u H ( 1 ) ( W T ) u ) = vec ( u = 0 ∑ ∞ W u H ( 1 ) ( W T ) u ) .
Now by (43) , we have
H ( 1 ) = W diag ( ξ ) W diag ( α − 1 ) = W diag ( ξ ) diag ( α − 1 ) W T = W diag ( ξ ⊙ α − 1 ) W T , H^{(1)} = W\operatorname{diag}(\xi)W\operatorname{diag}(\alpha^{-1}) = W\operatorname{diag}(\xi)\operatorname{diag}(\alpha^{-1})W^{\mathsf{T}} = W\operatorname{diag}(\xi\mathbin{\odot} \alpha^{-1})W^{\mathsf{T}}, H ( 1 ) = W diag ( ξ ) W diag ( α − 1 ) = W diag ( ξ ) diag ( α − 1 ) W T = W diag ( ξ ⊙ α − 1 ) W T ,
so that W u H ( 1 ) ( W T ) u = W u + 1 diag ( ξ ⊙ α − 1 ) ( W T ) u + 1 W^u H^{(1)}(W^{\mathsf{T}})^u = W^{u+1}\operatorname{diag}(\xi\mathbin{\odot} \alpha^{-1})(W^{\mathsf{T}})^{u+1} W u H ( 1 ) ( W T ) u = W u + 1 diag ( ξ ⊙ α − 1 ) ( W T ) u + 1 . As a consequence, we obtain that
vec ( Ψ ) = vec ( ∑ u = 0 ∞ W u H ( 1 ) ( W T ) u ) = vec ( ∑ u = 0 ∞ W u + 1 diag ( ξ ⊙ α − 1 ) ( W T ) u + 1 ) , \operatorname{vec}(\Psi) = \operatorname{vec}\left(\sum_{u=0}^{\infty}W^u H^{(1)}(W^{\mathsf{T}})^u\right) = \operatorname{vec}\left(\sum_{u=0}^{\infty}W^{u+1}\operatorname{diag}(\xi\mathbin{\odot} \alpha^{-1})(W^{\mathsf{T}})^{u+1}\right), vec ( Ψ ) = vec ( u = 0 ∑ ∞ W u H ( 1 ) ( W T ) u ) = vec ( u = 0 ∑ ∞ W u + 1 diag ( ξ ⊙ α − 1 ) ( W T ) u + 1 ) ,
establishing (19).
Finally, using that ∥ G ( D ) ∥ ∞ ≤ ( 1 − ∥ D ∥ ∞ ) − 1 \lVert\mathcal{G}(D)\rVert_{\infty} \leq(1-\lVert D\rVert_{\infty})^{-1} ∥ G ( D ) ∥ ∞ ≤ ( 1 − ∥ D ∥ ∞ ) − 1 for any matrix D D D such that ∥ D ∥ ∞ < 1 \lVert D\rVert_{\infty}<1 ∥ D ∥ ∞ < 1 , together with ∥ W ⊗ W ∥ ∞ = ∥ W ∥ ∞ 2 \lVert W\otimes W\rVert_{\infty}=\lVert W\rVert_{\infty}^{2} ∥ W ⊗ W ∥ ∞ = ∥ W ∥ ∞ 2 and ∥ Q ∥ ∞ = α min − 1 \lVert Q\rVert_{\infty}=\alpha_{\min}^{-1} ∥ Q ∥ ∞ = α m i n − 1 , one can check that
∥ E ∥ max = ∥ vec ( E ) ∥ ∞ ≤ ∥ Q ∥ ∞ ∥ W ∥ ∞ 2 ( 1 − ∥ W ∥ ∞ 2 ) 2 ∥ H ( 1 ) ∥ max ≤ ∥ W ∥ ∞ 4 1 − ∥ W ∥ ∞ 2 ⋅ 1 α min ⋅ max a ∈ [ k ] ξ a α a = : K \lVert E\rVert_{\max}=\lVert\operatorname{vec}(E)\rVert_{\infty}\leq\frac{\lVert Q\rVert_{\infty}\lVert W\rVert_{\infty}^{2}}{(1-\lVert W\rVert_{\infty}^{2})^{2}}\lVert H^{(1)}\rVert_{\max}\leq\frac{\lVert W\rVert_{\infty}^{4}}{1-\lVert W\rVert_{\infty}^{2}}\cdot\frac{1}{\alpha_{\min}}\cdot\max_{a\in[k]}\frac{\xi_a}{\alpha_a}=:K ∥ E ∥ m a x = ∥ vec ( E ) ∥ ∞ ≤ ( 1 − ∥ W ∥ ∞ 2 ) 2 ∥ Q ∥ ∞ ∥ W ∥ ∞ 2 ∥ H ( 1 ) ∥ m a x ≤ 1 − ∥ W ∥ ∞ 2 ∥ W ∥ ∞ 4 ⋅ α m i n 1 ⋅ a ∈ [ k ] max α a ξ a =: K
Clearly K < ∞ K<\infty K < ∞ because ∥ W ∥ ∞ ≤ M < 1 / 2 \lVert W\rVert_{\infty}\leq M<1/2 ∥ W ∥ ∞ ≤ M < 1/2 by Condition (15).
Inverting the vectorization in equation (39) yields Φ = Ψ + 1 n E \Phi=\Psi+\frac{1}{n}E Φ = Ψ + n 1 E with ∥ E ∥ max ≤ K \lVert E\rVert_{\max}\leq K ∥ E ∥ m a x ≤ K , and the result follows from (37) and the fact that 1 n ∑ a , c ∈ [ k ] Ψ a c v ( a , c ) = vec ( d 0 ( Σ ‾ ( 0 ) ) ) \frac{1}{n}\sum_{a,c\in[k]}\Psi_{ac}\mathbf{v}^{(a,c)}=\operatorname{vec}(d_0(\overline{\Sigma}^{(0)})) n 1 ∑ a , c ∈ [ k ] Ψ a c v ( a , c ) = vec ( d 0 ( Σ ( 0 ) )) .
Asymptotic approximation for the lagged covariance matrices
Here, we prove Theorem 3.3.
Proof of Theorem 3.3. Throughout the proof, we denote E = { ( Θ , H ) ∈ Ω a d m } \mathcal{E}=\{(\Theta,H)\in\Omega^{\mathrm{adm}}\} E = {( Θ , H ) ∈ Ω adm } . Fix an integer t ≥ 1 t\geq1 t ≥ 1 . As in the proof of Theorem 3.1, for every τ > 0 \tau>0 τ > 0 , note that we can write
P ( n ) ( ∥ Σ ( Θ , H ) ( t ) − Σ ‾ ( Θ , H ) ( t ) ∥ max ≥ τ ) ≤ P ( n ) ( E c ) + P ( n ) ( E ∩ { ∥ Σ ( Θ , H ) ( t ) − Σ ‾ ( Θ , H ) ( t ) ∥ max ≥ τ } ) . (40) \mathbb{P}^{(n)}\left(\left\lVert\Sigma^{(t)}_{(\Theta,H)}-\overline{\Sigma}^{(t)}_{(\Theta,H)}\right\rVert_{\max}\geq\tau\right)\leq\mathbb{P}^{(n)}(\mathcal{E}^{c})+\mathbb{P}^{(n)}\left(\mathcal{E}\cap\left\{\left\lVert\Sigma^{(t)}_{(\Theta,H)}-\overline{\Sigma}^{(t)}_{(\Theta,H)}\right\rVert_{\max}\geq\tau\right\}\right). \tag*{(40)} P ( n ) ( Σ ( Θ , H ) ( t ) − Σ ( Θ , H ) ( t ) m a x ≥ τ ) ≤ P ( n ) ( E c ) + P ( n ) ( E ∩ { Σ ( Θ , H ) ( t ) − Σ ( Θ , H ) ( t ) m a x ≥ τ } ) . (40)
By Lemma 3.1, the first term is at most 2 n e − C n 2ne^{-C_n} 2 n e − C n . Therefore, it suffices to show that the second term is at most δ \delta δ , for τ = K log ( n / δ ) / n 3 \tau=K\sqrt{\log(n/\delta)/n^3} τ = K log ( n / δ ) / n 3 where K K K is a suitable positive independent of n n n and t t t .
By Proposition 2.3, we have that Σ ( Θ , H ) ( t ) = ( A ⊤ ) t Σ ( Θ , H ) ( 0 ) \Sigma^{(t)}_{(\Theta,H)}=(A^{\top})^t\Sigma^{(0)}_{(\Theta,H)} Σ ( Θ , H ) ( t ) = ( A ⊤ ) t Σ ( Θ , H ) ( 0 ) . Therefore, decomposing Σ ( Θ , H ) ( 0 ) = diag ( v ( Θ , H ) ( X 0 ) ) + d 0 ( Σ ( Θ , H ) ( 0 ) ) \Sigma^{(0)}_{(\Theta,H)}=\operatorname{diag}(v_{(\Theta,H)}(X_0))+d_0(\Sigma^{(0)}_{(\Theta,H)}) Σ ( Θ , H ) ( 0 ) = diag ( v ( Θ , H ) ( X 0 )) + d 0 ( Σ ( Θ , H ) ( 0 ) ) and recalling the definition of Σ ‾ ( Θ , H ) ( t ) \overline{\Sigma}^{(t)}_{(\Theta,H)} Σ ( Θ , H ) ( t ) (see (5)), we can write
Σ ( Θ , H ) ( t ) − Σ ‾ ( Θ , H ) ( t ) = R 1 + R 2 + R 3 + R 4 , \Sigma^{(t)}_{(\Theta,H)}-\overline{\Sigma}^{(t)}_{(\Theta,H)}=R_1+R_2+R_3+R_4, Σ ( Θ , H ) ( t ) − Σ ( Θ , H ) ( t ) = R 1 + R 2 + R 3 + R 4 ,
where
R 1 : = ( A ⊤ ) t ( d 0 ( Σ ( Θ , H ) ( 0 ) ) − d 0 ( Σ ‾ ( 0 ) ) ) , R 3 : = ( ( A ⊤ ) t − ( E [ A ⊤ ] ) t ) d 0 ( Σ ‾ ( 0 ) ) , R_1:= (A^{\top})^t\left(d_0(\Sigma^{(0)}_{(\Theta,H)})-d_0(\overline{\Sigma}^{(0)})\right),\qquad R_3:=\left((A^{\top})^t-\left(\mathbb{E}[A^{\top}]\right)^t\right)d_0(\overline{\Sigma}^{(0)}), R 1 := ( A ⊤ ) t ( d 0 ( Σ ( Θ , H ) ( 0 ) ) − d 0 ( Σ ( 0 ) ) ) , R 3 := ( ( A ⊤ ) t − ( E [ A ⊤ ] ) t ) d 0 ( Σ ( 0 ) ) ,
R 2 : = ( A ⊤ ) t ( diag ( v ( Θ , H ) ( X 0 ) ) − diag ( v ‾ ) ) , R 4 : = ( E [ A ⊤ ] ) t d 0 ( Σ ‾ ( 0 ) ) − 1 n ∑ a , b ∈ [ k ] ( W t Ψ ) a b 1 C a 1 C b ⊤ . R_2:= (A^{\top})^t\left(\operatorname{diag}(v_{(\Theta,H)}(X_0))-\operatorname{diag}(\overline{v})\right),\qquad R_4:=\left(\mathbb{E}[A^{\top}]\right)^td_0(\overline{\Sigma}^{(0)})-\frac{1}{n}\sum_{a,b\in[k]}(W^t\Psi)_{ab}\mathbf{1}_{C_a}\mathbf{1}_{C_b}^{\top}. R 2 := ( A ⊤ ) t ( diag ( v ( Θ , H ) ( X 0 )) − diag ( v ) ) , R 4 := ( E [ A ⊤ ] ) t d 0 ( Σ ( 0 ) ) − n 1 a , b ∈ [ k ] ∑ ( W t Ψ ) ab 1 C a 1 C b ⊤ .
Below, we upper bound each term on the event E \mathcal{E} E . In what follows, K 1 K_1 K 1 denotes a positive constant independent of n n n and t t t that may change from line to line.
Term R 1 R_1 R 1 . Using that ∥ D 1 D 2 ∥ max ≤ ∥ D 1 ∥ ∞ ∥ D 2 ∥ max \lVert D_1D_2\rVert_{\max}\leq\lVert D_1\rVert_{\infty}\lVert D_2\rVert_{\max} ∥ D 1 D 2 ∥ m a x ≤ ∥ D 1 ∥ ∞ ∥ D 2 ∥ m a x and that ∥ ( A ⊤ ) t ∥ ∞ ≤ ∥ A ⊤ ∥ ∞ t < 1 \lVert(A^{\top})^t\rVert_{\infty}\leq\lVert A^{\top}\rVert_{\infty}^{t}<1 ∥( A ⊤ ) t ∥ ∞ ≤ ∥ A ⊤ ∥ ∞ t < 1 on the event E \mathcal{E} E , we have
∥ R 1 ∥ max ≤ ∥ ( A ⊤ ) t ∥ ∞ ∥ d 0 ( Σ ( Θ , H ) ( 0 ) ) − d 0 ( Σ ‾ ( 0 ) ) ∥ max ≤ ∥ d 0 ( Σ ( Θ , H ) ( 0 ) ) − d 0 ( Σ ‾ ( 0 ) ) ∥ max , \lVert R_1\rVert_{\max}\leq\lVert(A^{\top})^t\rVert_{\infty}\lVert d_0(\Sigma^{(0)}_{(\Theta,H)})-d_0(\overline{\Sigma}^{(0)})\rVert_{\max}\leq\lVert d_0(\Sigma^{(0)}_{(\Theta,H)})-d_0(\overline{\Sigma}^{(0)})\rVert_{\max}, ∥ R 1 ∥ m a x ≤ ∥( A ⊤ ) t ∥ ∞ ∥ d 0 ( Σ ( Θ , H ) ( 0 ) ) − d 0 ( Σ ( 0 ) ) ∥ m a x ≤ ∥ d 0 ( Σ ( Θ , H ) ( 0 ) ) − d 0 ( Σ ( 0 ) ) ∥ m a x ,
on the event E \mathcal{E} E . By the proof of Theorem 3.2 (see (32)), we can then conclude from the above inequality that
P ( n ) ( E ∩ { ∥ R 1 ∥ max ≥ K 1 log ( n / δ ) / n 3 } ) ≤ δ . \mathbb{P}^{(n)}\left(\mathcal{E}\cap\left\{\lVert R_1\rVert_{\max}\geq K_1\sqrt{\log(n/\delta)/n^3}\right\}\right)\leq\delta. P ( n ) ( E ∩ { ∥ R 1 ∥ m a x ≥ K 1 log ( n / δ ) / n 3 } ) ≤ δ .
Term R 2 R_2 R 2 . Using the inequalities ∥ ( A ⊤ ) t B ∥ max ≤ ∥ ( A ⊤ ) t − 1 ∥ ∞ ∥ A ⊤ B ∥ max \lVert(A^{\top})^tB\rVert_{\max} \le\lVert(A^{\top})^{t-1}\rVert_{\infty}\lVert A^{\top}B\rVert_{\max} ∥( A ⊤ ) t B ∥ m a x ≤ ∥( A ⊤ ) t − 1 ∥ ∞ ∥ A ⊤ B ∥ m a x and ∥ A ⊤ B ∥ max ≤ ∥ A ⊤ ∥ max ∥ B ∥ ∞ \lVert A^{\top}B\rVert_{\max} \le\lVert A^{\top}\rVert_{\max}\lVert B\rVert_{\infty} ∥ A ⊤ B ∥ m a x ≤ ∥ A ⊤ ∥ m a x ∥ B ∥ ∞ both with B = diag ( v ( Θ , H ) ( X 0 ) ) − diag ( v ‾ ) B=\operatorname{diag}(v_{(\Theta,H)}(X_0))-\operatorname{diag}(\overline{v}) B = diag ( v ( Θ , H ) ( X 0 )) − diag ( v ) , we deduce that
∥ R 2 ∥ max ≤ ∥ ( A ⊤ ) t − 1 ∥ ∞ ∥ A ⊤ ∥ max ∥ v ( Θ , H ) ( X 0 ) − v ‾ ∥ ∞ . \lVert R_2\rVert_{\max}\le\lVert(A^{\top})^{t-1}\rVert_{\infty}\lVert A^{\top}\rVert_{\max}\lVert v_{(\Theta,H)}(X_0)-\overline{v}\rVert_{\infty}. ∥ R 2 ∥ m a x ≤ ∥( A ⊤ ) t − 1 ∥ ∞ ∥ A ⊤ ∥ m a x ∥ v ( Θ , H ) ( X 0 ) − v ∥ ∞ .
Since ∥ A ⊤ ∥ max ≤ n − 1 ( max b ∈ [ k ] μ b ∨ μ ≁ ) \lVert A^{\top}\rVert_{\max}\le n^{-1}\left(\max_{b\in[k]}\mu_b\vee\mu_{\not\sim}\right) ∥ A ⊤ ∥ m a x ≤ n − 1 ( max b ∈ [ k ] μ b ∨ μ ∼ ) and ∥ ( A ⊤ ) t ∥ ∞ ≤ ( ∥ A ⊤ ∥ ∞ ) t < 1 \lVert(A^{\top})^t\rVert_{\infty}\le\left(\lVert A^{\top}\rVert_{\infty}\right)^t<1 ∥( A ⊤ ) t ∥ ∞ ≤ ( ∥ A ⊤ ∥ ∞ ) t < 1 on the event E \mathcal{E} E , we then obtain
∥ R 2 ∥ max ≤ max b ∈ [ k ] μ b ∨ μ ≁ n ∥ v ( Θ , H ) ( X 0 ) − v ‾ ∥ ∞ . \lVert R_2\rVert_{\max}\le\frac{\max_{b\in[k]}\mu_b\vee\mu_{\not\sim}}{n}\lVert v_{(\Theta,H)}(X_0)-\overline{v}\rVert_{\infty}. ∥ R 2 ∥ m a x ≤ n max b ∈ [ k ] μ b ∨ μ ∼ ∥ v ( Θ , H ) ( X 0 ) − v ∥ ∞ .
Finally, since ∥ v ( Θ , H ) ( X 0 ) − v ‾ ∥ ∞ ≤ ∥ E ( Θ , H ) [ X 0 ] − m ‾ ∥ ∞ \lVert v_{(\Theta,H)}(X_0)-\overline{v}\rVert_{\infty}\le\lVert\mathbb{E}_{(\Theta,H)}[X_0]-\overline{m}\rVert_{\infty} ∥ v ( Θ , H ) ( X 0 ) − v ∥ ∞ ≤ ∥ E ( Θ , H ) [ X 0 ] − m ∥ ∞ , we can combine the above inequality with inequalities (26) and (25) to deduce that
P ( n ) ( E ∩ { ∥ R 2 ∥ max ≥ K 1 log ( n / δ ) n 3 } ) ≤ δ . \mathbb{P}^{(n)}\left(\mathcal{E}\cap\left\{\lVert R_2\rVert_{\max}\ge K_1\sqrt{\frac{\log(n/\delta)}{n^3}}\right\}\right)\le\delta. P ( n ) ( E ∩ { ∥ R 2 ∥ m a x ≥ K 1 n 3 log ( n / δ ) } ) ≤ δ .
Term R 3 R_3 R 3 . Let Δ : = A ⊤ − E ( n ) [ A ⊤ ] \Delta:=A^{\top}-\mathbb{E}^{(n)}[A^{\top}] Δ := A ⊤ − E ( n ) [ A ⊤ ] and recall the identity (which can be easily proved by induction on t t t ),
( A ⊤ ) t − ( E ( n ) [ A ⊤ ] ) t = ∑ s = 0 t − 1 ( A ⊤ ) t − 1 − s Δ ( E ( n ) [ A ⊤ ] ) s . (A^{\top})^t-\left(\mathbb{E}^{(n)}[A^{\top}]\right)^t=\sum_{s=0}^{t-1}(A^{\top})^{t-1-s}\Delta\left(\mathbb{E}^{(n)}[A^{\top}]\right)^s. ( A ⊤ ) t − ( E ( n ) [ A ⊤ ] ) t = s = 0 ∑ t − 1 ( A ⊤ ) t − 1 − s Δ ( E ( n ) [ A ⊤ ] ) s .
Using ∥ D 1 D 2 ∥ max ≤ ∥ D 1 ∥ ∞ ∥ D 2 ∥ max \lVert D_1D_2\rVert_{\max}\le\lVert D_1\rVert_{\infty}\lVert D_2\rVert_{\max} ∥ D 1 D 2 ∥ m a x ≤ ∥ D 1 ∥ ∞ ∥ D 2 ∥ m a x , the sub-multiplicativity of ∥ ⋅ ∥ ∞ \lVert\cdot\rVert_{\infty} ∥ ⋅ ∥ ∞ , the bound ∥ A ⊤ ∥ ∞ < 1 \lVert A^{\top}\rVert_{\infty}<1 ∥ A ⊤ ∥ ∞ < 1 on the event E \mathcal{E} E , and the fact that ∥ E [ A ⊤ ] ∥ ∞ = ∥ W ∥ ∞ ≤ M ≤ 1 / 2 \lVert\mathbb{E}[A^{\top}]\rVert_{\infty}=\lVert W\rVert_{\infty}\le M\le1/2 ∥ E [ A ⊤ ] ∥ ∞ = ∥ W ∥ ∞ ≤ M ≤ 1/2 (the last inequality holds because Condition (15) is granted), we then obtain
∥ R 3 ∥ max ≤ ∥ ( A ⊤ ) t − ( E [ A ⊤ ] ) t ∥ ∞ ∥ d 0 ( Σ ‾ ( 0 ) ) ∥ max ≤ ∥ Δ ∥ ∞ ( ∑ s = 0 t − 1 2 − s ) ∥ d 0 ( Σ ‾ ( 0 ) ) ∥ max ≤ 2 ∥ Δ ∥ ∞ ∥ d 0 ( Σ ‾ ( 0 ) ) ∥ max . \begin{aligned}
\lVert R_3\rVert_{\max}\le\left\lVert(A^{\top})^t-\left(\mathbb{E}[A^{\top}]\right)^t\right\rVert_{\infty}\lVert d_0(\overline{\Sigma}^{(0)})\rVert_{\max} \\
&\le\lVert\Delta\rVert_{\infty}\left(\sum_{s=0}^{t-1}2^{-s}\right)\lVert d_0(\overline{\Sigma}^{(0)})\rVert_{\max}\le2\lVert\Delta\rVert_{\infty}\lVert d_0(\overline{\Sigma}^{(0)})\rVert_{\max}.
\end{aligned} ∥ R 3 ∥ m a x ≤ ( A ⊤ ) t − ( E [ A ⊤ ] ) t ∞ ∥ d 0 ( Σ ( 0 ) ) ∥ m a x ≤ ∥ Δ ∥ ∞ ( s = 0 ∑ t − 1 2 − s ) ∥ d 0 ( Σ ( 0 ) ) ∥ m a x ≤ 2 ∥ Δ ∥ ∞ ∥ d 0 ( Σ ( 0 ) ) ∥ m a x .
Finally, using that ∥ d 0 ( Σ ‾ ( 0 ) ) ∥ max ≤ n − 1 ∥ Ψ ∥ max \lVert d_0(\overline{\Sigma}^{(0)})\rVert_{\max}\le n^{-1}\lVert\Psi\rVert_{\max} ∥ d 0 ( Σ ( 0 ) ) ∥ m a x ≤ n − 1 ∥ Ψ ∥ m a x and that ∥ Δ ∥ ∞ = ∥ A ⊤ − E ( n ) [ A ⊤ ] ∥ ∞ ≤ K 1 log ( n / δ ) / n \lVert\Delta\rVert_{\infty}=\lVert A^{\top}-\mathbb{E}^{(n)}[A^{\top}]\rVert_{\infty}\le K_1\sqrt{\log(n/\delta)/n} ∥ Δ ∥ ∞ = ∥ A ⊤ − E ( n ) [ A ⊤ ] ∥ ∞ ≤ K 1 log ( n / δ ) / n with probability at least 1 − δ 1-\delta 1 − δ (which can be proved similarly to what we did in the proof of Theorem 3.1), we conclude that
P ( n ) ( E ∩ { ∥ R 3 ∥ max ≥ K 1 log ( n / δ ) n 3 } ) ≤ δ . \mathbb{P}^{(n)}\left(\mathcal{E}\cap\left\{\lVert R_3\rVert_{\max}\ge K_1\sqrt{\frac{\log(n/\delta)}{n^3}}\right\}\right)\le\delta. P ( n ) ( E ∩ { ∥ R 3 ∥ m a x ≥ K 1 n 3 log ( n / δ ) } ) ≤ δ .
Term R 4 R_4 R 4 . Note that R 4 R_4 R 4 is a deterministic matrix. By Lemma D.2, we know that ( E ( n ) [ A ⊤ ] ) t 1 C a = ∑ c ∈ [ k ] ( W t ) c a 1 C c \left(\mathbb{E}^{(n)}[A^{\top}]\right)^t\mathbf{1}_{C_a}=\sum_{c\in[k]}(W^t)_{ca}\mathbf{1}_{C_c} ( E ( n ) [ A ⊤ ] ) t 1 C a = ∑ c ∈ [ k ] ( W t ) c a 1 C c for every t ≥ 1 t\ge1 t ≥ 1 . Writing d 0 ( 1 C a 1 C b ⊤ ) = 1 C a 1 C b ⊤ − 1 { a = b } diag ( 1 C a ) d_0(\mathbf{1}_{C_a}\mathbf{1}_{C_b}^{\top})=\mathbf{1}_{C_a}\mathbf{1}_{C_b}^{\top}-\mathbb{1}_{\{a=b\}}\operatorname{diag}(\mathbf{1}_{C_a}) d 0 ( 1 C a 1 C b ⊤ ) = 1 C a 1 C b ⊤ − 1 { a = b } diag ( 1 C a ) and also using that B 1 C a 1 C b ⊤ = ( B 1 C a ) 1 C b ⊤ B\mathbf{1}_{C_a}\mathbf{1}_{C_b}^{\top}=(B\mathbf{1}_{C_a})\mathbf{1}_{C_b}^{\top} B 1 C a 1 C b ⊤ = ( B 1 C a ) 1 C b ⊤ for any n × n n\times n n × n matrix B B B , we obtain
( E [ A ⊤ ] ) t d 0 ( Σ ‾ ( 0 ) ) = 1 n ∑ a , b ∈ [ k ] Ψ a b ( ( E ( n ) [ A ⊤ ] ) t 1 C a ) 1 C b ⊤ − 1 n ∑ a ∈ [ k ] Ψ a a ( E ( n ) [ A ⊤ ] ) t diag ( 1 C a ) = 1 n ∑ a , b ∈ [ k ] Ψ a b ( ∑ c ∈ [ k ] ( W t ) c a 1 C c ) 1 C b ⊤ − 1 n ∑ a ∈ [ k ] Ψ a a ( E ( n ) [ A ⊤ ] ) t diag ( 1 C a ) . \begin{aligned}
\left(\mathbb{E}[A^{\top}]\right)^t d_0(\overline{\Sigma}^{(0)})=\frac{1}{n}\sum_{a,b\in[k]}\Psi_{ab}\left(\left(\mathbb{E}^{(n)}[A^{\top}]\right)^t\mathbf{1}_{C_a}\right)\mathbf{1}_{C_b}^{\top}-\frac{1}{n}\sum_{a\in[k]}\Psi_{aa}\left(\mathbb{E}^{(n)}[A^{\top}]\right)^t\operatorname{diag}(\mathbf{1}_{C_a}) \\
&=\frac{1}{n}\sum_{a,b\in[k]}\Psi_{ab}\left(\sum_{c\in[k]}(W^t)_{ca}\mathbf{1}_{C_c}\right)\mathbf{1}_{C_b}^{\top}-\frac{1}{n}\sum_{a\in[k]}\Psi_{aa}\left(\mathbb{E}^{(n)}[A^{\top}]\right)^t\operatorname{diag}(\mathbf{1}_{C_a}).
\end{aligned} ( E [ A ⊤ ] ) t d 0 ( Σ ( 0 ) ) = n 1 a , b ∈ [ k ] ∑ Ψ ab ( ( E ( n ) [ A ⊤ ] ) t 1 C a ) 1 C b ⊤ − n 1 a ∈ [ k ] ∑ Ψ aa ( E ( n ) [ A ⊤ ] ) t diag ( 1 C a ) = n 1 a , b ∈ [ k ] ∑ Ψ ab c ∈ [ k ] ∑ ( W t ) c a 1 C c 1 C b ⊤ − n 1 a ∈ [ k ] ∑ Ψ aa ( E ( n ) [ A ⊤ ] ) t diag ( 1 C a ) .
Since ∑ a ( W t ) c a Ψ a b = ( W t Ψ ) c b \sum_a(W^t)_{ca}\Psi_{ab}=(W^t\Psi)_{cb} ∑ a ( W t ) c a Ψ ab = ( W t Ψ ) c b , the first sum is equal to 1 n ∑ c , b ∈ [ k ] ( W t Ψ ) c b 1 C c 1 C b ⊤ \frac{1}{n}\sum_{c,b\in[k]}(W^t\Psi)_{cb}\mathbf{1}_{C_c}\mathbf{1}_{C_b}^{\top} n 1 ∑ c , b ∈ [ k ] ( W t Ψ ) c b 1 C c 1 C b ⊤ , and therefore
R 4 = − 1 n ∑ a ∈ [ k ] Ψ a a ( E [ A ⊤ ] ) t diag ( 1 C a ) . R_4=-\frac{1}{n}\sum_{a\in[k]}\Psi_{aa}\left(\mathbb{E}[A^{\top}]\right)^t\operatorname{diag}(\mathbf{1}_{C_a}). R 4 = − n 1 a ∈ [ k ] ∑ Ψ aa ( E [ A ⊤ ] ) t diag ( 1 C a ) .
Combining the above identity with the inequalities ∥ ( E [ A ⊤ ] ) t diag ( 1 C a ) ∥ max ≤ ∥ ( E [ A ⊤ ] ) t ∥ max ≤ ∥ E [ A ⊤ ] ∥ ∞ t − 1 ∥ E [ A ⊤ ] ∥ max \lVert(\mathbb{E}[A^{\top}])^{t}\operatorname{diag}(1_{C_a})\rVert_{\max} \le\lVert(\mathbb{E}[A^{\top}])^{t}\rVert_{\max} \le\lVert\mathbb{E}[A^{\top}]\rVert_{\infty}^{t-1}\lVert\mathbb{E}[A^{\top}]\rVert_{\max} ∥( E [ A ⊤ ] ) t diag ( 1 C a ) ∥ m a x ≤ ∥( E [ A ⊤ ] ) t ∥ m a x ≤ ∥ E [ A ⊤ ] ∥ ∞ t − 1 ∥ E [ A ⊤ ] ∥ m a x and ∥ E [ A ⊤ ] ∥ max = n − 1 ∥ W ∥ max \lVert\mathbb{E}[A^{\top}]\rVert_{\max}=n^{-1}\lVert\mathcal{W}\rVert_{\max} ∥ E [ A ⊤ ] ∥ m a x = n − 1 ∥ W ∥ m a x , it follows that
∥ R 4 ∥ max ≤ 1 n ( ∑ a ∈ [ k ] ∣ Ψ a a ∣ ) ∥ E [ A ⊤ ] ∥ ∞ t − 1 ∥ W ∥ max n ≤ K 1 ∥ E [ A ⊤ ] ∥ ∞ t − 1 n 2 . \lVert R_4\rVert_{\max} \le\frac{1}{n}\left(\sum_{a\in[k]}|\Psi_{aa}|\right)\lVert\mathbb{E}[A^{\top}]\rVert_{\infty}^{t-1}\frac{\lVert\mathcal{W}\rVert_{\max}}{n} \le K_1\frac{\lVert\mathbb{E}[A^{\top}]\rVert_{\infty}^{t-1}}{n^2}. ∥ R 4 ∥ m a x ≤ n 1 a ∈ [ k ] ∑ ∣ Ψ aa ∣ ∥ E [ A ⊤ ] ∥ ∞ t − 1 n ∥ W ∥ m a x ≤ K 1 n 2 ∥ E [ A ⊤ ] ∥ ∞ t − 1 .
Finally, using that ∥ E [ A ⊤ ] ∥ ∞ t − 1 < 1 \lVert\mathbb{E}[A^{\top}]\rVert_{\infty}^{t-1}<1 ∥ E [ A ⊤ ] ∥ ∞ t − 1 < 1 on the event E \mathcal{E} E , we deduce that ∥ R 4 ∥ max ≤ K 1 n − 2 \lVert R_4\rVert_{\max}\le K_1n^{-2} ∥ R 4 ∥ m a x ≤ K 1 n − 2 which can be absorbed in the error K 1 log ( n / δ ) / n 3 K_1\sqrt{\log(n/\delta)}/n^3 K 1 log ( n / δ ) / n 3 .
Combining the bounds on error terms R 1 R_1 R 1 , R 2 R_2 R 2 , R 3 R_3 R 3 and R 4 R_4 R 4 via union bound with δ / 4 \delta/4 δ /4 replacing δ \delta δ shows that that the second term of (40) is at most δ \delta δ , for τ = K log ( n / δ ) / n 3 \tau=K\sqrt{\log(n/\delta)}/n^3 τ = K log ( n / δ ) / n 3 where K K K is a suitable positive independent of n n n and t t t , concluding the proof.
On the mean of the matrices A and L
In this section, we collect some auxiliary results concerning the mean of the interaction matrix A A A and of the matrix L L L defined in (30) .
Before stating the first auxiliary result, let W \mathcal{W} W be the k × k k\times k k × k matrix defined by
W a b : = ( 2 β − 1 ) ( μ a p a 1 { a = b } + μ ≁ q 1 { a ≠ b } ) , a , b ∈ [ k ] . (41) \mathcal{W}_{ab}:=(2\beta-1)(\mu_a p_a1_{\{a=b\}}+\mu_{\not\sim}q1_{\{a\ne b\}}),\qquad a,b\in[k]. \tag*{(41)} W ab := ( 2 β − 1 ) ( μ a p a 1 { a = b } + μ ∼ q 1 { a = b } ) , a , b ∈ [ k ] . (41)
Observe that the matrix W \mathcal{W} W is symmetric, i.e. W = W ⊤ \mathcal{W}=\mathcal{W}^{\top} W = W ⊤ , and that it is related to the matrix W W W defined in (16) by
W = W diag ( α ) with α = ( α 1 , … , α k ) . (42) W=\mathcal{W}\operatorname{diag}(\alpha)\ \text{with}\ \alpha=(\alpha_1,\ldots,\alpha_k). \tag*{(42)} W = W diag ( α ) with α = ( α 1 , … , α k ) . (42)
As a consequence of (42) and the symmetry of W \mathcal{W} W , one can easily check that
W diag ( α − 1 ) = W = diag ( α − 1 ) W ⊤ , (43) W\operatorname{diag}(\alpha^{-1})=\mathcal{W}=\operatorname{diag}(\alpha^{-1})W^{\top}, \tag*{(43)} W diag ( α − 1 ) = W = diag ( α − 1 ) W ⊤ , (43)
Iterating (43) , we deduce that ( W ) u diag ( α − 1 ) = diag ( α − 1 ) ( W ⊤ ) u (W)^u\operatorname{diag}(\alpha^{-1})=\operatorname{diag}(\alpha^{-1})(W^{\top})^u ( W ) u diag ( α − 1 ) = diag ( α − 1 ) ( W ⊤ ) u for all u ≥ 0 u\ge0 u ≥ 0 .
The first result of this section reads as follows.
Lemma D.1. Let A A A be the interaction matrix defined in (8) and W \mathcal{W} W the matrix defined in (41) . Then we can write
E [ A ⊤ ] = E [ A ] = 1 n ∑ a , b ∈ [ k ] W a b 1 C a 1 C b ⊤ . (44) \mathbb{E}[A^{\top}]=\mathbb{E}[A]=\frac{1}{n}\sum_{a,b\in[k]}\mathcal{W}_{ab}1_{C_a}1_{C_b}^{\top}. \tag*{(44)} E [ A ⊤ ] = E [ A ] = n 1 a , b ∈ [ k ] ∑ W ab 1 C a 1 C b ⊤ . (44) Moreover, for all a , b ∈ [ k ] a,b\in[k] a , b ∈ [ k ] ,
E [ A ⊤ ] 1 C a 1 C b ⊤ = ∑ c ∈ [ k ] W c a 1 C c 1 C b ⊤ . (45) \mathbb{E}[A^{\top}]1_{C_a}1_{C_b}^{\top}=\sum_{c\in[k]}W_{ca}1_{C_c}1_{C_b}^{\top}. \tag*{(45)} E [ A ⊤ ] 1 C a 1 C b ⊤ = c ∈ [ k ] ∑ W c a 1 C c 1 C b ⊤ . (45) Furthermore,
E [ A ⊤ ] 1 n 1 n ⊤ = ∑ c ∈ [ k ] ( W 1 k ) c 1 C c 1 n ⊤ . (46) \mathbb{E}[A^{\top}]1_n1_n^{\top}=\sum_{c\in[k]}(W1_k)_c1_{C_c}1_n^{\top}. \tag*{(46)} E [ A ⊤ ] 1 n 1 n ⊤ = c ∈ [ k ] ∑ ( W 1 k ) c 1 C c 1 n ⊤ . (46) Proof of Lemma D.1. By the definition of interaction matrix A A A and the independence of the environment, for all i , j ∈ [ n ] i,j\in[n] i , j ∈ [ n ] , we have
E [ A i j ] = 2 β − 1 n ( ∑ a ∈ [ k ] μ a p a 1 { i ∼ a j } + μ ≁ q 1 { i ≁ j } ) . \mathbb{E}[A_{ij}]=\frac{2\beta-1}{n}\left(\sum_{a\in[k]}\mu_ap_a1_{\{i\sim_a j\}}+\mu_{\not\sim}q1_{\{i\not\sim j\}}\right). E [ A ij ] = n 2 β − 1 a ∈ [ k ] ∑ μ a p a 1 { i ∼ a j } + μ ∼ q 1 { i ∼ j } .
Since the right-hand side is symmetric in ( i , j ) (i,j) ( i , j ) , we have E [ A i j ] = E [ A j i ] \mathbb{E}[A_{ij}]=\mathbb{E}[A_{ji}] E [ A ij ] = E [ A j i ] for all i , j ∈ [ n ] i,j\in[n] i , j ∈ [ n ] which, in turn, implies the first equality in (44). Moreover, the expression above for E [ A i j ] \mathbb{E}[A_{ij}] E [ A ij ] and (41) imply that
E [ A ] = 2 β − 1 n ( ∑ a ∈ [ k ] μ a p a 1 C a 1 C a ⊤ + μ ≠ q ∑ a ∈ [ k ] ∑ b ∈ [ k ] b ≠ a 1 C a 1 C b ⊤ ) = 1 n ∑ a , b ∈ [ k ] W a b 1 C a 1 C b ⊤ . \mathbb{E}[A]=\frac{2\beta-1}{n}\left(\sum_{a\in[k]}\mu_a p_a\mathbf{1}_{C_a}\mathbf{1}_{C_a}^{\top}+\mu_{\ne q}\sum_{a\in[k]}\sum_{\substack{b\in[k]\\b\ne a}}\mathbf{1}_{C_a}\mathbf{1}_{C_b}^{\top}\right)=\frac{1}{n}\sum_{a,b\in[k]}W_{ab}\mathbf{1}_{C_a}\mathbf{1}_{C_b}^{\top}. E [ A ] = n 2 β − 1 a ∈ [ k ] ∑ μ a p a 1 C a 1 C a ⊤ + μ = q a ∈ [ k ] ∑ b ∈ [ k ] b = a ∑ 1 C a 1 C b ⊤ = n 1 a , b ∈ [ k ] ∑ W ab 1 C a 1 C b ⊤ .
proving the second equality in (44). Next, we will prove (45). For that end, fix a , b ∈ [ k ] a,b\in[k] a , b ∈ [ k ] and observe that by (44) and (42), we have
E [ A ⊤ ] 1 C a 1 C b ⊤ = ( 1 n ∑ c , d ∈ [ k ] W c d 1 C c 1 C d ⊤ ) 1 C a 1 C b ⊤ = 1 n ∑ c , d ∈ [ k ] W c d ( 1 C c ⊤ 1 C a ) 1 C d 1 C b ⊤ = ∑ c ∈ [ k ] W c a ∣ C a ∣ n 1 C c 1 C b ⊤ = ∑ c ∈ [ k ] W c a 1 C c 1 C b ⊤ , \begin{aligned}
\mathbb{E}[A^{\top}]\mathbf{1}_{C_a}\mathbf{1}_{C_b}^{\top}=\left(\frac{1}{n}\sum_{c,d\in[k]}W_{cd}\mathbf{1}_{C_c}\mathbf{1}_{C_d}^{\top}\right)\mathbf{1}_{C_a}\mathbf{1}_{C_b}^{\top} \\
&=\frac{1}{n}\sum_{c,d\in[k]}W_{cd}\left(\mathbf{1}_{C_c}^{\top}\mathbf{1}_{C_a}\right)\mathbf{1}_{C_d}\mathbf{1}_{C_b}^{\top} \\
&=\sum_{c\in[k]}W_{ca}\frac{|C_a|}{n}\mathbf{1}_{C_c}\mathbf{1}_{C_b}^{\top}=\sum_{c\in[k]}W_{ca}\mathbf{1}_{C_c}\mathbf{1}_{C_b}^{\top},
\end{aligned} E [ A ⊤ ] 1 C a 1 C b ⊤ = n 1 c , d ∈ [ k ] ∑ W c d 1 C c 1 C d ⊤ 1 C a 1 C b ⊤ = n 1 c , d ∈ [ k ] ∑ W c d ( 1 C c ⊤ 1 C a ) 1 C d 1 C b ⊤ = c ∈ [ k ] ∑ W c a n ∣ C a ∣ 1 C c 1 C b ⊤ = c ∈ [ k ] ∑ W c a 1 C c 1 C b ⊤ ,
proving (45). Finally, using 1 n 1 n ⊤ = ∑ a ∈ [ k ] ∑ b ∈ [ k ] 1 C a 1 C b ⊤ \mathbf{1}_n\mathbf{1}_n^{\top}=\sum_{a\in[k]}\sum_{b\in[k]}\mathbf{1}_{C_a}\mathbf{1}_{C_b}^{\top} 1 n 1 n ⊤ = ∑ a ∈ [ k ] ∑ b ∈ [ k ] 1 C a 1 C b ⊤ together with (45), we obtain that
E [ A ⊤ ] 1 n 1 n ⊤ = ∑ a ∈ [ k ] ∑ b ∈ [ k ] ∑ c ∈ [ k ] W c a 1 C c 1 C b ⊤ = ∑ a ∈ [ k ] ∑ c ∈ [ k ] W c a 1 C c ( ∑ b ∈ [ k ] 1 C b ) ⊤ = ∑ c ∈ [ k ] ( ∑ a ∈ [ k ] W c a ) 1 C c 1 n ⊤ = ∑ c ∈ [ k ] ( W 1 k ) c 1 C c 1 n ⊤ , \begin{aligned}
\mathbb{E}[A^{\top}]\mathbf{1}_n\mathbf{1}_n^{\top}=\sum_{a\in[k]}\sum_{b\in[k]}\sum_{c\in[k]}W_{ca}\mathbf{1}_{C_c}\mathbf{1}_{C_b}^{\top}=\sum_{a\in[k]}\sum_{c\in[k]}W_{ca}\mathbf{1}_{C_c}\left(\sum_{b\in[k]}\mathbf{1}_{C_b}\right)^{\top} \\
&=\sum_{c\in[k]}\left(\sum_{a\in[k]}W_{ca}\right)\mathbf{1}_{C_c}\mathbf{1}_n^{\top}=\sum_{c\in[k]}(W\mathbf{1}_k)_c\mathbf{1}_{C_c}\mathbf{1}_n^{\top},
\end{aligned} E [ A ⊤ ] 1 n 1 n ⊤ = a ∈ [ k ] ∑ b ∈ [ k ] ∑ c ∈ [ k ] ∑ W c a 1 C c 1 C b ⊤ = a ∈ [ k ] ∑ c ∈ [ k ] ∑ W c a 1 C c b ∈ [ k ] ∑ 1 C b ⊤ = c ∈ [ k ] ∑ a ∈ [ k ] ∑ W c a 1 C c 1 n ⊤ = c ∈ [ k ] ∑ ( W 1 k ) c 1 C c 1 n ⊤ ,
establishing (46), concluding the proof.
Our second result is the following.
Lemma D.2. For every a ∈ [ k ] a\in[k] a ∈ [ k ] and every integer u ≥ 1 u\ge1 u ≥ 1 ,
( E [ A ⊤ ] ) u 1 C a = ∑ c ∈ [ k ] ( W u ) c a 1 C c . \left(\mathbb{E}[A^{\top}]\right)^u\mathbf{1}_{C_a}=\sum_{c\in[k]}(W^u)_{ca}\mathbf{1}_{C_c}. ( E [ A ⊤ ] ) u 1 C a = c ∈ [ k ] ∑ ( W u ) c a 1 C c .
Proof. We argue by induction on u ≥ 1 u\ge1 u ≥ 1 . To establish the result for u = 1 u=1 u = 1 , we argue as follows. Since E [ A ⊤ ] = 1 n ∑ c , d ∈ [ k ] W c d 1 C c 1 C d ⊤ \mathbb{E}[A^{\top}]=\frac{1}{n}\sum_{c,d\in[k]}W_{cd}\mathbf{1}_{C_c}\mathbf{1}_{C_d}^{\top} E [ A ⊤ ] = n 1 ∑ c , d ∈ [ k ] W c d 1 C c 1 C d ⊤ by (44), we have
E [ A ⊤ ] 1 C a = 1 n ∑ c , d ∈ [ k ] W c d 1 C c ( 1 C d ⊤ 1 C a ) = 1 n ∑ c ∈ [ k ] W c a ∣ C a ∣ 1 C c = ∑ c ∈ [ k ] W c a α a 1 C c , \mathbb{E}[A^{\top}]\mathbf{1}_{C_a}=\frac{1}{n}\sum_{c,d\in[k]}W_{cd}\mathbf{1}_{C_c}\left(\mathbf{1}_{C_d}^{\top}\mathbf{1}_{C_a}\right)=\frac{1}{n}\sum_{c\in[k]}W_{ca}|C_a|\mathbf{1}_{C_c}=\sum_{c\in[k]}W_{ca}\alpha_a\mathbf{1}_{C_c}, E [ A ⊤ ] 1 C a = n 1 c , d ∈ [ k ] ∑ W c d 1 C c ( 1 C d ⊤ 1 C a ) = n 1 c ∈ [ k ] ∑ W c a ∣ C a ∣ 1 C c = c ∈ [ k ] ∑ W c a α a 1 C c ,
where in the second and third equalities we used 1 C d ⊤ 1 C a = ∣ C a ∣ 1 { d = a } \mathbf{1}_{C_d}^{\top}\mathbf{1}_{C_a}=|C_a|\mathbf{1}_{\{d=a\}} 1 C d ⊤ 1 C a = ∣ C a ∣ 1 { d = a } and ∣ C a ∣ = n α a |C_a|=n\alpha_a ∣ C a ∣ = n α a , respectively. Using that W c a = W c a α a W_{ca}=W_{ca}\alpha_a W c a = W c a α a by (42), we see that the right-hand side is equal to ∑ c ∈ [ k ] W c a 1 C c \sum_{c\in[k]}W_{ca}\mathbf{1}_{C_c} ∑ c ∈ [ k ] W c a 1 C c , which is the claim for u = 1 u=1 u = 1 .
Now, assume that the result holds for some u ≥ 1 u\ge1 u ≥ 1 . Using linearity and the inductive hypothesis,
( E [ A ⊤ ] ) u + 1 1 C a = E [ A ⊤ ] ( ∑ c ∈ [ k ] ( W u ) c a 1 C c ) = ∑ c ∈ [ k ] ( W u ) c a E [ A ⊤ ] 1 C c . \left(\mathbb{E}[A^{\top}]\right)^{u+1}\mathbf{1}_{C_a}=\mathbb{E}[A^{\top}]\left(\sum_{c\in[k]}(W^u)_{ca}\mathbf{1}_{C_c}\right)=\sum_{c\in[k]}(W^u)_{ca}\mathbb{E}[A^{\top}]\mathbf{1}_{C_c}. ( E [ A ⊤ ] ) u + 1 1 C a = E [ A ⊤ ] c ∈ [ k ] ∑ ( W u ) c a 1 C c = c ∈ [ k ] ∑ ( W u ) c a E [ A ⊤ ] 1 C c .
Finally, applying the base case to each E [ A T ] 1 C c \mathbb{E}[A^{\mathsf{T}}]1_{C_c} E [ A T ] 1 C c gives
( E [ A T ] ) u + 1 1 C a = ∑ c , d ∈ [ k ] ( W u ) c a W d c 1 C d = ∑ d ∈ [ k ] ( ∑ c ∈ [ k ] W d c ( W u ) c a ) 1 C d = ∑ d ∈ [ k ] ( W u + 1 ) d a 1 C d , \left(\mathbb{E}[A^{\mathsf{T}}]\right)^{u+1}1_{C_a}
= \sum_{c,d\in[k]} (W^u)_{ca}W_{dc}1_{C_d}
= \sum_{d\in[k]}\left(\sum_{c\in[k]}W_{dc}(W^u)_{ca}\right)1_{C_d}
= \sum_{d\in[k]}(W^{u+1})_{da}1_{C_d}, ( E [ A T ] ) u + 1 1 C a = c , d ∈ [ k ] ∑ ( W u ) c a W d c 1 C d = d ∈ [ k ] ∑ c ∈ [ k ] ∑ W d c ( W u ) c a 1 C d = d ∈ [ k ] ∑ ( W u + 1 ) d a 1 C d ,
which concludes the proof.
Our last result focus on the mean of the matrix L L L .
Lemma D.3. Suppose Condition (15) is granted and let ξ \xi ξ be the vector defined in (18). For all a , b ∈ [ k ] a,b\in[k] a , b ∈ [ k ] , let v ( a , b ) v^{(a,b)} v ( a , b ) be the vector defined in (31). Let W W W and L L L be the matrices defined in (16) and (30), respectively. Then the following identities hold.
(i) For all b ∈ [ k ] b\in[k] b ∈ [ k ] ,
E ( n ) [ L ] vec ( diag ( 1 C b ) ) = 1 n ∑ a ∈ [ k ] ∑ c ∈ [ k ] W a b W b c α c − 1 v ( a , c ) . \mathbb{E}^{(n)}[L]\operatorname{vec}\left(\operatorname{diag}(1_{C_b})\right)
= \frac{1}{n}\sum_{a\in[k]}\sum_{c\in[k]}W_{ab}W_{bc}\alpha_c^{-1}v^{(a,c)}. E ( n ) [ L ] vec ( diag ( 1 C b ) ) = n 1 a ∈ [ k ] ∑ c ∈ [ k ] ∑ W ab W b c α c − 1 v ( a , c ) . (ii) For all a , b ∈ [ k ] a,b\in[k] a , b ∈ [ k ] ,
E ( n ) [ L ] v ( a , b ) = ∑ c ∈ [ k ] ∑ d ∈ [ k ] W c a W d b ( 1 − 1 α a n 1 { a = b } ) v ( c , d ) . \mathbb{E}^{(n)}[L]v^{(a,b)}
= \sum_{c\in[k]}\sum_{d\in[k]}W_{ca}W_{db}\left(1-\frac{1}{\alpha_a n}1_{\{a=b\}}\right)v^{(c,d)}. E ( n ) [ L ] v ( a , b ) = c ∈ [ k ] ∑ d ∈ [ k ] ∑ W c a W d b ( 1 − α a n 1 1 { a = b } ) v ( c , d ) . (iii) For all u ≥ 1 u\ge1 u ≥ 1 ,
( E ( n ) [ L ] ) u ( ∑ b ∈ [ k ] ξ b vec ( diag ( 1 C b ) ) ) = 1 n ∑ a ∈ [ k ] ∑ c ∈ [ k ] ( H ( u ) ) a c v ( a , c ) , \left(\mathbb{E}^{(n)}[L]\right)^u
\left(\sum_{b\in[k]}\xi_b\operatorname{vec}\left(\operatorname{diag}(1_{C_b})\right)\right)
= \frac{1}{n}\sum_{a\in[k]}\sum_{c\in[k]}(H^{(u)})_{ac}v^{(a,c)}, ( E ( n ) [ L ] ) u b ∈ [ k ] ∑ ξ b vec ( diag ( 1 C b ) ) = n 1 a ∈ [ k ] ∑ c ∈ [ k ] ∑ ( H ( u ) ) a c v ( a , c ) , where H ( 1 ) = W diag ( ξ ) W diag ( α − 1 ) H^{(1)}=W\operatorname{diag}(\xi)W\operatorname{diag}(\alpha^{-1}) H ( 1 ) = W diag ( ξ ) W diag ( α − 1 ) and, for u ≥ 1 u\ge1 u ≥ 1 ,
H ( u + 1 ) = W ( H ( u ) − 1 n Diag ( H ( u ) ) diag ( α − 1 ) ) W T . (47) H^{(u+1)}
= W\left(H^{(u)}-\frac{1}{n}\operatorname{Diag}(H^{(u)})\operatorname{diag}(\alpha^{-1})\right)W^{\mathsf{T}} . \tag*{(47)} H ( u + 1 ) = W ( H ( u ) − n 1 Diag ( H ( u ) ) diag ( α − 1 ) ) W T . (47)
Proof. We start by proving (i). For all i , j ∈ [ n ] i,j\in[n] i , j ∈ [ n ] , using the definition of L L L , the independence of ( A ℓ i ) ℓ ∈ [ n ] (A_{\ell i})_{\ell\in[n]} ( A ℓ i ) ℓ ∈ [ n ] and ( A ℓ j ) ℓ ∈ [ n ] (A_{\ell j})_{\ell\in[n]} ( A ℓ j ) ℓ ∈ [ n ] for i ≠ j i\ne j i = j , and the first equality of (44), we have
( E ( n ) [ L ] vec ( diag ( 1 C b ) ) ) i j = 1 { i ≠ j } ∑ ℓ ∈ [ n ] E ( n ) [ A ℓ i A ℓ j ] 1 { ℓ ∈ C b } = 1 { i ≠ j } ∑ ℓ ∈ [ n ] E ( n ) [ A i ℓ ] E ( n ) [ A ℓ j ] 1 { ℓ ∈ C b } . \begin{aligned}
\left(\mathbb{E}^{(n)}[L]\operatorname{vec}\left(\operatorname{diag}(1_{C_b})\right)\right)_{ij}
=1_{\{i\ne j\}}\sum_{\ell\in[n]}\mathbb{E}^{(n)}[A_{\ell i}A_{\ell j}]1_{\{\ell\in C_b\}} \\
&=1_{\{i\ne j\}}\sum_{\ell\in[n]}\mathbb{E}^{(n)}[A_{i\ell}]\mathbb{E}^{(n)}[A_{\ell j}]1_{\{\ell\in C_b\}}.
\end{aligned} ( E ( n ) [ L ] vec ( diag ( 1 C b ) ) ) ij = 1 { i = j } ℓ ∈ [ n ] ∑ E ( n ) [ A ℓ i A ℓ j ] 1 { ℓ ∈ C b } = 1 { i = j } ℓ ∈ [ n ] ∑ E ( n ) [ A i ℓ ] E ( n ) [ A ℓ j ] 1 { ℓ ∈ C b } .
By the second equality of (44), we have E ( n ) [ A i ℓ ] = 1 n W a b \mathbb{E}^{(n)}[A_{i\ell}]=\frac{1}{n}W_{ab} E ( n ) [ A i ℓ ] = n 1 W ab for i ∈ C a i\in C_a i ∈ C a , ℓ ∈ C b \ell\in C_b ℓ ∈ C b , and E ( n ) [ A ℓ j ] = 1 n W b c \mathbb{E}^{(n)}[A_{\ell j}]=\frac{1}{n}W_{bc} E ( n ) [ A ℓ j ] = n 1 W b c for ℓ ∈ C b \ell\in C_b ℓ ∈ C b , j ∈ C c j\in C_c j ∈ C c . Hence, using that ∣ C b ∣ = n α b \lvert C_b\rvert=n\alpha_b ∣ C b ∣ = n α b , we obtain that
∑ ℓ ∈ [ n ] E ( n ) [ A i ℓ ] E ( n ) [ A ℓ j ] 1 { ℓ ∈ C b } = ∑ a ∈ [ k ] ∑ c ∈ [ k ] 1 { i ∈ C a } 1 { j ∈ C c } 1 n 2 ∑ ℓ ∈ C b W a b W b c = 1 n ∑ a ∈ [ k ] ∑ c ∈ [ k ] W a b W b c α b ( 1 C a 1 C a T ) i j = 1 n ∑ a ∈ [ k ] ∑ c ∈ [ k ] W a b W b c α c − 1 ( 1 C a 1 C c T ) i j , (48) \begin{aligned}
\sum_{\ell\in[n]}\mathbb{E}^{(n)}[A_{i\ell}]\mathbb{E}^{(n)}[A_{\ell j}]1_{\{\ell\in C_b\}}
= \sum_{a\in[k]}\sum_{c\in[k]}1_{\{i\in C_a\}}1_{\{j\in C_c\}}\frac{1}{n^2}\sum_{\ell\in C_b}W_{ab}W_{bc} \\
&= \frac{1}{n}\sum_{a\in[k]}\sum_{c\in[k]}W_{ab}W_{bc}\alpha_b(1_{C_a}1_{C_a}^{\mathsf{T}})_{ij} \\
&= \frac{1}{n}\sum_{a\in[k]}\sum_{c\in[k]}W_{ab}W_{bc}\alpha_c^{-1}(1_{C_a}1_{C_c}^{\mathsf{T}})_{ij}, \tag*{(48)}
\end{aligned} ℓ ∈ [ n ] ∑ E ( n ) [ A i ℓ ] E ( n ) [ A ℓ j ] 1 { ℓ ∈ C b } = a ∈ [ k ] ∑ c ∈ [ k ] ∑ 1 { i ∈ C a } 1 { j ∈ C c } n 2 1 ℓ ∈ C b ∑ W ab W b c = n 1 a ∈ [ k ] ∑ c ∈ [ k ] ∑ W ab W b c α b ( 1 C a 1 C a T ) ij = n 1 a ∈ [ k ] ∑ c ∈ [ k ] ∑ W ab W b c α c − 1 ( 1 C a 1 C c T ) ij , (48)
where in the last equality we used W a b α b = W a b \mathcal{W}_{ab}\alpha_b=\mathcal{W}_{ab} W ab α b = W ab and W b c = W b c α c − 1 \mathcal{W}_{bc}=\mathcal{W}_{bc}\alpha_c^{-1} W b c = W b c α c − 1 , which follow from (42). Combining the above identities, we deduce that
( E ( n ) [ L ] vec ( diag ( 1 C b ) ) ) i j = 1 n ∑ a ∈ [ k ] ∑ c ∈ [ k ] W a b W b c α c − 1 1 { i ≠ j } ( 1 C a 1 C c T ) i j . \left(\mathbb{E}^{(n)}[L]\operatorname{vec}\left(\operatorname{diag}(\mathbf{1}_{C_b})\right)\right)_{ij}
=\frac{1}{n}\sum_{a\in[k]}\sum_{c\in[k]}\mathcal{W}_{ab}\mathcal{W}_{bc}\alpha_c^{-1}\mathbf{1}_{\{i\ne j\}}\left(\mathbf{1}_{C_a}\mathbf{1}_{C_c}^{\mathsf{T}}\right)_{ij}. ( E ( n ) [ L ] vec ( diag ( 1 C b ) ) ) ij = n 1 a ∈ [ k ] ∑ c ∈ [ k ] ∑ W ab W b c α c − 1 1 { i = j } ( 1 C a 1 C c T ) ij .
proving (i).
We now prove (ii). We only consider the case a = b a=b a = b . The case a ≠ b a\ne b a = b is treated similarly. Hence, suppose a = b a=b a = b . Take i , j ∈ [ n ] i,j\in[n] i , j ∈ [ n ] and note that, by the definition of v ( a , a ) v^{(a,a)} v ( a , a ) and the independence of ( A ℓ i ) ℓ ∈ [ n ] (A_{\ell i})_{\ell\in[n]} ( A ℓ i ) ℓ ∈ [ n ] and ( A h i ) h ∈ [ n ] (A_{hi})_{h\in[n]} ( A hi ) h ∈ [ n ] for i ≠ j i\ne j i = j ,
( E ( n ) [ L ] v ( a , a ) ) i j = 1 { i ≠ j } ∑ ( h , ℓ ) ∈ C a × C a 1 { h ≠ ℓ } E ( n ) [ A ℓ j A h i ] = 1 { i ≠ j } ∑ ( h , ℓ ) ∈ C a × C a 1 { h ≠ ℓ } E ( n ) [ A h i ] E ( n ) [ A ℓ j ] = 1 { i ≠ j } [ ( ∑ h ∈ C a E ( n ) [ A h i ] ) ( ∑ ℓ ∈ C a E ( n ) [ A ℓ j ] ) − ∑ h ∈ C a E ( n ) [ A h i ] E ( n ) [ A h j ] ] . \begin{aligned}
\left(\mathbb{E}^{(n)}[L]v^{(a,a)}\right)_{ij}
=\mathbf{1}_{\{i\ne j\}}\sum_{(h,\ell)\in C_a\times C_a}\mathbf{1}_{\{h\ne\ell\}}\mathbb{E}^{(n)}[A_{\ell j}A_{hi}] \\
&=\mathbf{1}_{\{i\ne j\}}\sum_{(h,\ell)\in C_a\times C_a}\mathbf{1}_{\{h\ne\ell\}}\mathbb{E}^{(n)}[A_{hi}]\mathbb{E}^{(n)}[A_{\ell j}] \\
&=\mathbf{1}_{\{i\ne j\}}\left[\left(\sum_{h\in C_a}\mathbb{E}^{(n)}[A_{hi}]\right)\left(\sum_{\ell\in C_a}\mathbb{E}^{(n)}[A_{\ell j}]\right)-\sum_{h\in C_a}\mathbb{E}^{(n)}[A_{hi}]\mathbb{E}^{(n)}[A_{hj}]\right].
\end{aligned} ( E ( n ) [ L ] v ( a , a ) ) ij = 1 { i = j } ( h , ℓ ) ∈ C a × C a ∑ 1 { h = ℓ } E ( n ) [ A ℓ j A hi ] = 1 { i = j } ( h , ℓ ) ∈ C a × C a ∑ 1 { h = ℓ } E ( n ) [ A hi ] E ( n ) [ A ℓ j ] = 1 { i = j } [ ( h ∈ C a ∑ E ( n ) [ A hi ] ) ( ℓ ∈ C a ∑ E ( n ) [ A ℓ j ] ) − h ∈ C a ∑ E ( n ) [ A hi ] E ( n ) [ A hj ] ] .
Next, using (44), we can check that for i ∈ C b i\in C_b i ∈ C b and j ∈ C c j\in C_c j ∈ C c ,
∑ h ∈ C a E ( n ) [ A h i ] = α a W a b and ∑ h ∈ C a E ( n ) [ A h i ] E ( n ) [ A h j ] = α a n W a b W a c . \sum_{h\in C_a}\mathbb{E}^{(n)}[A_{hi}]=\alpha_a\mathcal{W}_{ab}
\qquad\text{and}\qquad
\sum_{h\in C_a}\mathbb{E}^{(n)}[A_{hi}]\mathbb{E}^{(n)}[A_{hj}]
=\frac{\alpha_a}{n}\mathcal{W}_{ab}\mathcal{W}_{ac}. h ∈ C a ∑ E ( n ) [ A hi ] = α a W ab and h ∈ C a ∑ E ( n ) [ A hi ] E ( n ) [ A hj ] = n α a W ab W a c .
Putting together the above identities, we then obtain that
( E ( n ) [ L ] v ( a , a ) ) i j = ∑ b ∈ [ k ] ∑ c ∈ [ k ] ( α a 2 W a b W a c − α a n W a b W a c ) 1 { i ≠ j } ( 1 C b 1 C c T ) i j . \left(\mathbb{E}^{(n)}[L]v^{(a,a)}\right)_{ij}
=\sum_{b\in[k]}\sum_{c\in[k]}\left(\alpha_a^2\mathcal{W}_{ab}\mathcal{W}_{ac}-\frac{\alpha_a}{n}\mathcal{W}_{ab}\mathcal{W}_{ac}\right)\mathbf{1}_{\{i\ne j\}}\left(\mathbf{1}_{C_b}\mathbf{1}_{C_c}^{\mathsf{T}}\right)_{ij}. ( E ( n ) [ L ] v ( a , a ) ) ij = b ∈ [ k ] ∑ c ∈ [ k ] ∑ ( α a 2 W ab W a c − n α a W ab W a c ) 1 { i = j } ( 1 C b 1 C c T ) ij .
Using W b a = W a b α a \mathcal{W}_{ba}=\mathcal{W}_{ab}\alpha_a W ba = W ab α a and W c a = W a c α a \mathcal{W}_{ca}=\mathcal{W}_{ac}\alpha_a W c a = W a c α a (see (42) and the symmetry of W \mathcal{W} W ), the term inside the bracket is equal to W b a W c a ( 1 − 1 n α a ) \mathcal{W}_{ba}\mathcal{W}_{ca}\left(1-\frac{1}{n\alpha_a}\right) W ba W c a ( 1 − n α a 1 ) , whence
E ( n ) [ L ] v ( a , a ) = ∑ b ∈ [ k ] ∑ c ∈ [ k ] W b a W c a ( 1 − 1 n α a ) v ( b , c ) . \mathbb{E}^{(n)}[L]v^{(a,a)}
=\sum_{b\in[k]}\sum_{c\in[k]}\mathcal{W}_{ba}\mathcal{W}_{ca}\left(1-\frac{1}{n\alpha_a}\right)v^{(b,c)}. E ( n ) [ L ] v ( a , a ) = b ∈ [ k ] ∑ c ∈ [ k ] ∑ W ba W c a ( 1 − n α a 1 ) v ( b , c ) .
showing (ii) for a = b a=b a = b .
Finally, we prove (iii) by induction on u ≥ 1 u\ge1 u ≥ 1 . For u = 1 u=1 u = 1 , using (i), we have
E ( n ) [ L ] ( ∑ b ∈ [ k ] ξ b vec ( diag ( 1 C b ) ) ) = ∑ b ∈ [ k ] ξ b 1 n ∑ a ∈ [ k ] ∑ c ∈ [ k ] W a b W b c α c − 1 v ( a , c ) = 1 n ∑ a ∈ [ k ] ∑ c ∈ [ k ] ( ∑ b ∈ [ k ] W a b ξ b W b c α c − 1 ) v ( a , c ) = 1 n ∑ a ∈ [ k ] ∑ c ∈ [ k ] ( H ( 1 ) ) a c v ( a , c ) . \begin{aligned}
\mathbb{E}^{(n)}[L]\left(\sum_{b\in[k]}\xi_b\operatorname{vec}\left(\operatorname{diag}(\mathbf{1}_{C_b})\right)\right)
=\sum_{b\in[k]}\xi_b\frac{1}{n}\sum_{a\in[k]}\sum_{c\in[k]}\mathcal{W}_{ab}\mathcal{W}_{bc}\alpha_c^{-1}v^{(a,c)} \\
&=\frac{1}{n}\sum_{a\in[k]}\sum_{c\in[k]}\left(\sum_{b\in[k]}\mathcal{W}_{ab}\xi_b\mathcal{W}_{bc}\alpha_c^{-1}\right)v^{(a,c)} \\
&=\frac{1}{n}\sum_{a\in[k]}\sum_{c\in[k]}\left(H^{(1)}\right)_{ac}v^{(a,c)}.
\end{aligned} E ( n ) [ L ] b ∈ [ k ] ∑ ξ b vec ( diag ( 1 C b ) ) = b ∈ [ k ] ∑ ξ b n 1 a ∈ [ k ] ∑ c ∈ [ k ] ∑ W ab W b c α c − 1 v ( a , c ) = n 1 a ∈ [ k ] ∑ c ∈ [ k ] ∑ b ∈ [ k ] ∑ W ab ξ b W b c α c − 1 v ( a , c ) = n 1 a ∈ [ k ] ∑ c ∈ [ k ] ∑ ( H ( 1 ) ) a c v ( a , c ) .
with H ( 1 ) = W diag ( ξ ) W diag ( α − 1 ) H^{(1)}=W\operatorname{diag}(\boldsymbol{\xi})W\operatorname{diag}(\boldsymbol{\alpha}^{-1}) H ( 1 ) = W diag ( ξ ) W diag ( α − 1 ) . Next, suppose the identity holds for some u ≥ 1 u \ge1 u ≥ 1 . Using the inductive hypothesis and (ii), we obtain that
( E ( n ) [ L ] ) u + 1 ( ∑ b ∈ [ k ] ξ b vec ( diag ( 1 C b ) ) ) = 1 n ∑ a , c ∈ [ k ] ( H ( u ) ) a c E ( n ) [ L ] v ( a , c ) = 1 n ∑ c ′ , d ∈ [ k ] ( ∑ a , c ∈ [ k ] W c ′ a ( H ( u ) ) a c W d c ( 1 − 1 α a n 1 { a = c } ) ) v ( c ′ , d ) . \begin{aligned}
\left(\mathbb{E}^{(n)}[L]\right)^{u+1}\left(\sum_{b\in[k]}\xi_b\operatorname{vec}\left(\operatorname{diag}(\mathbf{1}_{C_b})\right)\right)
=\frac{1}{n}\sum_{a,c\in[k]} \left(H^{(u)}\right)_{ac}\mathbb{E}^{(n)}[L]v^{(a,c)} \\
&=\frac{1}{n}\sum_{c',d\in[k]}\left(\sum_{a,c\in[k]}W_{c'a}\left(H^{(u)}\right)_{ac}W_{dc}\left(1-\frac{1}{\alpha_a n}\mathbf{1}_{\{a=c\}}\right)\right)v^{(c',d)}.
\end{aligned} ( E ( n ) [ L ] ) u + 1 b ∈ [ k ] ∑ ξ b vec ( diag ( 1 C b ) ) = n 1 a , c ∈ [ k ] ∑ ( H ( u ) ) a c E ( n ) [ L ] v ( a , c ) = n 1 c ′ , d ∈ [ k ] ∑ a , c ∈ [ k ] ∑ W c ′ a ( H ( u ) ) a c W d c ( 1 − α a n 1 1 { a = c } ) v ( c ′ , d ) .
Observing that
∑ a , c ∈ [ k ] W c ′ a ( H ( u ) ) a c W d c ( 1 − 1 α a n 1 { a = c } ) = ( W H ( u ) W T − 1 n W Diag ( H ( u ) ) diag ( α − 1 ) W T ) c ′ d = H c ′ d ( u + 1 ) , \begin{aligned}
\sum_{a,c\in[k]}W_{c'a}\left(H^{(u)}\right)_{ac}W_{dc}\left(1-\frac{1}{\alpha_a n}\mathbf{1}_{\{a=c\}}\right)
=\left(WH^{(u)}W^{\mathsf{T}}-\frac{1}{n}W\operatorname{Diag}\left(H^{(u)}\right)\operatorname{diag}\left(\boldsymbol{\alpha}^{-1}\right)W^{\mathsf{T}}\right)_{c'd} \\
&=H^{(u+1)}_{c'd},
\end{aligned} a , c ∈ [ k ] ∑ W c ′ a ( H ( u ) ) a c W d c ( 1 − α a n 1 1 { a = c } ) = ( W H ( u ) W T − n 1 W Diag ( H ( u ) ) diag ( α − 1 ) W T ) c ′ d = H c ′ d ( u + 1 ) ,
we then conclude that
( E ( n ) [ L ] ) u + 1 ( ∑ b ∈ [ k ] ξ b vec ( diag ( 1 C b ) ) ) = 1 n ∑ c ′ , d ∈ [ k ] H c ′ d ( u + 1 ) v ( c ′ , d ) . \left(\mathbb{E}^{(n)}[L]\right)^{u+1}\left(\sum_{b\in[k]}\xi_b\operatorname{vec}\left(\operatorname{diag}(\mathbf{1}_{C_b})\right)\right)
=\frac{1}{n}\sum_{c',d\in[k]}H^{(u+1)}_{c'd}v^{(c',d)}. ( E ( n ) [ L ] ) u + 1 b ∈ [ k ] ∑ ξ b vec ( diag ( 1 C b ) ) = n 1 c ′ , d ∈ [ k ] ∑ H c ′ d ( u + 1 ) v ( c ′ , d ) .
proving (iii).
Proof of Lemma 3.1
In this section, we prove Lemma 3.1.
Proof of Lemma 3.1. Let us begin observing that
P ( n ) ( ( Θ , H ) ∈ Ω a d m ) = P ( n ) ( ( Θ , H ) : λ ∈ ( ∥ A ( Θ , H ) ∥ ∞ , 1 − ∥ A ( Θ , H ) T ∥ ∞ ) ) . \mathbb{P}^{(n)}\left((\Theta,H)\in\Omega^{\mathrm{adm}}\right)
=\mathbb{P}^{(n)}\left((\Theta,H):\lambda\in\left(\left\lVert A_{(\Theta,H)}\right\rVert_{\infty},1-\left\lVert A^{\mathsf{T}}_{(\Theta,H)}\right\rVert_{\infty}\right)\right). P ( n ) ( ( Θ , H ) ∈ Ω adm ) = P ( n ) ( ( Θ , H ) : λ ∈ ( A ( Θ , H ) ∞ , 1 − A ( Θ , H ) T ∞ ) ) .
Since H i j ∈ { − 1 , + 1 } H_{ij}\in\{-1,+1\} H ij ∈ { − 1 , + 1 } , in order to show that λ ∈ ( ∥ A ( Θ , H ) T ∥ ∞ , 1 − ∥ A ( Θ , H ) T ∥ ∞ ) \lambda\in\left(\left\lVert A^{\mathsf{T}}_{(\Theta,H)}\right\rVert_{\infty},1-\left\lVert A^{\mathsf{T}}_{(\Theta,H)}\right\rVert_{\infty}\right) λ ∈ ( A ( Θ , H ) T ∞ , 1 − A ( Θ , H ) T ∞ ) it suffices that
max a ∈ [ k ] max i ∈ C a { μ a n ∑ j : j ∼ a i Θ j i + μ ≁ n ∑ j : j ≁ i Θ j i } < λ < 1 − max a ∈ [ k ] max i ∈ C a { μ a n ∑ j : j ∼ a i Θ j i + μ ≁ n ∑ j : j ≁ i Θ j i } . (49) \max_{a\in[k]}\max_{i\in C_a}\left\{\frac{\mu_a}{n}\sum_{j:j\overset{a}{\sim}i}\Theta_{ji}+\frac{\mu_{\not\sim}}{n}\sum_{j:j\not\sim i}\Theta_{ji}\right\}<\lambda<1-\max_{a\in[k]}\max_{i\in C_a}\left\{\frac{\mu_a}{n}\sum_{j:j\overset{a}{\sim}i}\Theta_{ji}+\frac{\mu_{\not\sim}}{n}\sum_{j:j\not\sim i}\Theta_{ji}\right\}. \tag*{(49)} a ∈ [ k ] max i ∈ C a max ⎩ ⎨ ⎧ n μ a j : j ∼ a i ∑ Θ j i + n μ ∼ j : j ∼ i ∑ Θ j i ⎭ ⎬ ⎫ < λ < 1 − a ∈ [ k ] max i ∈ C a max ⎩ ⎨ ⎧ n μ a j : j ∼ a i ∑ Θ j i + n μ ∼ j : j ∼ i ∑ Θ j i ⎭ ⎬ ⎫ . (49)
Using that E ( n ) [ Θ j i ] = p a \mathbb{E}^{(n)}[\Theta_{ji}]=p_a E ( n ) [ Θ j i ] = p a if i ∼ a j i\overset{a}{\sim}j i ∼ a j for a ∈ [ k ] a\in[k] a ∈ [ k ] and E ( n ) [ Θ j i ] = q \mathbb{E}^{(n)}[\Theta_{ji}]=q E ( n ) [ Θ j i ] = q if i ≁ j i\not\sim j i ∼ j , and defining for a ∈ [ k ] a\in[k] a ∈ [ k ] and i ∈ C a i\in C_a i ∈ C a
Y a , i : = μ a n ∑ j : j ∼ a i ( Θ j i − p a ) + μ ≁ n ∑ j : j ≁ i ( Θ j i − q ) , Y_{a,i}:=\frac{\mu_a}{n}\sum_{j:j\overset{a}{\sim}i}(\Theta_{ji}-p_a)+\frac{\mu_{\not\sim}}{n}\sum_{j:j\not\sim i}(\Theta_{ji}-q), Y a , i := n μ a j : j ∼ a i ∑ ( Θ j i − p a ) + n μ ∼ j : j ∼ i ∑ ( Θ j i − q ) ,
we can rewrite (48) (recall that ∣ C a ∣ = α a n \lvert C_a\rvert=\alpha_a n ∣ C a ∣ = α a n ) as
max a ∈ [ k ] { max i ∈ C a Y a , i + μ a α a p a + μ ≁ ( 1 − α a ) q } < λ < 1 − max a ∈ [ k ] { max i ∈ C a Y a , i + μ a α a p a + μ ≁ ( 1 − α k ) q } . \max_{a\in[k]}\left\{\max_{i\in C_a}Y_{a,i}+\mu_a\alpha_a p_a+\mu_{\not\sim}(1-\alpha_a)q\right\}<\lambda<1-\max_{a\in[k]}\left\{\max_{i\in C_a}Y_{a,i}+\mu_a\alpha_a p_a+\mu_{\not\sim}(1-\alpha_k)q\right\}. a ∈ [ k ] max { i ∈ C a max Y a , i + μ a α a p a + μ ∼ ( 1 − α a ) q } < λ < 1 − a ∈ [ k ] max { i ∈ C a max Y a , i + μ a α a p a + μ ∼ ( 1 − α k ) q } .
Setting M = max a ∈ [ k ] { μ a α a p a + μ ≁ ( 1 − α a ) q } M=\max_{a\in[k]}\{\mu_a\alpha_a p_a+\mu_{\not\sim}(1-\alpha_a)q\} M = max a ∈ [ k ] { μ a α a p a + μ ∼ ( 1 − α a ) q } , we obtain that to verify the claim, it is enough to show
P ( n ) ( M + max a ∈ [ k ] max i ∈ C a Y a , i < λ < 1 − M − max a ∈ [ k ] max i ∈ C a Y a , i ) ≥ 1 − 2 n e − c n . (50) \mathbb{P}^{(n)}\left(M+\max_{a\in[k]}\max_{i\in C_a}Y_{a,i}<\lambda<1-M-\max_{a\in[k]}\max_{i\in C_a}Y_{a,i}\right)\ge1-2ne^{-c n}. \tag*{(50)} P ( n ) ( M + a ∈ [ k ] max i ∈ C a max Y a , i < λ < 1 − M − a ∈ [ k ] max i ∈ C a max Y a , i ) ≥ 1 − 2 n e − c n . (50)
where c c c is a constant c > 0 c > 0 c > 0 independent of n n n , depending on the parameters λ \lambda λ , ( α a ) a ∈ [ k ] (\alpha_a)_{a\in[k]} ( α a ) a ∈ [ k ] , ( μ a ) a ∈ [ k ] (\mu_a)_{a\in[k]} ( μ a ) a ∈ [ k ] , μ ℵ \mu_{\aleph} μ ℵ , ( p a ) a ∈ [ k ] (p_a)_{a\in[k]} ( p a ) a ∈ [ k ] and q q q .
Observe that, for any fixed a ∈ [ k ] a \in[k] a ∈ [ k ] and any i ∈ C a i \in C_a i ∈ C a , the random variables { μ a n ( Θ j i − p a ) } j ∉ C a \left\{\frac{\mu_a}{n}(\Theta_{ji}-p_a)\right\}_{j\notin C_a} { n μ a ( Θ j i − p a ) } j ∈ / C a are i.i.d. and assume values in the bounded interval [ − μ a n p a , μ a n ( 1 − p a ) ] \left[-\frac{\mu_a}{n}p_a,\frac{\mu_a}{n}(1-p_a)\right] [ − n μ a p a , n μ a ( 1 − p a ) ] . Thus, by Hoeffding’s Lemma, they are subgaussian with variance factor μ a 2 4 n 2 \frac{\mu_a^2}{4n^2} 4 n 2 μ a 2 . Moreover, by independence, we have that μ a n ∑ j ∉ C a ( Θ j i − p a ) \frac{\mu_a}{n}\sum_{j\notin C_a}(\Theta_{ji}-p_a) n μ a ∑ j ∈ / C a ( Θ j i − p a ) is subgaussian with variance factor μ a 2 4 n 2 ∣ C a ∣ \frac{\mu_a^2}{4n^2}|C_a| 4 n 2 μ a 2 ∣ C a ∣ . Similarly, we have that μ ℵ n ∑ j ≻ i ( Θ j i − q ) \frac{\mu_{\aleph}}{n}\sum_{j\succ i}(\Theta_{ji}-q) n μ ℵ ∑ j ≻ i ( Θ j i − q ) is subgaussian with variance factor μ ℵ 2 4 n 2 ( n − ∣ C a ∣ ) \frac{\mu_{\aleph}^2}{4n^2}(n-|C_a|) 4 n 2 μ ℵ 2 ( n − ∣ C a ∣ ) . Using again independence, we obtain that for any a ∈ [ k ] a \in[k] a ∈ [ k ] and i ∈ C a i \in C_a i ∈ C a the random variable Y a , i Y_{a,i} Y a , i is subgaussian with parameter μ a 2 4 n α a + μ ℵ 2 4 n ( 1 − α a ) \frac{\mu_a^2}{4n}\alpha_a+\frac{\mu_{\aleph}^2}{4n}(1-\alpha_a) 4 n μ a 2 α a + 4 n μ ℵ 2 ( 1 − α a ) . Setting σ 2 : = max a ∈ [ k ] { μ a 2 α a + μ ℵ 2 ( 1 − α a ) } \sigma^2:=\max_{a\in[k]}\{\mu_a^2\alpha_a+\mu_{\aleph}^2(1-\alpha_a)\} σ 2 := max a ∈ [ k ] { μ a 2 α a + μ ℵ 2 ( 1 − α a )} , we obtain that the random variables { Y a , i } a ∈ [ k ] , i ∈ C a \{Y_{a,i}\}_{a\in[k],i\in C_a} { Y a , i } a ∈ [ k ] , i ∈ C a are subgaussians with variance factor 1 4 n σ 2 \frac{1}{4n}\sigma^2 4 n 1 σ 2 . Moreover, E ( n ) [ Y a , i ] = 0 \mathbb{E}^{(n)}[Y_{a,i}]=0 E ( n ) [ Y a , i ] = 0 , for all a ∈ [ k ] a\in[k] a ∈ [ k ] and all i ∈ C a i\in C_a i ∈ C a . Thus, for all for all a ∈ [ k ] a\in[k] a ∈ [ k ] , i ∈ C a i\in C_a i ∈ C a and all y > 0 y>0 y > 0 , it holds that
P ( n ) ( Y a , i > y ) ≤ e − y 2 4 n 2 σ 2 ⟹ Union bound P ( n ) ( max a ∈ [ k ] max i ∈ C a Y a , i > y ) ≤ n e − y 2 4 n 2 σ 2 . \mathbb{P}^{(n)}(Y_{a,i}>y)\leq e^{-\frac{y^2 4n}{2\sigma^2}}
\quad\underset{\text{Union bound}}{\Longrightarrow}\quad
\mathbb{P}^{(n)}\left(\max_{a\in[k]}\max_{i\in C_a}Y_{a,i}>y\right)\leq ne^{-\frac{y^2 4n}{2\sigma^2}}. P ( n ) ( Y a , i > y ) ≤ e − 2 σ 2 y 2 4 n Union bound ⟹ P ( n ) ( a ∈ [ k ] max i ∈ C a max Y a , i > y ) ≤ n e − 2 σ 2 y 2 4 n .
By the assumption M < λ < 1 − M M<\lambda<1-M M < λ < 1 − M , setting first y = λ − M > 0 y=\lambda-M>0 y = λ − M > 0 and then y = 1 − M − λ > 0 y=1-M-\lambda>0 y = 1 − M − λ > 0 , we obtain
P ( n ) ( M + max a ∈ [ k ] max i ∈ C a Y a , i > λ ) ≤ n e − ( λ − M ) 2 4 n 2 σ 2 , \mathbb{P}^{(n)}\left(M+\max_{a\in[k]}\max_{i\in C_a}Y_{a,i}>\lambda\right)
\leq ne^{-\frac{(\lambda-M)^2 4n}{2\sigma^2}}, P ( n ) ( M + a ∈ [ k ] max i ∈ C a max Y a , i > λ ) ≤ n e − 2 σ 2 ( λ − M ) 2 4 n ,
P ( n ) ( λ > 1 − M − max a ∈ [ k ] max i ∈ C a Y a , i ) ≤ n e − ( 1 − M − λ ) 2 4 n 2 σ 2 , \mathbb{P}^{(n)}\left(\lambda>1-M-\max_{a\in[k]}\max_{i\in C_a}Y_{a,i}\right)
\leq ne^{-\frac{(1-M-\lambda)^2 4n}{2\sigma^2}}, P ( n ) ( λ > 1 − M − a ∈ [ k ] max i ∈ C a max Y a , i ) ≤ n e − 2 σ 2 ( 1 − M − λ ) 2 4 n ,
which combined imply (50) .
Community identifiability
In this section, we provide the proofs of the results of § 3.2 .
We start proving Lemma 3.2.
Proof of Lemma 3.2. Suppose W 1 k = c 1 k W\mathbf{1}_k=c\mathbf{1}_k W 1 k = c 1 k . Since (15) is granted, then necessarily ∣ c ∣ < 1 |c|<1 ∣ c ∣ < 1 . In this case, we have W u 1 k = c u 1 k W^u\mathbf{1}_k=c^u\mathbf{1}_k W u 1 k = c u 1 k for all u ≥ 0 u\geq0 u ≥ 0 , so that ζ = ( ∑ u = 0 ∞ c u ) 1 k = ( 1 − c ) − 1 1 k \zeta=\left(\sum_{u=0}^{\infty}c^u\right)\mathbf{1}_k=(1-c)^{-1}\mathbf{1}_k ζ = ( ∑ u = 0 ∞ c u ) 1 k = ( 1 − c ) − 1 1 k . For the other direction, suppose ζ = c 1 k \zeta=c\mathbf{1}_k ζ = c 1 k . Since (15) is granted, ζ \zeta ζ solves the equation ζ = 1 k + W ζ \zeta=\mathbf{1}_k+W\zeta ζ = 1 k + W ζ . It then follows that c 1 k = 1 k + c W 1 k c\mathbf{1}_k=\mathbf{1}_k+cW\mathbf{1}_k c 1 k = 1 k + c W 1 k , and (15) assures that c > 0 c>0 c > 0 . Then, we must have that W 1 k = c − 1 c 1 k W\mathbf{1}_k=\frac{c-1}{c}\mathbf{1}_k W 1 k = c c − 1 1 k .
Next, we prove Proposition 3.1. In the proof, we shall use the following proposition, which is proved at the end of this section.
Proposition F.1. Let W W W be the matrix defined in (16) . Suppose (15) and (21) are granted. Denote ξ = λ ζ 1 ( 1 − λ ζ 1 ) \xi=\lambda\zeta_1(1-\lambda\zeta_1) ξ = λ ζ 1 ( 1 − λ ζ 1 ) . Let γ \gamma γ and ρ ℵ \rho_{\aleph} ρ ℵ as defined in (23) . Then, the matrix Ψ \Psi Ψ defined in (19) satisfies
Ψ = ξ 1 G ≥ 1 ( W 2 ) diag ( α − 1 ) = ξ 1 ( γ diag ( α − 1 ) + ρ ℵ 1 k ( 1 k ) T ) , (51) \Psi=\xi_1\mathcal{G}_{\geq1}(W^2)\operatorname{diag}(\alpha^{-1})
=\xi_1\left(\gamma\operatorname{diag}(\alpha^{-1})+\rho_{\aleph}\mathbf{1}_k(\mathbf{1}_k)^\mathsf{T}\right), \tag*{(51)} Ψ = ξ 1 G ≥ 1 ( W 2 ) diag ( α − 1 ) = ξ 1 ( γ diag ( α − 1 ) + ρ ℵ 1 k ( 1 k ) T ) , (51)
where α − 1 = ( α 1 − 1 , … , α k − 1 ) \alpha^{-1}=(\alpha_1^{-1},\ldots,\alpha_k^{-1}) α − 1 = ( α 1 − 1 , … , α k − 1 ) and ξ 1 \xi_1 ξ 1 is defined in (18) .
Proof of Proposition 3.1. First, summing W a b = ( 2 β − 1 ) ( μ a α b p b 1 { a = b } + μ ≁ α b q 1 { a ≠ b } ) W_{ab}=(2\beta-1)(\mu_a\alpha_b p_b \mathbf{1}_{\{a=b\}}+\mu_{\not\sim}\alpha_b q\mathbf{1}_{\{a\ne b\}}) W ab = ( 2 β − 1 ) ( μ a α b p b 1 { a = b } + μ ∼ α b q 1 { a = b } ) over b ∈ [ k ] b\in[k] b ∈ [ k ] and using that μ a α a p a + μ ≁ q ( 1 − α a ) = κ + μ ≁ q = M \mu_a\alpha_a p_a+\mu_{\not\sim}q(1-\alpha_a)=\kappa+\mu_{\not\sim}q=M μ a α a p a + μ ∼ q ( 1 − α a ) = κ + μ ∼ q = M for all a ∈ [ k ] a\in[k] a ∈ [ k ] (this holds by (21) ), we deduce that ( W 1 k ) a = ( 2 β − 1 ) M (W\mathbf{1}_k)_a=(2\beta-1)M ( W 1 k ) a = ( 2 β − 1 ) M for all a ∈ [ k ] a\in[k] a ∈ [ k ] , i.e. W 1 k = ( 2 β − 1 ) M 1 k W\mathbf{1}_k=(2\beta-1)M\mathbf{1}_k W 1 k = ( 2 β − 1 ) M 1 k . As a consequence of this identity, we obtain that for all t ≥ 0 t\ge0 t ≥ 0 ,
W t 1 k 1 k T = ( 2 β − 1 ) t M t 1 k 1 k T . (52) W^t\mathbf{1}_k\mathbf{1}_k^{\mathsf T}=(2\beta-1)^tM^t\mathbf{1}_k\mathbf{1}_k^{\mathsf T}. \tag*{(52)} W t 1 k 1 k T = ( 2 β − 1 ) t M t 1 k 1 k T . (52)
Second, observe that by (43) , we have that W diag ( α − 1 ) = W W\operatorname{diag}(\alpha^{-1})=\mathcal{W} W diag ( α − 1 ) = W . Using this, (41) and the fact that μ ≁ q = M − κ \mu_{\not\sim}q=M-\kappa μ ∼ q = M − κ , together with μ a p a − μ ≁ q = κ α a − 1 \mu_a p_a-\mu_{\not\sim}q=\kappa\alpha_a^{-1} μ a p a − μ ∼ q = κ α a − 1 (which holds by (21) ), we then deduce that
W diag ( α − 1 ) = W = ( 2 β − 1 ) κ diag ( α − 1 ) + ( 2 β − 1 ) ( M − κ ) 1 k 1 k T . (53) W\operatorname{diag}(\alpha^{-1})=\mathcal{W}=(2\beta-1)\kappa\operatorname{diag}(\alpha^{-1})+(2\beta-1)(M-\kappa)\mathbf{1}_k\mathbf{1}_k^{\mathsf T}. \tag*{(53)} W diag ( α − 1 ) = W = ( 2 β − 1 ) κ diag ( α − 1 ) + ( 2 β − 1 ) ( M − κ ) 1 k 1 k T . (53)
Claim. For every integer t ≥ 0 t\ge0 t ≥ 0 ,
W t diag ( α − 1 ) = ( 2 β − 1 ) t κ t diag ( α − 1 ) + ( 2 β − 1 ) t ( M t − κ t ) 1 k 1 k T . (54) W^t\operatorname{diag}(\alpha^{-1})=(2\beta-1)^t\kappa^t\operatorname{diag}(\alpha^{-1})+(2\beta-1)^t(M^t-\kappa^t)\mathbf{1}_k\mathbf{1}_k^{\mathsf T}. \tag*{(54)} W t diag ( α − 1 ) = ( 2 β − 1 ) t κ t diag ( α − 1 ) + ( 2 β − 1 ) t ( M t − κ t ) 1 k 1 k T . (54) Proof of the Claim. We argue by induction on t t t . The case t = 0 t=0 t = 0 is immediate. Moreover, note that the case t = 1 t=1 t = 1 is (53) . Now, assume (54) holds for some t ≥ 1 t\ge1 t ≥ 1 . Using that W diag ( α − 1 ) = W W\operatorname{diag}(\alpha^{-1})=\mathcal{W} W diag ( α − 1 ) = W , (53) and (52) with t = 0 t=0 t = 0 , we obtain
W t + 1 diag ( α − 1 ) = W ( W t diag ( α − 1 ) ) = W ( ( 2 β − 1 ) t κ t diag ( α − 1 ) + ( 2 β − 1 ) t ( M t − κ t ) 1 k 1 k T ) = ( 2 β − 1 ) t κ t W + ( 2 β − 1 ) t ( M t − κ t ) ( 2 β − 1 ) M 1 k 1 k T = ( 2 β − 1 ) t + 1 κ t + 1 diag ( α − 1 ) + ( 2 β − 1 ) t + 1 [ κ t ( M − κ ) + M ( M t − κ t ) ] 1 k 1 k T . \begin{aligned}
W^{t+1}\operatorname{diag}(\alpha^{-1})
&=W\left(W^t\operatorname{diag}(\alpha^{-1})\right)\\
&=W\left((2\beta-1)^t\kappa^t\operatorname{diag}(\alpha^{-1})+(2\beta-1)^t(M^t-\kappa^t)\mathbf{1}_k\mathbf{1}_k^{\mathsf T}\right)\\
&=(2\beta-1)^t\kappa^t\mathcal{W}+(2\beta-1)^t(M^t-\kappa^t)(2\beta-1)M\mathbf{1}_k\mathbf{1}_k^{\mathsf T}\\
&=(2\beta-1)^{t+1}\kappa^{t+1}\operatorname{diag}(\alpha^{-1})+(2\beta-1)^{t+1}\left[\kappa^t(M-\kappa)+M(M^t-\kappa^t)\right]\mathbf{1}_k\mathbf{1}_k^{\mathsf T}.
\end{aligned} W t + 1 diag ( α − 1 ) = W ( W t diag ( α − 1 ) ) = W ( ( 2 β − 1 ) t κ t diag ( α − 1 ) + ( 2 β − 1 ) t ( M t − κ t ) 1 k 1 k T ) = ( 2 β − 1 ) t κ t W + ( 2 β − 1 ) t ( M t − κ t ) ( 2 β − 1 ) M 1 k 1 k T = ( 2 β − 1 ) t + 1 κ t + 1 diag ( α − 1 ) + ( 2 β − 1 ) t + 1 [ κ t ( M − κ ) + M ( M t − κ t ) ] 1 k 1 k T . Combining the above identity with the fact that κ t ( M − κ ) + M ( M t − κ t ) = M t + 1 − κ t + 1 \kappa^t(M-\kappa)+M(M^t-\kappa^t)=M^{t+1}-\kappa^{t+1} κ t ( M − κ ) + M ( M t − κ t ) = M t + 1 − κ t + 1 , the claim for t + 1 t+1 t + 1 follows.
Next, combining (51) , (54) and (52) , we obtain that for all integer t ≥ 0 t\ge0 t ≥ 0 ,
W t Ψ = ξ 1 γ W t diag ( α − 1 ) + ξ 1 ρ ≁ W t 1 k 1 k T = ξ 1 ( 2 β − 1 ) t κ t γ diag ( α − 1 ) + ξ 1 ( 2 β − 1 ) t ( γ ( M t − κ t ) + ρ ≁ M t ) 1 k 1 k T , \begin{aligned}
W^t\Psi&=\xi_1\gamma W^t\operatorname{diag}(\alpha^{-1})+\xi_1\rho_{\not\sim}W^t\mathbf{1}_k\mathbf{1}_k^{\mathsf T}\\
&=\xi_1(2\beta-1)^t\kappa^t\gamma\operatorname{diag}(\alpha^{-1})+\xi_1(2\beta-1)^t\left(\gamma(M^t-\kappa^t)+\rho_{\not\sim}M^t\right)\mathbf{1}_k\mathbf{1}_k^{\mathsf T},
\end{aligned} W t Ψ = ξ 1 γ W t diag ( α − 1 ) + ξ 1 ρ ∼ W t 1 k 1 k T = ξ 1 ( 2 β − 1 ) t κ t γ diag ( α − 1 ) + ξ 1 ( 2 β − 1 ) t ( γ ( M t − κ t ) + ρ ∼ M t ) 1 k 1 k T ,
that is, for all a , b ∈ [ k ] a,b\in[k] a , b ∈ [ k ] ,
( W t Ψ ) a b = ξ 1 ( 2 β − 1 ) t κ t γ α a − 1 1 { a = b } + ξ 1 ( 2 β − 1 ) t ( γ ( M t − κ t ) + ρ ≁ M t ) . (55) (W^t\Psi)_{ab}=\xi_1(2\beta-1)^t\kappa^t\gamma\alpha_a^{-1}\mathbf{1}_{\{a=b\}}+\xi_1(2\beta-1)^t\left(\gamma(M^t-\kappa^t)+\rho_{\not\sim}M^t\right). \tag*{(55)} ( W t Ψ ) ab = ξ 1 ( 2 β − 1 ) t κ t γ α a − 1 1 { a = b } + ξ 1 ( 2 β − 1 ) t ( γ ( M t − κ t ) + ρ ∼ M t ) . (55)
Substituting (55) into 1 n ∑ a , b ∈ [ k ] ( W t Ψ ) a b 1 C a 1 C b T \frac{1}{n}\sum_{a,b\in[k]}(W^t\Psi)_{ab}\mathbf{1}_{C_a}\mathbf{1}_{C_b}^{\mathsf T} n 1 ∑ a , b ∈ [ k ] ( W t Ψ ) ab 1 C a 1 C b T and using ∑ a , b 1 C a 1 C b T = 1 n 1 n T \sum_{a,b}\mathbf{1}_{C_a}\mathbf{1}_{C_b}^{\mathsf T}=\mathbf{1}_n\mathbf{1}_n^{\mathsf T} ∑ a , b 1 C a 1 C b T = 1 n 1 n T , one can then check that for all integer t ≥ 0 t\ge0 t ≥ 0 ,
1 n ∑ a , b ∈ [ k ] ( W t Ψ ) a b 1 C a 1 C b T = ξ 1 ( S ( t ) + B ‾ ( t ) ) , t ≥ 0. (56) \frac{1}{n}\sum_{a,b\in[k]}(W^t\Psi)_{ab}\mathbf{1}_{C_a}\mathbf{1}_{C_b}^{\mathsf T}=\xi_1\left(S^{(t)}+\overline{B}^{(t)}\right),\qquad t\ge0. \tag*{(56)} n 1 a , b ∈ [ k ] ∑ ( W t Ψ ) ab 1 C a 1 C b T = ξ 1 ( S ( t ) + B ( t ) ) , t ≥ 0. (56)
Since Σ ‾ ( 0 ) \overline{\Sigma}^{(0)} Σ ( 0 ) corresponds to the left-hand side of (56) with t = 0 t=0 t = 0 (note that W 0 = I k W^0=I_k W 0 = I k ), we conclude that Σ ‾ ( 0 ) = ξ 1 ( S ( 0 ) + B ‾ ( 0 ) ) \overline{\Sigma}^{(0)}=\xi_1(S^{(0)}+\overline{B}^{(0)}) Σ ( 0 ) = ξ 1 ( S ( 0 ) + B ( 0 ) ) .
Finally, note that under (21) the entries of ζ \zeta ζ are all equal (see Lemma 3.2), so that ξ = ξ 1 1 k \xi=\xi_1\mathbf{1}_k ξ = ξ 1 1 k and v ˉ = ξ 1 1 n \bar{v}=\xi_1\mathbf{1}_n v ˉ = ξ 1 1 n , which in turn implies diag ( v ˉ ) = ξ 1 I n \operatorname{diag}(\bar{v})=\xi_1 I_n diag ( v ˉ ) = ξ 1 I n . Using this and (56) for t ≥ 1 t\geq1 t ≥ 1 , and recalling the definition of Σ ˉ ( Θ , H ) ( t ) \bar{\Sigma}_{(\Theta,H)}^{(t)} Σ ˉ ( Θ , H ) ( t ) in (5) , we obtain that
Σ ˉ ( Θ , H ) ( t ) = ( A ⊤ ) t diag ( v ˉ ) + 1 n ∑ a , b ( W t Ψ ) a b 1 C a 1 C b ⊤ = ξ 1 ( ( A ⊤ ) t + S ( t ) + B ˉ ( t ) ) , \bar{\Sigma}_{(\Theta,H)}^{(t)}=(A^{\top})^t\operatorname{diag}(\bar{v})+\frac{1}{n}\sum_{a,b}(W^t\Psi)_{ab}\mathbf{1}_{C_a}\mathbf{1}_{C_b}^{\top}=\xi_1\left((A^{\top})^t+S^{(t)}+\bar{B}^{(t)}\right), Σ ˉ ( Θ , H ) ( t ) = ( A ⊤ ) t diag ( v ˉ ) + n 1 a , b ∑ ( W t Ψ ) ab 1 C a 1 C b ⊤ = ξ 1 ( ( A ⊤ ) t + S ( t ) + B ˉ ( t ) ) ,
and the result follows.
Finally, we prove Proposition F.1.
Proof of Proposition F.1. We will begin showing that Ψ = ξ 1 G ≥ 1 ( W 2 ) diag ( α − 1 ) \Psi=\xi_1\mathcal{G}_{\geq1}(W^2)\operatorname{diag}(\alpha^{-1}) Ψ = ξ 1 G ≥ 1 ( W 2 ) diag ( α − 1 ) . To show this identity, first note that under Condition (21) , we have ξ = ξ 1 1 k \xi=\xi_1\mathbf{1}_k ξ = ξ 1 1 k . Then, use this and (43) to obtain that
Ψ = ξ 1 ∑ u = 1 ∞ W u diag ( α − 1 ) ( W ⊤ ) u = ξ 1 ∑ u = 1 ∞ W 2 u diag ( α − 1 ) = ξ 1 G ≥ 1 ( W 2 ) diag ( α − 1 ) , \Psi=\xi_1\sum_{u=1}^{\infty}W^{u}\operatorname{diag}(\alpha^{-1})(W^{\top})^u=\xi_1\sum_{u=1}^{\infty}W^{2u}\operatorname{diag}(\alpha^{-1})=\xi_1\mathcal{G}_{\geq1}(W^2)\operatorname{diag}(\alpha^{-1}), Ψ = ξ 1 u = 1 ∑ ∞ W u diag ( α − 1 ) ( W ⊤ ) u = ξ 1 u = 1 ∑ ∞ W 2 u diag ( α − 1 ) = ξ 1 G ≥ 1 ( W 2 ) diag ( α − 1 ) ,
concluding the first half of the proof.
Hence, to conclude the proof, it suffices to show that G ≥ 1 ( W 2 ) diag ( α − 1 ) = γ diag ( α − 1 ) + ρ ≁ 1 k ( 1 k ) ⊤ \mathcal{G}_{\geq1}(W^2)\operatorname{diag}(\alpha^{-1})=\gamma\operatorname{diag}(\alpha^{-1})+\rho_{\not\sim}\mathbf{1}_k(\mathbf{1}_k)^{\top} G ≥ 1 ( W 2 ) diag ( α − 1 ) = γ diag ( α − 1 ) + ρ ∼ 1 k ( 1 k ) ⊤ . This is can be done as follows.
First of all, let us check that the matrix W 2 diag ( α − 1 ) W^2\operatorname{diag}(\alpha^{-1}) W 2 diag ( α − 1 ) is invertible. To that end, first observe that by (54) with t = 2 t=2 t = 2 , we can rewrite W 2 diag ( α − 1 ) W^2\operatorname{diag}(\alpha^{-1}) W 2 diag ( α − 1 ) as
W 2 diag ( α − 1 ) = ( 2 β − 1 ) 2 κ 2 diag ( α − 1 ) + ϕ ≁ 1 k ( 1 k ) ⊤ , W^2\operatorname{diag}(\alpha^{-1})=(2\beta-1)^2\kappa^2\operatorname{diag}(\alpha^{-1})+\phi_{\not\sim}\mathbf{1}_k(\mathbf{1}_k)^{\top}, W 2 diag ( α − 1 ) = ( 2 β − 1 ) 2 κ 2 diag ( α − 1 ) + ϕ ∼ 1 k ( 1 k ) ⊤ ,
where ϕ ≁ = ( 2 β − 1 ) 2 ( μ ≁ q ) ( μ ≁ q + 2 κ ) \phi_{\not\sim}=(2\beta-1)^2(\mu_{\not\sim}q)(\mu_{\not\sim}q+2\kappa) ϕ ∼ = ( 2 β − 1 ) 2 ( μ ∼ q ) ( μ ∼ q + 2 κ ) . Hence, the matrix W 2 diag ( α − 1 ) W^2\operatorname{diag}(\alpha^{-1}) W 2 diag ( α − 1 ) can be written as a sum of an invertible matrix and a rank 1 matrix. Since
1 + ϕ ≁ 1 k ⊤ ( ( 2 β − 1 ) 2 κ 2 ) − 1 diag ( α ) 1 k = ( μ ≁ q + κ ) 2 κ 2 = ( W 1 k ) 1 ( 2 β − 1 ) 2 κ 2 ≠ 0 , 1+\phi_{\not\sim}\mathbf{1}_k^{\top}\left((2\beta-1)^2\kappa^2\right)^{-1}\operatorname{diag}(\alpha)\mathbf{1}_k=\frac{(\mu_{\not\sim}q+\kappa)^2}{\kappa^2}=\frac{(W\mathbf{1}_k)_1}{(2\beta-1)^2\kappa^2}\ne0, 1 + ϕ ∼ 1 k ⊤ ( ( 2 β − 1 ) 2 κ 2 ) − 1 diag ( α ) 1 k = κ 2 ( μ ∼ q + κ ) 2 = ( 2 β − 1 ) 2 κ 2 ( W 1 k ) 1 = 0 ,
it then follows from Sherman-Morrison formula that W 2 diag ( α − 1 ) W^2\operatorname{diag}(\alpha^{-1}) W 2 diag ( α − 1 ) is invertible and its inverse is given by
1 ( 2 β − 1 ) 2 κ 2 ( diag ( α ) − μ ≁ q ( μ ≁ q + 2 κ ) ( μ ≁ q + κ ) 2 α ( α ⊤ ) ) . (57) \frac{1}{(2\beta-1)^2\kappa^2}\left(\operatorname{diag}(\alpha)-\frac{\mu_{\not\sim}q(\mu_{\not\sim}q+2\kappa)}{(\mu_{\not\sim}q+\kappa)^2}\alpha(\alpha^{\top})\right). \tag*{(57)} ( 2 β − 1 ) 2 κ 2 1 ( diag ( α ) − ( μ ∼ q + κ ) 2 μ ∼ q ( μ ∼ q + 2 κ ) α ( α ⊤ ) ) . (57)
where α = ( α 1 , … , α k ) \alpha=(\alpha_1,\ldots,\alpha_k) α = ( α 1 , … , α k ) . Since diag ( α ) \operatorname{diag}(\alpha) diag ( α ) is clearly invertible, the above argument also proves that W 2 W^2 W 2 is invertible.
Next, we use (57) and the fact that W 2 W^2 W 2 is invertible to conclude the proof of the proposition. Using that G ≥ 1 ( W 2 ) = G ( W 2 ) W 2 \mathcal{G}_{\geq1}(W^2)=\mathcal{G}(W^2)W^2 G ≥ 1 ( W 2 ) = G ( W 2 ) W 2 together with (57) and the fact that W 2 W^2 W 2 and W 2 diag ( α − 1 ) W^2\operatorname{diag}(\alpha^{-1}) W 2 diag ( α − 1 ) are invertible, we deduce that G ≥ 1 ( W 2 ) diag ( α − 1 ) \mathcal{G}_{\geq1}(W^2)\operatorname{diag}(\alpha^{-1}) G ≥ 1 ( W 2 ) diag ( α − 1 ) is also invertible and its inverse is
( W 2 diag ( α − 1 ) ) − 1 G ( W 2 ) − 1 = ( W 2 diag ( α − 1 ) ) − 1 ( I k − W 2 ) = ( W 2 diag ( α − 1 ) ) − 1 I k − diag ( α ) ( W 2 ) − 1 W 2 = ( W 2 diag ( α − 1 ) ) − 1 − diag ( α ) = 1 − ( 2 β − 1 ) 2 κ 2 ( 2 β − 1 ) 2 κ 2 diag ( α ) − μ ≁ q ( μ ≁ q + 2 κ ) ( μ ≁ q + κ ) 2 ( 2 β − 1 ) 2 κ 2 α ( α ⊤ ) . \begin{aligned}
\left(W^2\operatorname{diag}(\alpha^{-1})\right)^{-1}\mathcal{G}(W^2)^{-1}
&=\left(W^2\operatorname{diag}(\alpha^{-1})\right)^{-1}(I_k-W^2)\\
&=\left(W^2\operatorname{diag}(\alpha^{-1})\right)^{-1}I_k-\operatorname{diag}(\alpha)(W^2)^{-1}W^2\\
&=\left(W^2\operatorname{diag}(\alpha^{-1})\right)^{-1}-\operatorname{diag}(\alpha)\\
&=\frac{1-(2\beta-1)^2\kappa^2}{(2\beta-1)^2\kappa^2}\operatorname{diag}(\alpha)-\frac{\mu_{\not\sim}q(\mu_{\not\sim}q+2\kappa)}{(\mu_{\not\sim}q+\kappa)^2(2\beta-1)^2\kappa^2}\alpha(\alpha^{\top}).
\end{aligned} ( W 2 diag ( α − 1 ) ) − 1 G ( W 2 ) − 1 = ( W 2 diag ( α − 1 ) ) − 1 ( I k − W 2 ) = ( W 2 diag ( α − 1 ) ) − 1 I k − diag ( α ) ( W 2 ) − 1 W 2 = ( W 2 diag ( α − 1 ) ) − 1 − diag ( α ) = ( 2 β − 1 ) 2 κ 2 1 − ( 2 β − 1 ) 2 κ 2 diag ( α ) − ( μ ∼ q + κ ) 2 ( 2 β − 1 ) 2 κ 2 μ ∼ q ( μ ∼ q + 2 κ ) α ( α ⊤ ) .
Using the identity M = κ + μ ≁ q M=\kappa+\mu_{\not\sim q} M = κ + μ ∼ q (thus, κ < M \kappa<M κ < M ) and the fact that when (15) holds 0 < M < 1 / 2 0<M<1/2 0 < M < 1/2 , we have that
1 − μ ≁ q ( μ ≁ q + 2 κ ) ( μ ≁ q + κ ) 2 ( 1 − ( 2 β − 1 ) 2 κ 2 ) = κ 2 ( 1 − M 2 ( 2 β − 1 ) 2 ) M 2 ( 1 − κ 2 ( 2 β − 1 ) 2 ) ≠ 0. 1-\frac{\mu_{\not\sim q}(\mu_{\not\sim q}+2\kappa)}{(\mu_{\not\sim q}+\kappa)^2(1-(2\beta-1)^2\kappa^2)}
=\frac{\kappa^2(1-M^2(2\beta-1)^2)}{M^2(1-\kappa^2(2\beta-1)^2)}\ne0. 1 − ( μ ∼ q + κ ) 2 ( 1 − ( 2 β − 1 ) 2 κ 2 ) μ ∼ q ( μ ∼ q + 2 κ ) = M 2 ( 1 − κ 2 ( 2 β − 1 ) 2 ) κ 2 ( 1 − M 2 ( 2 β − 1 ) 2 ) = 0.
Thus, applying Sherman-Morrison formula in the above matrix and setting γ = ( 2 β − 1 ) 2 κ 2 1 − ( 2 β − 1 ) 2 κ 2 \gamma=\frac{(2\beta-1)^2\kappa^2}{1-(2\beta-1)^2\kappa^2} γ = 1 − ( 2 β − 1 ) 2 κ 2 ( 2 β − 1 ) 2 κ 2 , we obtain that
G ≥ 1 ( W 2 ) diag ( α − 1 ) = γ diag ( α − 1 ) + ρ ≁ q 1 k 1 k T , \mathcal{G}_{\geq1}(W^2)\operatorname{diag}(\alpha^{-1})=\gamma\operatorname{diag}(\alpha^{-1})+\rho_{\not\sim q}\mathbf{1}_k\mathbf{1}_k^{\mathsf{T}}, G ≥ 1 ( W 2 ) diag ( α − 1 ) = γ diag ( α − 1 ) + ρ ∼ q 1 k 1 k T ,
where ρ ≁ q = γ ( ( M / κ ) 2 − 1 ) ( 1 − ( 2 β − 1 ) 2 M 2 ) − 1 \rho_{\not\sim q}=\gamma((M/\kappa)^2-1)(1-(2\beta-1)^2M^2)^{-1} ρ ∼ q = γ (( M / κ ) 2 − 1 ) ( 1 − ( 2 β − 1 ) 2 M 2 ) − 1 , concluding the proof.
Proof of the preliminary results
In the section, we prove the results stated in § 2.2 , namely, Propositions 2.1, 2.2 and 2.3. We start proving Proposition 2.1.
Proof of Proposition 2.1. For each i ∈ [ n ] i\in[n] i ∈ [ n ] , denote m i = E G [ X i , t ] m_i=\mathbb{E}_{G}[X_{i,t}] m i = E G [ X i , t ] and observe that m i m_i m i can be writen as follows:
m i = P G ( X i , t = 1 ) = E G [ P G ( X i , t = 1 ∣ X t − 1 ) ] . m_i=\mathbb{P}_{G}(X_{i,t}=1)=\mathbb{E}_{G}\left[\mathbb{P}_{G}(X_{i,t}=1\mid X_{t-1})\right]. m i = P G ( X i , t = 1 ) = E G [ P G ( X i , t = 1 ∣ X t − 1 ) ] .
Using (7) in the latter expression, we obtain that
m i = E G [ λ + ∑ a ∈ [ k ] 1 { i ∈ C a } ( μ a n ∑ j : j ∼ i θ j i η j i X j , t − 1 + μ ≁ n ∑ j : j ≁ i θ j i η j i X j , t − 1 ) ] = λ + ∑ a ∈ [ k ] 1 { i ∈ C a } ( μ a n ∑ j : j ∼ i θ j i η j i E G [ X j , t − 1 ] + μ ≁ n ∑ j : j ≁ i θ j i η j i E G [ X j , t − 1 ] ) = λ + ∑ a ∈ [ k ] 1 { i ∈ C k } ( μ a n ∑ j : j ∼ i θ j i η j i m j + μ ≁ n ∑ j : j ≁ i θ j i η j i m j ) , (58) \begin{aligned}
m_i&=\mathbb{E}_{G}\left[\lambda+\sum_{a\in[k]}\mathbb{1}_{\{i\in C_a\}}\left(\frac{\mu_a}{n}\sum_{j:j\sim i}\theta_{ji}\eta_{ji}X_{j,t-1}+\frac{\mu_{\not\sim}}{n}\sum_{j:j\not\sim i}\theta_{ji}\eta_{ji}X_{j,t-1}\right)\right]\\
&=\lambda+\sum_{a\in[k]}\mathbb{1}_{\{i\in C_a\}}\left(\frac{\mu_a}{n}\sum_{j:j\sim i}\theta_{ji}\eta_{ji}\mathbb{E}_{G}[X_{j,t-1}]+\frac{\mu_{\not\sim}}{n}\sum_{j:j\not\sim i}\theta_{ji}\eta_{ji}\mathbb{E}_{G}[X_{j,t-1}]\right)\\
&=\lambda+\sum_{a\in[k]}\mathbb{1}_{\{i\in C_k\}}\left(\frac{\mu_a}{n}\sum_{j:j\sim i}\theta_{ji}\eta_{ji}m_j+\frac{\mu_{\not\sim}}{n}\sum_{j:j\not\sim i}\theta_{ji}\eta_{ji}m_j\right), \tag*{(58)}
\end{aligned} m i = E G λ + a ∈ [ k ] ∑ 1 { i ∈ C a } n μ a j : j ∼ i ∑ θ j i η j i X j , t − 1 + n μ ∼ j : j ∼ i ∑ θ j i η j i X j , t − 1 = λ + a ∈ [ k ] ∑ 1 { i ∈ C a } n μ a j : j ∼ i ∑ θ j i η j i E G [ X j , t − 1 ] + n μ ∼ j : j ∼ i ∑ θ j i η j i E G [ X j , t − 1 ] = λ + a ∈ [ k ] ∑ 1 { i ∈ C k } n μ a j : j ∼ i ∑ θ j i η j i m j + n μ ∼ j : j ∼ i ∑ θ j i η j i m j , (58)
where in the third equality we used the assumption that the Markov chain { X t } t ≥ 0 \{X_t\}_{t\geq0} { X t } t ≥ 0 is stationary implying that m i = E G [ X i , t ] m_i=\mathbb{E}_{G}[X_{i,t}] m i = E G [ X i , t ] , for all t t t . Since (58) holds for every i ∈ [ n ] i\in[n] i ∈ [ n ] , setting m = ( m 1 , ⋯ , m n ) T = E G [ X 0 ] m=(m_1,\cdots,m_n)^{\mathsf{T}}=\mathbb{E}_{G}[X_0] m = ( m 1 , ⋯ , m n ) T = E G [ X 0 ] , we can rewrite the system of equation in a matrix form as follows:
m = λ 1 n + A T m ⟺ ( I n − A T ) m = λ 1 n . m=\lambda\mathbf{1}_n+A^{\mathsf{T}}m\Longleftrightarrow(I_n-A^{\mathsf{T}})m=\lambda\mathbf{1}_n. m = λ 1 n + A T m ⟺ ( I n − A T ) m = λ 1 n .
Under the assumption that (9) holds, the matrix ( I n − A T ) (I_n-A^{\mathsf{T}}) ( I n − A T ) is invertible and the claim follows.
Next, we will prove Proposition 2.2. As we will see, the proof that we present relies on the following result.
Proposition G.1 (Synchronous firing covariance). Let [ n ] [n] [ n ] be the set of neurons, C = { C 1 , … , C k } C=\{C_1,\ldots,C_k\} C = { C 1 , … , C k } the k k k communities, G = ( C , θ , η ) G=(C,\theta,\eta) G = ( C , θ , η ) the interaction graph and λ ∈ [ 0 , 1 ] \lambda\in[0,1] λ ∈ [ 0 , 1 ] the spontaneous activity. Assume that the corresponding interaction matrix A A A and λ \lambda λ satisfy (9) . Then, the synchronous firing covariance matrix is the unique solution of the following matrix equation
Σ G ( 0 ) = d 0 ( A T Σ G ( 0 ) A ) + diag ( v G ( X 0 ) ) , (59) \Sigma_G^{(0)}=d_0(A^{\mathsf{T}}\Sigma_G^{(0)}A)+\operatorname{diag}(v_G(X_0)), \tag*{(59)} Σ G ( 0 ) = d 0 ( A T Σ G ( 0 ) A ) + diag ( v G ( X 0 )) , (59) where v G ( X 0 ) v_G(X_0) v G ( X 0 ) is the vector of the firing variances defined in (11) .
Proof. To avoid clutter in the notation, we shall omit the dependence on G G G and simply write Σ ( 0 ) \Sigma^{(0)} Σ ( 0 ) . Let us begin observing that for i = j i=j i = j , Σ i j ( 0 ) = Var G ( X i , 0 ) \Sigma_{ij}^{(0)}=\operatorname{Var}_{G}(X_{i,0}) Σ ij ( 0 ) = Var G ( X i , 0 ) and d 0 ( A T Σ ( 0 ) A ) i j = 0 d_{0}(A^{\mathsf T}\Sigma^{(0)}A)_{ij}=0 d 0 ( A T Σ ( 0 ) A ) ij = 0 , thus (58) is satisfied for the diagonal elements. Let us denote m i = E G [ X i , 0 ] m_i=\mathbb{E}_{G}[X_{i,0}] m i = E G [ X i , 0 ] for i ∈ [ n ] i\in[n] i ∈ [ n ] . For i ≠ j i\ne j i = j , using the fact that the Markov chain { X t } t ≥ 0 \{X_t\}_{t\ge0} { X t } t ≥ 0 is stationary, we have that
Σ i j ( 0 ) = Cov G ( X i , 1 , X j , 1 ) = E G [ X i , 1 X j , 1 ] − E G [ X i , 1 ] E G [ X j , 1 ] = E G [ E G [ X i , 1 X j , 1 ∣ X 0 ] ] − m i m j = ( ⋆ ) E G [ E G [ X i , 1 ∣ X 0 ] E G [ X j , 1 ∣ X 0 ] ] − m i m j = E G [ ( λ + ( A T X 0 ) i ) ( λ + ( A T X 0 ) j ) ] − E G [ λ + ( A T X 0 ) i ] E G [ λ + ( A T X 0 ) j ] . \begin{aligned}
\Sigma_{ij}^{(0)}
&=\operatorname{Cov}_{G}(X_{i,1},X_{j,1})=\mathbb{E}_{G}[X_{i,1}X_{j,1}]-\mathbb{E}_{G}[X_{i,1}]\mathbb{E}_{G}[X_{j,1}]\\
&=\mathbb{E}_{G}\left[\mathbb{E}_{G}[X_{i,1}X_{j,1}\mid X_{0}]\right]-m_i m_j\\
&\overset{(\star)}{=}\mathbb{E}_{G}\left[\mathbb{E}_{G}[X_{i,1}\mid X_{0}]\mathbb{E}_{G}[X_{j,1}\mid X_{0}]\right]-m_i m_j\\
&=\mathbb{E}_{G}\left[(\lambda+(A^{\mathsf T}X_{0})_i)(\lambda+(A^{\mathsf T}X_{0})_j)\right]-\mathbb{E}_{G}\left[\lambda+(A^{\mathsf T}X_{0})_i\right]\mathbb{E}_{G}\left[\lambda+(A^{\mathsf T}X_{0})_j\right].
\end{aligned} Σ ij ( 0 ) = Cov G ( X i , 1 , X j , 1 ) = E G [ X i , 1 X j , 1 ] − E G [ X i , 1 ] E G [ X j , 1 ] = E G [ E G [ X i , 1 X j , 1 ∣ X 0 ] ] − m i m j = ( ⋆ ) E G [ E G [ X i , 1 ∣ X 0 ] E G [ X j , 1 ∣ X 0 ] ] − m i m j = E G [ ( λ + ( A T X 0 ) i ) ( λ + ( A T X 0 ) j ) ] − E G [ λ + ( A T X 0 ) i ] E G [ λ + ( A T X 0 ) j ] .
where in ( ⋆ ) (\star) ( ⋆ ) we used the conditional independence of the model. Using further the linearity of the expectation, we obtain that
Σ i j ( 0 ) = E G [ ( A T X 0 ) i ( A T X 0 ) j ] − ( A T E G [ X 0 ] ) i ( A T E G [ X 0 ] ) j = ∑ h ∈ [ n ] ∑ ℓ ∈ [ n ] A i h T E [ X h , 0 X ℓ , 0 ] A j ℓ T − ∑ h ∈ [ n ] A i h T E G [ X h , 0 ] ∑ ℓ ∈ [ n ] A j ℓ T E G [ X ℓ , 0 ] = ∑ h ∈ [ n ] ∑ ℓ ∈ [ n ] A i h T ( E [ X h , 0 X ℓ , 0 ] − E [ X h , 0 ] E [ X ℓ , 0 ] ) A j ℓ T = ∑ h ∈ [ n ] ∑ ℓ ∈ [ n ] A i h T Σ h ℓ ( 0 ) A j ℓ T = ( A T Σ ( 0 ) A ) i j = d 0 ( A T Σ ( 0 ) A ) i j , \begin{aligned}
\Sigma_{ij}^{(0)}
&=\mathbb{E}_{G}\left[(A^{\mathsf T}X_{0})_i(A^{\mathsf T}X_{0})_j\right]-(A^{\mathsf T}\mathbb{E}_{G}[X_{0}])_i(A^{\mathsf T}\mathbb{E}_{G}[X_{0}])_j\\
&=\sum_{h\in[n]}\sum_{\ell\in[n]}A_{ih}^{\mathsf T}\mathbb{E}[X_{h,0}X_{\ell,0}]A_{j\ell}^{\mathsf T}
-\sum_{h\in[n]}A_{ih}^{\mathsf T}\mathbb{E}_{G}[X_{h,0}]
\sum_{\ell\in[n]}A_{j\ell}^{\mathsf T}\mathbb{E}_{G}[X_{\ell,0}]\\
&=\sum_{h\in[n]}\sum_{\ell\in[n]}A_{ih}^{\mathsf T}\left(\mathbb{E}[X_{h,0}X_{\ell,0}]-\mathbb{E}[X_{h,0}]\mathbb{E}[X_{\ell,0}]\right)A_{j\ell}^{\mathsf T}\\
&=\sum_{h\in[n]}\sum_{\ell\in[n]}A_{ih}^{\mathsf T}\Sigma_{h\ell}^{(0)}A_{j\ell}^{\mathsf T}
=(A^{\mathsf T}\Sigma^{(0)}A)_{ij}=d_{0}(A^{\mathsf T}\Sigma^{(0)}A)_{ij},
\end{aligned} Σ ij ( 0 ) = E G [ ( A T X 0 ) i ( A T X 0 ) j ] − ( A T E G [ X 0 ] ) i ( A T E G [ X 0 ] ) j = h ∈ [ n ] ∑ ℓ ∈ [ n ] ∑ A ih T E [ X h , 0 X ℓ , 0 ] A j ℓ T − h ∈ [ n ] ∑ A ih T E G [ X h , 0 ] ℓ ∈ [ n ] ∑ A j ℓ T E G [ X ℓ , 0 ] = h ∈ [ n ] ∑ ℓ ∈ [ n ] ∑ A ih T ( E [ X h , 0 X ℓ , 0 ] − E [ X h , 0 ] E [ X ℓ , 0 ] ) A j ℓ T = h ∈ [ n ] ∑ ℓ ∈ [ n ] ∑ A ih T Σ h ℓ ( 0 ) A j ℓ T = ( A T Σ ( 0 ) A ) ij = d 0 ( A T Σ ( 0 ) A ) ij ,
and thus (58) is satisfied also for the off-diagonal entries.
We now prove Proposition 2.2.
Proof of Proposition 2.2. To avoid clutter in the notation, we shall omit the dependence on G G G and simply write Σ ( 0 ) \Sigma^{(0)} Σ ( 0 ) . We begin pointing out that the operator vec ( ⋅ ) \operatorname{vec}(\cdot) vec ( ⋅ ) is linear and invertible. Therefore, the unique solution of (58) in a vectorized form is the solution of the following equation
vec ( Σ ( 0 ) ) = vec ( d 0 ( A T Σ ( 0 ) A ) ) + vec ( diag ( v G ( X 0 ) ) ) . \operatorname{vec}(\Sigma^{(0)})=\operatorname{vec}(d_{0}(A^{\mathsf T}\Sigma^{(0)}A))+\operatorname{vec}(\operatorname{diag}(v_{G}(X_{0}))). vec ( Σ ( 0 ) ) = vec ( d 0 ( A T Σ ( 0 ) A )) + vec ( diag ( v G ( X 0 ))) .
Claim. The matrix L L L satisfies L vec ( Σ ( 0 ) ) = vec ( d 0 ( A T Σ ( 0 ) A ) ) L\operatorname{vec}(\Sigma^{(0)})=\operatorname{vec}(d_{0}(A^{\mathsf T}\Sigma^{(0)}A)) L vec ( Σ ( 0 ) ) = vec ( d 0 ( A T Σ ( 0 ) A )) .
Proof of the claim. Since L u v = 0 L_{uv}=0 L uv = 0 for all v ∈ [ n 2 ] v\in[n^{2}] v ∈ [ n 2 ] and u = n ( i − 1 ) + i u=n(i-1)+i u = n ( i − 1 ) + i with i ∈ [ n ] i\in[n] i ∈ [ n ] , we have that L vec ( Σ ( 0 ) ) u = 0 = d 0 ( A T Σ ( 0 ) A ) i i = vec ( d 0 ( A T Σ ( 0 ) A ) ) u L\operatorname{vec}(\Sigma^{(0)})_u=0=d_{0}(A^{\mathsf T}\Sigma^{(0)}A)_{ii}=\operatorname{vec}(d_{0}(A^{\mathsf T}\Sigma^{(0)}A))_u L vec ( Σ ( 0 ) ) u = 0 = d 0 ( A T Σ ( 0 ) A ) ii = vec ( d 0 ( A T Σ ( 0 ) A ) ) u . Now, take u = n ( j − 1 ) + i u=n(j-1)+i u = n ( j − 1 ) + i with i , j ∈ [ n ] i,j\in[n] i , j ∈ [ n ] such that i ≠ j i\ne j i = j and observe that
L vec ( Σ ( 0 ) ) u = ∑ v ∈ [ n 2 ] L u v vec ( Σ ( 0 ) ) v = ∑ v ∈ [ n 2 ] ( A T ⊗ A T ) u v vec ( Σ ( 0 ) ) v = ∑ h ∈ [ n ] ∑ ℓ ∈ [ n ] A j h T A i ℓ T Σ h ℓ ( 0 ) = ∑ h ∈ [ n ] ∑ ℓ ∈ [ n ] A i ℓ T Σ ℓ h ( 0 ) A h j = ( A T Σ ( 0 ) A ) i j = d 0 ( A T Σ ( 0 ) A ) i j = vec ( d 0 ( A T Σ ( 0 ) A ) ) u , \begin{aligned}
L\operatorname{vec}(\Sigma^{(0)})_u
&=\sum_{v\in[n^{2}]}L_{uv}\operatorname{vec}(\Sigma^{(0)})_v
=\sum_{v\in[n^{2}]}(A^{\mathsf T}\otimes A^{\mathsf T})_{uv}\operatorname{vec}(\Sigma^{(0)})_v\\
&=\sum_{h\in[n]}\sum_{\ell\in[n]}A_{jh}^{\mathsf T}A_{i\ell}^{\mathsf T}\Sigma_{h\ell}^{(0)}
=\sum_{h\in[n]}\sum_{\ell\in[n]}A_{i\ell}^{\mathsf T}\Sigma_{\ell h}^{(0)}A_{hj}\\
&=(A^{\mathsf T}\Sigma^{(0)}A)_{ij}
=d_{0}(A^{\mathsf T}\Sigma^{(0)}A)_{ij}
=\operatorname{vec}(d_{0}(A^{\mathsf T}\Sigma^{(0)}A))_u,
\end{aligned} L vec ( Σ ( 0 ) ) u = v ∈ [ n 2 ] ∑ L uv vec ( Σ ( 0 ) ) v = v ∈ [ n 2 ] ∑ ( A T ⊗ A T ) uv vec ( Σ ( 0 ) ) v = h ∈ [ n ] ∑ ℓ ∈ [ n ] ∑ A j h T A i ℓ T Σ h ℓ ( 0 ) = h ∈ [ n ] ∑ ℓ ∈ [ n ] ∑ A i ℓ T Σ ℓ h ( 0 ) A hj = ( A T Σ ( 0 ) A ) ij = d 0 ( A T Σ ( 0 ) A ) ij = vec ( d 0 ( A T Σ ( 0 ) A ) ) u ,
concluding the proof of the claim.
Using the claim, the proof then simply follows from observing that ( I n 2 − L ) (I_{n^2}-L) ( I n 2 − L ) is invertible. This can be seen by observing that
∥ L ∥ ∞ = max s ∈ [ n 2 ] ∑ u ∈ [ n 2 ] ∣ L s u ∣ = max i ≠ j ∑ h ∈ [ n ] ∑ ℓ ∈ [ n ] ∣ A i h T A j ℓ T ∣ ≤ ∥ A T ∥ ∞ 2 . \lVert L\rVert_{\infty}=\max_{s\in[n^2]}\sum_{u\in[n^2]}|L_{su}|=\max_{i\ne j}\sum_{h\in[n]}\sum_{\ell\in[n]}|A^{\mathsf{T}}_{ih}A^{\mathsf{T}}_{j\ell}|\leq\lVert A^{\mathsf{T}}\rVert_{\infty}^{2}. ∥ L ∥ ∞ = s ∈ [ n 2 ] max u ∈ [ n 2 ] ∑ ∣ L s u ∣ = i = j max h ∈ [ n ] ∑ ℓ ∈ [ n ] ∑ ∣ A ih T A j ℓ T ∣ ≤ ∥ A T ∥ ∞ 2 .
The above bound and the assumption that A A A satisfies (9) implies that ∥ L ∥ ∞ < 1 \lVert L\rVert_{\infty}<1 ∥ L ∥ ∞ < 1 which in turn assures that ( I n 2 − L ) (I_{n^2}-L) ( I n 2 − L ) is invertible.
Finally, we prove Proposition 2.3.
Proof of Proposition 2.3. Let m i = E G [ X i , 0 ] m_i=\mathbb{E}_G[X_{i,0}] m i = E G [ X i , 0 ] for i ∈ [ n ] i\in[n] i ∈ [ n ] . We first show that, for every t ≥ 1 t\geq1 t ≥ 1 ,
Σ G ( t ) = A T Σ G ( t − 1 ) . (60) \Sigma_G^{(t)}=A^{\mathsf{T}}\Sigma_G^{(t-1)}. \tag*{(60)} Σ G ( t ) = A T Σ G ( t − 1 ) . (60)
Fix t ≥ 1 t\geq1 t ≥ 1 and i , j ∈ [ n ] i,j\in[n] i , j ∈ [ n ] . Using (6) together with the Markov property of { X t } t ≥ 0 \{X_t\}_{t\geq0} { X t } t ≥ 0 , we obtain
E G [ X i , t X j , 0 ] = E G [ X j , 0 E G [ X i , t ∣ X 0 , … , X t − 1 ] ] = E G [ X j , 0 E G [ X i , t ∣ X t − 1 ] ] = E G [ X j , 0 ( λ + ( A T X t − 1 ) i ) ] , \mathbb{E}_G[X_{i,t}X_{j,0}]=\mathbb{E}_G\left[X_{j,0}\mathbb{E}_G[X_{i,t}\mid X_0,\ldots,X_{t-1}]\right]
=\mathbb{E}_G\left[X_{j,0}\mathbb{E}_G[X_{i,t}\mid X_{t-1}]\right]
=\mathbb{E}_G\left[X_{j,0}\left(\lambda+(A^{\mathsf{T}}X_{t-1})_i\right)\right], E G [ X i , t X j , 0 ] = E G [ X j , 0 E G [ X i , t ∣ X 0 , … , X t − 1 ] ] = E G [ X j , 0 E G [ X i , t ∣ X t − 1 ] ] = E G [ X j , 0 ( λ + ( A T X t − 1 ) i ) ] ,
where in the second equality we used E G [ X i , t ∣ X t − 1 ] = ϕ G , i ( X t − 1 ) = λ + ( A T X t − 1 ) i \mathbb{E}_G[X_{i,t}\mid X_{t-1}]=\phi_{G,i}(X_{t-1})=\lambda+(A^{\mathsf{T}}X_{t-1})_i E G [ X i , t ∣ X t − 1 ] = ϕ G , i ( X t − 1 ) = λ + ( A T X t − 1 ) i . Expanding ( A T X t − 1 ) i = ∑ ℓ ∈ [ n ] A i ℓ T X ℓ , t − 1 (A^{\mathsf{T}}X_{t-1})_i=\sum_{\ell\in[n]}A^{\mathsf{T}}_{i\ell}X_{\ell,t-1} ( A T X t − 1 ) i = ∑ ℓ ∈ [ n ] A i ℓ T X ℓ , t − 1 and using the linearity of the expectation, it follows that
E G [ X i , t X j , 0 ] = λ m j + ∑ ℓ ∈ [ n ] A i ℓ T E G [ X ℓ , t − 1 X j , 0 ] . (61) \mathbb{E}_G[X_{i,t}X_{j,0}]=\lambda m_j+\sum_{\ell\in[n]}A^{\mathsf{T}}_{i\ell}\mathbb{E}_G[X_{\ell,t-1}X_{j,0}]. \tag*{(61)} E G [ X i , t X j , 0 ] = λ m j + ℓ ∈ [ n ] ∑ A i ℓ T E G [ X ℓ , t − 1 X j , 0 ] . (61)
On the other hand, by stationarity E G [ X i , t ] = m i \mathbb{E}_G[X_{i,t}]=m_i E G [ X i , t ] = m i for all t ≥ 0 t\geq0 t ≥ 0 , and conditioning on X t − 1 X_{t-1} X t − 1 similarly as above yields m i = E G [ X i , t ] = λ + ( A T m ) i = λ + ∑ ℓ ∈ [ n ] A i ℓ T m ℓ m_i=\mathbb{E}_G[X_{i,t}]=\lambda+(A^{\mathsf{T}}m)_i=\lambda+\sum_{\ell\in[n]}A^{\mathsf{T}}_{i\ell}m_\ell m i = E G [ X i , t ] = λ + ( A T m ) i = λ + ∑ ℓ ∈ [ n ] A i ℓ T m ℓ . Hence, subtracting m i , m j m_i,m_j m i , m j from both sides of (60), we deduce that
( Σ G ( t ) ) i j = E G [ X i , t X j , 0 ] − m i m j = ∑ ℓ ∈ [ n ] A i ℓ T ( E G [ X ℓ , t − 1 X j , 0 ] − m ℓ m j ) = ∑ ℓ ∈ [ n ] A i ℓ T ( Σ G ( t − 1 ) ) ℓ j = ( A T Σ G ( t ) ) i j . \begin{aligned}
(\Sigma_G^{(t)})_{ij}
&=\mathbb{E}_G[X_{i,t}X_{j,0}]-m_i m_j\\
&=\sum_{\ell\in[n]}A^{\mathsf{T}}_{i\ell}\left(\mathbb{E}_G[X_{\ell,t-1}X_{j,0}]-m_\ell m_j\right)
=\sum_{\ell\in[n]}A^{\mathsf{T}}_{i\ell}(\Sigma_G^{(t-1)})_{\ell j}
=(A^{\mathsf{T}}\Sigma_G^{(t)})_{ij}.
\end{aligned} ( Σ G ( t ) ) ij = E G [ X i , t X j , 0 ] − m i m j = ℓ ∈ [ n ] ∑ A i ℓ T ( E G [ X ℓ , t − 1 X j , 0 ] − m ℓ m j ) = ℓ ∈ [ n ] ∑ A i ℓ T ( Σ G ( t − 1 ) ) ℓ j = ( A T Σ G ( t ) ) ij .
Since i , j ∈ [ n ] i,j\in[n] i , j ∈ [ n ] are arbitrary, this proves (59). Finally, the result follows by iterating (59).
A remark on Stein equations
The goal of this section is to prove the result below. For each square matrix B B B , we write ρ ( B ) \rho(B) ρ ( B ) to indicate its spectral radius.
Lemma H.1. Let W W W be the square matrix defined in (16). If ρ ( W ) < 1 \rho(W)<1 ρ ( W ) < 1 , then the matrix Ψ \Psi Ψ defined in (19) is well-defined and is the unique solution of Stein equation (20).
Proof. Denote C = W diag ( ξ ⊙ α − 1 ) W ⊤ C = W \operatorname{diag}(\xi\mathbin{\odot} \alpha^{-1}) W^\top C = W diag ( ξ ⊙ α − 1 ) W ⊤ and observe that using the vectorization the Stein equation (20) can be rewritten as ( I k 2 − W ⊗ W ) vec ( Ψ ) = vec ( C ) (I_{k^2} - W \otimes W)\operatorname{vec}(\Psi) = \operatorname{vec}(C) ( I k 2 − W ⊗ W ) vec ( Ψ ) = vec ( C ) . Now, since ρ ( W ) < 1 \rho(W) < 1 ρ ( W ) < 1 and the eigenvalues of W ⊗ W W \otimes W W ⊗ W are the products λ i λ j \lambda_i\lambda_j λ i λ j of the eigenvalues of W W W , we have ρ ( W ⊗ W ) = ρ ( W ) 2 < 1 \rho(W \otimes W) = \rho(W)^2 < 1 ρ ( W ⊗ W ) = ρ ( W ) 2 < 1 [8 ), Theorem 4.2.12] . As a consequence, ( I k 2 − W ⊗ W ) (I_{k^2} - W \otimes W) ( I k 2 − W ⊗ W ) is invertible and its inverse is G ( W ⊗ W ) = ∑ u ≥ 0 ( W ⊗ W ) u \mathcal{G}(W \otimes W) = \sum_{u\geq0}(W \otimes W)^u G ( W ⊗ W ) = ∑ u ≥ 0 ( W ⊗ W ) u . Moreover, since ( I k 2 − W ⊗ W ) (I_{k^2} - W \otimes W) ( I k 2 − W ⊗ W ) is invertible, the equation ( I k 2 − W ⊗ W ) vec ( Ψ ) = vec ( C ) (I_{k^2} - W \otimes W)\operatorname{vec}(\Psi) = \operatorname{vec}(C) ( I k 2 − W ⊗ W ) vec ( Ψ ) = vec ( C ) has a unique solution vec ( Ψ ) = G ( W ⊗ W ) vec ( C ) \operatorname{vec}(\Psi) = \mathcal{G}(W \otimes W)\operatorname{vec}(C) vec ( Ψ ) = G ( W ⊗ W ) vec ( C ) . Finally, using that ( W ⊗ W ) u = W u ⊗ W u (W \otimes W)^u = W^u \otimes W^u ( W ⊗ W ) u = W u ⊗ W u and W u ⊗ W u vec ( C ) = vec ( W u C ( W ⊤ ) u ) W^u \otimes W^u\operatorname{vec}(C) = \operatorname{vec}(W^u C(W^\top)^u) W u ⊗ W u vec ( C ) = vec ( W u C ( W ⊤ ) u ) for all u ≥ 0 u \geq0 u ≥ 0 , we then conclude that
vec ( Ψ ) = ∑ u ≥ 0 vec ( W u C ( W ⊤ ) u ) = vec ( ∑ u ≥ 0 W u C ( W ⊤ ) u ) \operatorname{vec}(\Psi) = \sum_{u\geq0} \operatorname{vec}(W^u C(W^\top)^u) = \operatorname{vec}\left(\sum_{u\geq0} W^u C(W^\top)^u\right) vec ( Ψ ) = u ≥ 0 ∑ vec ( W u C ( W ⊤ ) u ) = vec ( u ≥ 0 ∑ W u C ( W ⊤ ) u )
so that we must have Ψ = ∑ u ≥ 0 W u C ( W ⊤ ) u = ∑ u = 1 ∞ W u diag ( ξ ⊙ α − 1 ) ( W ⊤ ) u \Psi= \sum_{u\geq0} W^u C(W^\top)^u = \sum_{u=1}^{\infty} W^u \operatorname{diag}(\xi\mathbin{\odot} \alpha^{-1})(W^\top)^u Ψ = ∑ u ≥ 0 W u C ( W ⊤ ) u = ∑ u = 1 ∞ W u diag ( ξ ⊙ α − 1 ) ( W ⊤ ) u , concluding the proof.