Introduction
Relational data arise in a wide range of scientific and machine-learning applications, including social and information networks, recommender systems, voting records, citation patterns, and user–item interactions. Such data record associations either among objects from a single set (unipartite relational data, such as networks) or between objects from two distinct sets (bipartite relational data, such as user–item interactions). These two settings are geometrically connected. After pooling the two sets, a bipartite relation can be represented as a symmetric relation matrix in which the cross-set blocks are observed, whereas the two within-set blocks are structurally missing.
A central task is to uncover the relational structure among these objects for prediction, scientific interpretation, and exploratory visualization. A broad range of low-dimensional representation methods has been developed for this purpose. Multidimensional scaling (MDS) represents one set of objects from pairwise dissimilarities [38 , 7 ] , whereas multidimensional unfolding (MDU) jointly represents two sets of objects from a rectangular matrix of cross-set relations or preferences [35 , 7 ] . Latent space models assign embedding vectors to network nodes and relate the observed network to these embedding vectors [24 ] (Handcock et al. 2007). Correspondence analysis (CA) and biplots provide joint low-dimensional displays of the rows and columns of a data table [17 , 18 ] , while factor models represent two sets of objects through latent factor scores and loading vectors [3 , 36 , 5 ] . More broadly, machine learning has developed a diverse family of low-dimensional embedding methods for visualization and representation learning, including manifold and spectral methods, neighbor embeddings, and neural parametric approaches [37 , 6 , 22 , 39 , 34 de Bodt] et al., 2025). Although all these methods produce low-dimensional coordinates, proximity in the resulting representation does not have a universal interpretation. Depending on the method, the coordinates are designed to represent pairwise distances, inner-product interactions, standardized departures from independence, or local neighborhood relations.
Among these representations, a common low-dimensional Euclidean configuration is particularly attractive because it places all objects in a shared metric space, so that Euclidean distance has a direct relational interpretation ([35 , 7 ] ). Depending on the application, proximity may encode similarity, preference, approval, affinity, or interaction intensity. The resulting geometry can reveal clusters, latent continua, polarization, and objects occupying intermediate positions. Figure 1 provides a schematic illustration of this common Euclidean view for unipartite and bipartite relational data, where the smaller the distance between objects in the embedding space, the stronger their similarity tends to be as reflected by the observed relational data.
Figure 1. Schematic illustrations of latent Euclidean configurations for unipartite (left) and bipartite (right) relational data. In the unipartite setting, relations are observed among objects from a single set; in the bipartite setting, objects from two distinct sets are jointly represented in a common Euclidean space while only cross-set relations are observed.
Despite the utility of low-dimensional representations, existing approaches have four major limitations.
Validity of joint geometric representation. Many methods can provide a joint display of objects in a low-dimensional space, but this does not necessarily provide a valid joint metric geometry. In that case, the interpretation of low-dimensional coordinates depends on the structure that the method is designed to represent. For example, factor models represent row objects by factor scores and column objects by loadings, whose bilinear signal is invariant under paired nonsingular transformations [36 , 5 ] . Biplots and correspondence analysis provide joint displays, but their cross-set interpretation is based on inner products, projections, or standardized associations rather than row–column Euclidean distances [17 , 18 ] . Neighborhood-embedding methods primarily preserve local proximity rather than global Euclidean geometry [14 , 39 , 34 ] .
Modeling scope. While model-based approaches can yield valid joint geometric interpretations, current methods remain limited to specific observation structures and response types. For instance, multidimensional scaling (MDS), multidimensional unfolding (MDU), and distance-based latent space models are typically tailored to narrow settings [7 , 13 , 24 ] (Handcock et al. 2007), whereas more flexible generalized factor models do not induce a common Euclidean metric geometry [36 , 5 ] . Consequently, a unified distance-based framework that handles both unipartite and bipartite relational data across continuous, binary, ordinal, and count-valued responses is currently lacking.
Computational scalability. Classical MDS admits a direct eigendecomposition-based solution [38 , 7 ] . In contrast, metric and nonmetric MDS, classical and probabilistic MDU, and likelihood-based latent space models
commonly estimate one embedding vector per object through a high-dimensional non-convex optimization ([26 , 8 , 24 , 13 ] ). The dimension of such optimization problems grows with the number of objects, making these methods unsuitable for large data sets.
Uniform configuration recovery theory. By uniform configuration recovery, we mean simultaneous control of the estimation errors of all individual embedding vectors, rather than only an average error over the entire configuration. Such guarantees have been studied in selected unipartite settings ([44 , 31 ] (Li et al. 2020)), whereas model-based theory for bipartite Euclidean unfolding—where only cross-set relations are observed—has primarily concerned average configuration recovery under more specific response and identifiability assumptions ([35 , 13 ] ). A unified uniform-recovery theory covering both observation structures is lacking.
To address these limitations, we develop a unified distance-based formulation for learning latent Euclidean configurations from both unipartite and bipartite relational data. We assume that the mean of each observation is linked, through a prespecified strictly increasing link function, to a drift parameter minus the squared Euclidean distance between the corresponding embedding vectors. This formulation accommodates continuous, binary, ordinal, and count-valued relations while retaining a common proximity-based interpretation.
We propose a novel estimation procedure that uses a common signal-denoising stage under both relational settings, while adapting the geometric recovery step to their different observation structures. By separating signal estimation from geometric recovery, the procedure avoids joint optimization over all latent positions, and the recovery stage can be combined with any sufficiently accurate signal estimator. We first estimate the signal matrix using a global spectral procedure; for models with nonlinear link functions, we further apply one-step row- and column-wise estimating-equation refinement to obtain uniform entrywise accuracy. Given the denoised signal matrix, the unipartite configuration is recovered by the classical MDS construction based on double centering and eigendecomposition [38 , 19 ] . For bipartite data, the structurally missing within-set blocks prevent this construction from recovering the joint configuration: double centering identifies only the cross-product of the two centered sets of embedding vectors. To tackle this issue, we profile out the high-dimensional singular spaces and formulate this remaining ambiguity as an alignment problem, which is solved by an optimization whose dimension depends only on the latent dimension, rather than on the numbers of row and column objects.
Theoretically, we provide convergence guarantees linking the estimated signal matrix to the recovered Euclidean configuration. For bipartite data, we establish general identifiability conditions combining a quantitative measure of local identification strength with global separation. The resulting error bounds explicitly adapt to the local identification strength and are more general than the anchor-point conditions imposed in previous work [13 ] . Based on these identifiability conditions, our results further quantify how uniform accuracy of the estimated signal matrix propagates to simultaneous recovery of all embedding vectors. Taken together, these results establish a unified signal-to-configuration recovery theory for both unipartite and bipartite relational data.
Related Literature
Multidimensional scaling (MDS) and multidimensional unfolding (MDU) are most closely related to our work in terms of the target parameter, as both aim to recover a low-dimensional Euclidean configuration from relational or proximity data [38 , 35 , 7 ] . Classical and modern MDS/MDU methods recover such configurations through spectral or optimization-based procedures [7 ] de Leeuw and Mair, 2009). Statistical formulations of unfolding further treat the latent coordinates as model parameters and establish identifiability and configuration recovery guarantees under probabilistic response models [13 ] . Our work shares the goal of recovering an interpretable Euclidean geometry, but considers a common mean formulation for both unipartite and bipartite data, separates signal estimation from geometric recovery, and develops uniform configuration recovery guarantees.
Distance-based latent space models provide another closely related framework for model-based geometric representation of network data, in which each node is assigned a latent position and edge probabilities depend on distances between latent positions [24 ] (Handcock et al. 2007). Estimation has traditionally relied on likelihood or Bayesian procedures, while more scalable methods based on nuclear-norm regularization and projected gradient algorithms have been developed for broad classes of latent space models (Ma et al., 2020). Recent work has also established uniform consistency and asymptotic distribution theory for latent-position estimators under a range of edge types and link functions (Li et al., 2025). These developments primarily concern unipartite networks, whereas our framework treats unipartite and bipartite observation structures jointly and, in the latter case, addresses configuration recovery when only cross-set relations are observed.
Our signal-estimation problem is also connected to the literature on high-dimensional factor and generalized latent factor models, which represents a large data matrix through low-dimensional factors and loadings and targets the resulting bilinear low-rank structure [3 , 16 , 12 ] . Principal-component, projected-PCA, maximum-likelihood, and related scalable methods have been developed for estimating such latent structures in high dimensions [16 , 41 , 25 ] . This literature provides consistency, convergence rates, inferential theory, and results on factor identifiability under various high-dimensional regimes. In our setting, however, the low-rank factorization is primarily an intermediate device for estimating the signal matrix: the final target is a common Euclidean configuration whose row and column coordinates have the same metric interpretation, requiring an additional geometric recovery step beyond factor estimation.
Finally, our signal-estimation procedure is related to a growing literature on spectral estimation and refinement for high-dimensional latent-variable models. SVD-based methods exploit low-rank structure in the population signal to construct computationally efficient estimators or initializers, for example in high-dimensional item factor analysis, mixed-membership models, and deep latent-variable models [43 , 28 ] (Chen and Gu 2024). Related spectral-initialization ideas have also been developed for low-rank matrix and tensor estimation, with statistical and computational guarantees [42 ] . More recent work combines spectral initialization with model-specific one-step refinement to improve the statistical accuracy of the initial spectral estimator in high-dimensional latent-variable models [32 ] (Chen and Li 2024). Our procedure follows this broad spectral-plus-refinement principle for models with nonlinear link functions, while its output is a uniformly accurate estimate of a distance-based signal matrix that is subsequently passed to a separate configuration recovery procedure; in the bipartite setting, this additionally requires resolving the non-orthogonal alignment ambiguity left by the cross-product structure.
Notation
We introduce some notation for the rest of the paper. For a positive integer N N N , let [ N ] : = { 1 , … , N } [N] := \{1,\ldots,N\} [ N ] := { 1 , … , N } . For a matrix M \mathbf{M} M , let vec ( M ) \operatorname{vec}(\mathbf{M}) vec ( M ) denote its column-wise vectorization. Let S K : = { diag ( q 1 , … , q K ) : q k ∈ { − 1 , 1 } } \mathcal{S}_K := \{\operatorname{diag}(q_1,\ldots,q_K): q_k \in\{-1,1\}\} S K := { diag ( q 1 , … , q K ) : q k ∈ { − 1 , 1 }} denote the set of K × K K \times K K × K diagonal sign matrices. For two integers a , b a,b a , b , let a ∨ b = max { a , b } a \vee b = \max\{a,b\} a ∨ b = max { a , b } and a ∧ b = min { a , b } a \wedge b = \min\{a,b\} a ∧ b = min { a , b } . We let ∥ x ∥ \lVert x\rVert ∥ x ∥ denote the standard Euclidean norm of a vector x = ( x 1 , … , x n ) T \mathbf{x}=(x_1,\ldots,x_n)^{\mathsf{T}} x = ( x 1 , … , x n ) T and let ∥ x ∥ ∞ = max i ∣ x i ∣ \|\mathbf{x}\|_{\infty}=\max_i|x_i| ∥ x ∥ ∞ = max i ∣ x i ∣ denote the infinity norm of x \mathbf{x} x . For a matrix X = ( x i j ) m × n \mathbf{X}=(x_{ij})_{m\times n} X = ( x ij ) m × n and a function f : R → R f:\mathbb{R}\to\mathbb{R} f : R → R , let f ( X ) : = ( f ( x i j ) ) m × n f(\mathbf{X}):=(f(x_{ij}))_{m\times n} f ( X ) := ( f ( x ij ) ) m × n . The matrix max-norm is denoted by ∥ X ∥ max = max i , j ∣ x i j ∣ \|\mathbf{X}\|_{\max}=\max_{i,j}|x_{ij}| ∥ X ∥ m a x = max i , j ∣ x ij ∣ . In general, we define the operator p p p -norm of a matrix by ∥ X ∥ p = sup ∥ u ∥ p = 1 ∥ X u ∥ p \|\mathbf{X}\|_p=\sup_{\|\mathbf{u}\|_p=1}\|\mathbf{X}\mathbf{u}\|_p ∥ X ∥ p = sup ∥ u ∥ p = 1 ∥ Xu ∥ p . In particular, ∥ X ∥ 1 = max j ∑ i = 1 m ∣ x i j ∣ \|\mathbf{X}\|_1=\max_j\sum_{i=1}^{m}|x_{ij}| ∥ X ∥ 1 = max j ∑ i = 1 m ∣ x ij ∣ ; ∥ X ∥ ∞ = max i ∑ j = 1 n ∣ x i j ∣ \|\mathbf{X}\|_{\infty}=\max_i\sum_{j=1}^{n}|x_{ij}| ∥ X ∥ ∞ = max i ∑ j = 1 n ∣ x ij ∣ ; and ∥ ⋅ ∥ \|\cdot\| ∥ ⋅ ∥ denotes the spectral norm, or equivalently the matrix 2-norm ∥ ⋅ ∥ 2 \|\cdot\|_2 ∥ ⋅ ∥ 2 , when there is no ambiguity. Let ∥ X ∥ F \|\mathbf{X}\|_F ∥ X ∥ F and ∥ X ∥ 2 → ∞ \|\mathbf{X}\|_{2\to\infty} ∥ X ∥ 2 → ∞ denote its Frobenius and two-to-infinity norms, respectively, which are defined as ∥ X ∥ F = ( ∑ i = 1 m ∑ j = 1 n ∣ x i j ∣ 2 ) 1 / 2 \|\mathbf{X}\|_F=\left(\sum_{i=1}^{m}\sum_{j=1}^{n}|x_{ij}|^2\right)^{1/2} ∥ X ∥ F = ( ∑ i = 1 m ∑ j = 1 n ∣ x ij ∣ 2 ) 1/2 and ∥ X ∥ 2 → ∞ = max i ( ∑ j = 1 n x i j 2 ) 1 / 2 \|\mathbf{X}\|_{2\to\infty}=\max_i\left(\sum_{j=1}^{n}x_{ij}^2\right)^{1/2} ∥ X ∥ 2 → ∞ = max i ( ∑ j = 1 n x ij 2 ) 1/2 . ∥ X ∥ ∗ \|\mathbf{X}\|_* ∥ X ∥ ∗ denotes the nuclear norm of X \mathbf{X} X , which is the sum of all singular values. For any matrix A ∈ R N × J \mathbf{A}\in\mathbb{R}^{N\times J} A ∈ R N × J with singular value decomposition A = ∑ i = 1 N ∧ J σ i u i v i T \mathbf{A}=\sum_{i=1}^{N\wedge J}\sigma_i\mathbf{u}_i\mathbf{v}_i^{\mathsf{T}} A = ∑ i = 1 N ∧ J σ i u i v i T and σ 1 ≥ ⋯ ≥ σ N ∧ J ≥ 0 \sigma_1\ge\cdots\ge\sigma_{N\wedge J}\ge0 σ 1 ≥ ⋯ ≥ σ N ∧ J ≥ 0 , we define SVD K ( A ) : = ∑ i = 1 K σ i u i v i T \operatorname{SVD}_K(\mathbf{A}):=\sum_{i=1}^{K}\sigma_i\mathbf{u}_i\mathbf{v}_i^{\mathsf{T}} SVD K ( A ) := ∑ i = 1 K σ i u i v i T . For any symmetric matrix M ∈ R N × N \mathbf{M}\in\mathbb{R}^{N\times N} M ∈ R N × N with eigendecomposition M = ∑ i = 1 N λ i u i u i T \mathbf{M}=\sum_{i=1}^{N}\lambda_i\mathbf{u}_i\mathbf{u}_i^{\mathsf{T}} M = ∑ i = 1 N λ i u i u i T and λ 1 ≥ ⋯ ≥ λ N \lambda_1\ge\cdots\ge\lambda_N λ 1 ≥ ⋯ ≥ λ N , we define Eigen K ( M ) : = ∑ i = 1 K λ i u i u i T \operatorname{Eigen}_K(\mathbf{M}):=\sum_{i=1}^{K}\lambda_i\mathbf{u}_i\mathbf{u}_i^{\mathsf{T}} Eigen K ( M ) := ∑ i = 1 K λ i u i u i T . For a < b a<b a < b , let P [ a , b ] ( x ) = a 1 { x ≤ a } + x 1 { a < x < b } + b 1 { x ≥ b } \mathcal{P}_{[a,b]}(x)=a\mathbf{1}_{\{x\le a\}}+x\mathbf{1}_{\{a<x<b\}}+b\mathbf{1}_{\{x\ge b\}} P [ a , b ] ( x ) = a 1 { x ≤ a } + x 1 { a < x < b } + b 1 { x ≥ b } . For a matrix M = ( m i j ) \mathbf{M}=(m_{ij}) M = ( m ij ) , let P [ a , b ] ( M ) \mathcal{P}_{[a,b]}(\mathbf{M}) P [ a , b ] ( M ) denote the matrix whose ( i , j ) (i,j) ( i , j ) -th entry is P [ a , b ] ( m i j ) \mathcal{P}_{[a,b]}(m_{ij}) P [ a , b ] ( m ij ) . For a constant C > 0 C>0 C > 0 , let the projection operator P C , 2 → ∞ \mathcal{P}_{C,2\to\infty} P C , 2 → ∞ map an n × p n\times p n × p matrix onto the constraint set { A ∈ R n × p : ∥ A ∥ 2 → ∞ ≤ C } \{\mathbf{A}\in\mathbb{R}^{n\times p}:\|\mathbf{A}\|_{2\to\infty}\le C\} { A ∈ R n × p : ∥ A ∥ 2 → ∞ ≤ C } . For a matrix V = ( v 1 , … , v n ) T ∈ R n × p \mathbf{V}=(\mathbf{v}_1,\ldots,\mathbf{v}_n)^{\mathsf{T}}\in\mathbb{R}^{n\times p} V = ( v 1 , … , v n ) T ∈ R n × p , P C , 2 → ∞ ( V ) = ( v ~ 1 , … , v ~ n ) T \mathcal{P}_{C,2\to\infty}(\mathbf{V})=(\widetilde{\mathbf{v}}_1,\ldots,\widetilde{\mathbf{v}}_n)^{\mathsf{T}} P C , 2 → ∞ ( V ) = ( v 1 , … , v n ) T , where v ~ i = v i \widetilde{\mathbf{v}}_i=\mathbf{v}_i v i = v i if ∥ v i ∥ ≤ C \|\mathbf{v}_i\|\le C ∥ v i ∥ ≤ C and v ~ i = ( C / ∥ v i ∥ ) v i \widetilde{\mathbf{v}}_i=(C/\|\mathbf{v}_i\|)\mathbf{v}_i v i = ( C /∥ v i ∥ ) v i otherwise. Additionally, let J n = I n − 1 n 1 n T / n \mathbf{J}_n=\mathbf{I}_n-\mathbf{1}_n\mathbf{1}_n^{\mathsf{T}}/n J n = I n − 1 n 1 n T / n denote the centering matrix and we sometimes omit the subscript when the dimension is clear from matrix multiplication. Let O K \mathbb{O}_K O K denote the set of all K K K -by-K K K orthogonal matrices. For a square matrix M \mathbf{M} M , let Δ ( M ) \Delta(\mathbf{M}) Δ ( M ) denote the matrix whose off-diagonal entries are the same as those of M \mathbf{M} M while the diagonal entries are set to 0 0 0 . Let B a ( K ) ( C ) B_{\mathbf{a}}^{(K)}(C) B a ( K ) ( C ) denote the ball in R K \mathbb{R}^{K} R K centered at a ∈ R K \mathbf{a}\in\mathbb{R}^{K} a ∈ R K with radius C C C . The ψ 1 \psi_1 ψ 1 -Orlicz norm of a random variable X X X , where ψ 1 ( x ) = exp ( x ) − 1 \psi_1(x)=\exp(x)-1 ψ 1 ( x ) = exp ( x ) − 1 , is defined as ∥ X ∥ ψ 1 : = inf { t > 0 : E [ exp ( ∣ X ∣ / t ) ] ≤ 2 } \|X\|_{\psi_1}:=\inf\{t>0:\mathbb{E}[\exp(|X|/t)]\le2\} ∥ X ∥ ψ 1 := inf { t > 0 : E [ exp ( ∣ X ∣/ t )] ≤ 2 } . Note that X X X is sub-exponential if ∥ X ∥ ψ 1 < ∞ \|X\|_{\psi_1}<\infty ∥ X ∥ ψ 1 < ∞ . Throughout the paper, c > 0 c>0 c > 0 denotes a generic constant independent of ( N , J ) (N,J) ( N , J ) , whose value may change from line to line.
The remainder of the paper is organized as follows. Section 2 introduces the generalized distance-based latent space models and their connections to existing low-dimensional representation methods. Section 3 presents the proposed estimation procedure, including refined spectral denoising and configuration recovery. Section 4 establishes the corresponding theoretical guarantees for distance-signal estimation and configuration recovery. Section 5 presents simulation studies, and Section 6 applies the proposed method to a citation network of statisticians and European Parliament roll-call data. Section 7 concludes with a discussion, and the appendices contain proofs and additional numerical results.
Generalized Distance-Based Latent Space Models
To describe the unipartite and bipartite settings within a common notation, let P 1 \mathcal{P}_1 P 1 and P 2 \mathcal{P}_2 P 2 denote the object sets associated with the row and column indices of the observed relational matrix, respectively, and let P = P 1 ∪ P 2 \mathcal{P} = \mathcal{P}_1 \cup\mathcal{P}_2 P = P 1 ∪ P 2 denote the global object set. Define N = ∣ P 1 ∣ N = \lvert\mathcal{P}_1\rvert N = ∣ P 1 ∣ and J = ∣ P 2 ∣ J = \lvert\mathcal{P}_2\rvert J = ∣ P 2 ∣ . In the unipartite setting, P 1 = P 2 = [ N ] \mathcal{P}_1 = \mathcal{P}_2 = [N] P 1 = P 2 = [ N ] , whereas in the bipartite setting, P 1 = [ N ] \mathcal{P}_1 = [N] P 1 = [ N ] and P 2 = N + [ J ] = { N + 1 , … , N + J } \mathcal{P}_2 = N + [J] = \{N + 1, \ldots, N + J\} P 2 = N + [ J ] = { N + 1 , … , N + J } .
Each object u ∈ P u \in\mathcal{P} u ∈ P is associated with an embedding vector θ u ∈ R K \boldsymbol{\theta}_u \in\mathbb{R}^K θ u ∈ R K . We collect all embedding vectors into a global configuration Θ = ( θ u ) u ∈ P \boldsymbol{\Theta} = (\boldsymbol{\theta}_u)_{u \in\mathcal{P}} Θ = ( θ u ) u ∈ P . For g ∈ { 1 , 2 } g \in\{1, 2\} g ∈ { 1 , 2 } , let Θ P g = ( θ u ) u ∈ P g \boldsymbol{\Theta}_{\mathcal{P}_g} = (\boldsymbol{\theta}_u)_{u \in\mathcal{P}_g} Θ P g = ( θ u ) u ∈ P g denote the configuration consisting of the objects in P g \mathcal{P}_g P g . We use u u u and v v v to denote global object indices in P \mathcal{P} P , whereas i i i and j j j denote the local row and column indices of the observed relational matrix Y = ( y i j ) \mathbf{Y} = (y_{ij}) Y = ( y ij ) . The correspondence between the local indices ( i , j ) (i,j) ( i , j ) and the associated global objects depends on the observation structure and will be specified separately for the unipartite and bipartite settings.
For each observed entry y i j y_{ij} y ij , let d i j d_{ij} d ij denote the squared Euclidean distance between the two corresponding global objects, and define m i j m_{ij} m ij as
m i j = τ − d i j with d i j = { ∥ θ i − θ j ∥ 2 , unipartite , ∥ θ i − θ N + j ∥ 2 , bipartite , m_{ij}=\tau-d_{ij}\quad\text{with}\quad d_{ij}=
\begin{cases}
\lVert\theta_i-\theta_j\rVert^2, & \text{unipartite},\\
\lVert\theta_i-\theta_{N+j}\rVert^2, & \text{bipartite},
\end{cases} m ij = τ − d ij with d ij = { ∥ θ i − θ j ∥ 2 , ∥ θ i − θ N + j ∥ 2 , unipartite , bipartite ,
where τ ∈ R \tau\in\mathbb{R} τ ∈ R is an unknown drift parameter. We treat the embedding vectors as fixed unknown parameters and assume that the observations satisfy an independence structure appropriate to the underlying relational setting, as specified in Sections 2.1 and 2.2. We assume
E ( y i j ) = f ( m i j ) = f ( τ − d i j ) , (1) \mathbb{E}(y_{ij})=f(m_{ij})=f(\tau-d_{ij}), \tag*{(1)} E ( y ij ) = f ( m ij ) = f ( τ − d ij ) , (1)
where f : R → R f:\mathbb{R}\to\mathbb{R} f : R → R is a prespecified strictly increasing link function. We refer to M = ( m i j ) \mathbf{M}=(m_{ij}) M = ( m ij ) as the signal matrix. Under (1), a smaller latent distance corresponds to a larger expected relation. The parameter τ \tau τ controls the overall level of the observed responses. If the observed entries represent dissimilarities rather than similarities, one may instead model E ( y i j ) = f ( − m i j ) \mathbb{E}(y_{ij})=f(-m_{ij}) E ( y ij ) = f ( − m ij ) . For simplicity, we focus on the formulation in (1) throughout the paper.
The mean formulation in (1) accommodates different response types through the choice of f f f . For continuous responses, one may take f f f to be the identity function and regard y i j y_{ij} y ij as a noisy observation of m i j m_{ij} m ij . For binary responses, one may take f f f to be a logistic or probit function, so that the probability of observing a positive relation decreases with the latent distance. Ordinal responses can be handled analogously through an ordinal response model. For count data, one may take f ( x ) = exp ( x ) f(x)=\exp(x) f ( x ) = exp ( x ) , leading to a Poisson mean model with intensity exp ( m i j ) \exp(m_{ij}) exp ( m ij ) .
The object of interest is the latent Euclidean configuration Θ \boldsymbol{\Theta} Θ , rather than only the signal matrix or fitted mean relations. Because Euclidean distances are invariant under common shifts and orthogonal transformations, configuration recovery is understood modulo these intrinsic geometric ambiguities. In our theoretical and simulation analyses, recovery accuracy is measured by the largest row-wise Euclidean discrepancy between the estimated and true configurations after appropriate alignment, corresponding to a two-to-infinity norm loss. Specifically, let Θ ∗ \Theta^* Θ ∗ be the true latent configuration and Θ ^ \widehat{\Theta} Θ be any estimator. The recovery accuracy is measured by
d c ( Θ ^ , Θ ∗ ) = min Q ∈ O K , b ∈ R K ∥ Θ ^ − 1 ∣ P ∣ b T − Θ ∗ Q ∥ 2 → ∞ . d_c(\widehat{\Theta}, \Theta^*) = \min_{Q \in\mathbb{O}_K, b \in\mathbb{R}^K} \left\lVert\widehat{\Theta} - \mathbf{1}_{|\mathcal{P}|}b^{\mathsf{T}} - \Theta^* Q \right\rVert_{2 \to\infty}. d c ( Θ , Θ ∗ ) = Q ∈ O K , b ∈ R K min Θ − 1 ∣ P ∣ b T − Θ ∗ Q 2 → ∞ .
For unipartite data, the centered configuration is given by its Gram matrix J N Θ ( J N Θ ) T \mathbf{J}_N\Theta(\mathbf{J}_N\Theta)^{\mathsf{T}} J N Θ ( J N Θ ) T and can therefore be recovered spectrally once the signal matrix is known. For bipartite data, however, only cross-set relations are observed, so the double-centered signal identifies only a cross-product of the two centered configurations as J N Θ P 1 ( J Θ P 2 ) T \mathbf{J}_N\Theta_{\mathcal{P}_1}(\mathbf{J}\Theta_{\mathcal{P}_2})^{\mathsf{T}} J N Θ P 1 ( J Θ P 2 ) T ; recovering the joint Euclidean configuration therefore requires additional identifiability conditions and an alignment step. We give a more detailed discussion in § 3.2 .
Unipartite Model
We first consider unipartite relational data among a single set of N N N objects. Under the notation introduced above, we have P 1 = P 2 = P = [ N ] \mathcal{P}_1 = \mathcal{P}_2 = \mathcal{P} = [N] P 1 = P 2 = P = [ N ] , so that Θ P 1 = Θ P 2 = Θ = ( θ 1 , … , θ N ) T \Theta_{\mathcal{P}_1} = \Theta_{\mathcal{P}_2} = \Theta= (\theta_1, \ldots, \theta_N)^{\mathsf{T}} Θ P 1 = Θ P 2 = Θ = ( θ 1 , … , θ N ) T . The relational data matrix is symmetric, denoted by Y = ( y i j ) N × N \mathbf{Y} = (y_{ij})_{N \times N} Y = ( y ij ) N × N , where y i j = y j i y_{ij} = y_{ji} y ij = y j i records the observed relation between objects i i i and j j j . The diagonal entries correspond to self-relations and typically do not carry meaningful relational information. We set y i i = 0 y_{ii} = 0 y ii = 0 for notational convenience.
We assume that { y i j } i < j ∈ [ N ] \{y_{ij}\}_{i<j \in[N]} { y ij } i < j ∈ [ N ] are mutually independent. Let D = ( d i j ) N × N \mathbf{D} = (d_{ij})_{N \times N} D = ( d ij ) N × N with d i j = ∥ θ i − θ j ∥ 2 d_{ij} = \lVert\theta_i - \theta_j \rVert^2 d ij = ∥ θ i − θ j ∥ 2 . Thus, (1) gives
E ( y i j ) = f ( m i j ) = f ( τ − ∥ θ i − θ j ∥ 2 ) , 1 ≤ i < j ≤ N . (2) \mathbb{E}(y_{ij}) = f(m_{ij}) = f\left(\tau- \lVert\theta_i - \theta_j \rVert^2\right), \qquad1 \leq i < j \leq N. \tag*{(2)} E ( y ij ) = f ( m ij ) = f ( τ − ∥ θ i − θ j ∥ 2 ) , 1 ≤ i < j ≤ N . (2)
Therefore, objects that are closer in the latent space tend to have a larger expected relation.
Bipartite Model
We next consider bipartite relational data consisting of interactions between N N N row objects and J J J column objects. In this case, P 1 = [ N ] \mathcal{P}_1=[N] P 1 = [ N ] , P 2 = N + [ J ] \mathcal{P}_2=N+[J] P 2 = N + [ J ] and P = [ N + J ] \mathcal{P}=[N+J] P = [ N + J ] . Accordingly, Θ = ( θ 1 , … , θ N + J ) T \mathbf{\Theta}=(\theta_1,\ldots,\theta_{N+J})^{\mathsf{T}} Θ = ( θ 1 , … , θ N + J ) T with Θ P 1 = ( θ 1 , … , θ N ) T \mathbf{\Theta}_{\mathcal{P}_1}=(\theta_1,\ldots,\theta_N)^{\mathsf{T}} Θ P 1 = ( θ 1 , … , θ N ) T and Θ P 2 = ( θ N + 1 , … , θ N + J ) T \mathbf{\Theta}_{\mathcal{P}_2}=(\theta_{N+1},\ldots,\theta_{N+J})^{\mathsf{T}} Θ P 2 = ( θ N + 1 , … , θ N + J ) T . The observations are represented by an N × J N \times J N × J matrix Y = ( y i j ) N × J \mathbf{Y}=(y_{ij})_{N \times J} Y = ( y ij ) N × J , where the local entry y i j y_{ij} y ij corresponds to the relation between global objects i i i and N + j N+j N + j . We assume that { y i j } i ∈ [ N ] , j ∈ [ J ] \{y_{ij}\}_{i\in[N],\,j\in[J]} { y ij } i ∈ [ N ] , j ∈ [ J ] are mutually independent. Let D = ( d i j ) N × J \mathbf{D}=(d_{ij})_{N \times J} D = ( d ij ) N × J , where d i j = ∥ θ i − θ N + j ∥ 2 d_{ij}=\lVert\theta_i-\theta_{N+j}\rVert^2 d ij = ∥ θ i − θ N + j ∥ 2 . Then (1) becomes
E ( y i j ) = f ( m i j ) = f ( τ − ∥ θ i − θ N + j ∥ 2 ) , i ∈ [ N ] , j ∈ [ J ] . (3) \mathbb{E}(y_{ij})=f(m_{ij})=f\left(\tau-\lVert\theta_i-\theta_{N+j}\rVert^2\right), \qquad i\in[N],\ j\in[J]. \tag*{(3)} E ( y ij ) = f ( m ij ) = f ( τ − ∥ θ i − θ N + j ∥ 2 ) , i ∈ [ N ] , j ∈ [ J ] . (3)
Remark 3 (Connection to multidimensional unfolding). The bipartite model is rooted in multidimensional unfolding (MDU), where objects from the two index blocks are embedded in a common Euclidean space and the observed responses are represented through their mutual distances [35 , 7 ] . Classical MDU methods typically recover the joint configuration by directly optimizing a high-dimensional stress function over all embedding vectors. Such optimization can be computationally demanding and may suffer from degeneracy or non-identifiability [8 ] . The probabilistic unfolding model of Chen et al. [13 ] provides a statistical formulation for binary response data by linking response probabilities to latent distances, and establishes identifiability conditions and convergence rates for the recovered configuration. However, its estimation still relies on a high-dimensional optimization over the embedding vectors. Here we keep the same unfolding geometry, but work with the model (3) with an additional unknown drift parameter τ \tau τ . This formulation accommodates different response types through the link function f f f and separates the overall response level from the relative geometry of the embedding vectors.
Remark 4. The bipartite model is related to generalized linear factor models (GLFM) (Skrondal and Rabe-Hesketh, 2004; Bartholomew et al., [5 ] . A typical GLFM specifies
E ( y i j ∣ θ i , a j ) = g ( μ j + θ i T a j ) , \mathbb{E}(y_{ij}\mid\theta_i,\mathbf{a}_j)=g(\mu_j+\theta_i^{\mathsf{T}}\mathbf{a}_j), E ( y ij ∣ θ i , a j ) = g ( μ j + θ i T a j ) ,
where the item-specific intercept μ j \mu_j μ j is a main effect that controls the overall level of item j j j , while the inner product θ i T a j \theta_i^{\mathsf{T}}\mathbf{a}_j θ i T a j captures the interaction between row and column latent factors. Because the main effects and the interaction terms are freely parameterized, the latent factors θ i \theta_i θ i and loadings a j \mathbf{a}_j a j are generally identifiable only up to nonsingular linear transformations. This weakens their direct geometric interpretation. In contrast, our bipartite signal can be rewritten as
m i j = τ − ∥ θ i − a j ∥ 2 = τ − ∥ θ i ∥ 2 − ∥ a j ∥ 2 + 2 θ i T a j , m_{ij}=\tau-\lVert\theta_i-\mathbf{a}_j\rVert^2=\tau-\lVert\theta_i\rVert^2-\lVert\mathbf{a}_j\rVert^2+2\theta_i^{\mathsf{T}}\mathbf{a}_j, m ij = τ − ∥ θ i − a j ∥ 2 = τ − ∥ θ i ∥ 2 − ∥ a j ∥ 2 + 2 θ i T a j ,
which also contains a bilinear interaction term. The important difference is that the main-effect-like terms − ∥ θ i ∥ 2 -\lVert\theta_i\rVert^2 − ∥ θ i ∥ 2 and − ∥ a j ∥ 2 -\lVert\mathbf{a}_j\rVert^2 − ∥ a j ∥ 2 are not free parameters: they are tied to the same embedding vectors that generate the interaction term 2 θ i T a j 2\theta_i^{\mathsf{T}}\mathbf{a}_j 2 θ i T a j . This coupling turns the low-rank factor structure into a squared-distance structure, giving the embedding vectors a direct proximity-based interpretation. The price is that estimation is more involved than ordinary GLFM estimation: after recovering the signal matrix, one must further recover a Euclidean configuration satisfying the squared-distance representation. This motivates the denoising and configuration recovery procedures developed in the following sections.
Proposed Methods
We first introduce the refined spectral denoising procedure for signal matrix estimation and then present the configuration recovery procedures for unipartite and bipartite relational data.
Refined Spectral Denoising
Note that the same rank bound applies to both data types. In the bipartite case, recall that Θ P 1 = ( θ 1 , … , θ N ) T \boldsymbol{\Theta}_{P_1}=(\boldsymbol{\theta}_1,\ldots,\boldsymbol{\theta}_N)^{\mathsf{T}} Θ P 1 = ( θ 1 , … , θ N ) T and Θ P 2 = ( θ N + 1 , … , θ N + J ) T \boldsymbol{\Theta}_{P_2}=(\boldsymbol{\theta}_{N+1},\ldots,\boldsymbol{\theta}_{N+J})^{\mathsf{T}} Θ P 2 = ( θ N + 1 , … , θ N + J ) T , and define d P 1 = ( ∥ θ 1 ∥ 2 , … , ∥ θ N ∥ 2 ) T \boldsymbol{d}_{P_1}=(\lVert\boldsymbol{\theta}_1\rVert^2,\ldots,\lVert\boldsymbol{\theta}_N\rVert^2)^{\mathsf{T}} d P 1 = (∥ θ 1 ∥ 2 , … , ∥ θ N ∥ 2 ) T and d P 2 = ( ∥ θ N + 1 ∥ 2 , … , ∥ θ N + J ∥ 2 ) T \boldsymbol{d}_{P_2}=(\lVert\boldsymbol{\theta}_{N+1}\rVert^2,\ldots,\lVert\boldsymbol{\theta}_{N+J}\rVert^2)^{\mathsf{T}} d P 2 = (∥ θ N + 1 ∥ 2 , … , ∥ θ N + J ∥ 2 ) T . We have
M = 2 Θ P 1 Θ P 2 T + τ 1 N 1 J T − d P 1 1 J T − 1 N d P 2 T , rank ( M ) ≤ K + 2. (4) \mathbf{M}=2\boldsymbol{\Theta}_{P_1}\boldsymbol{\Theta}_{P_2}^{\mathsf{T}}+\tau\mathbf{1}_N\mathbf{1}_J^{\mathsf{T}}-\boldsymbol{d}_{P_1}\mathbf{1}_J^{\mathsf{T}}-\mathbf{1}_N\boldsymbol{d}_{P_2}^{\mathsf{T}},\qquad\operatorname{rank}(\mathbf{M})\le K+2. \tag*{(4)} M = 2 Θ P 1 Θ P 2 T + τ 1 N 1 J T − d P 1 1 J T − 1 N d P 2 T , rank ( M ) ≤ K + 2. (4)
At the level of the low-rank signal representation, the unipartite formula is obtained by taking P 1 = P 2 = [ N ] \mathcal{P}_1=\mathcal{P}_2=[N] P 1 = P 2 = [ N ] , so that Θ P 1 = Θ P 2 = Θ \boldsymbol{\Theta}_{P_1}=\boldsymbol{\Theta}_{P_2}=\boldsymbol{\Theta} Θ P 1 = Θ P 2 = Θ . Hence, letting d = ( ∥ θ 1 ∥ 2 , … , ∥ θ N ∥ 2 ) T \boldsymbol{d}=(\lVert\boldsymbol{\theta}_1\rVert^2,\ldots,\lVert\boldsymbol{\theta}_N\rVert^2)^{\mathsf{T}} d = (∥ θ 1 ∥ 2 , … , ∥ θ N ∥ 2 ) T , we have
M = 2 Θ Θ T + τ 1 N 1 N T − d 1 N T − 1 N d T , rank ( M ) ≤ K + 2. (5) \mathbf{M}=2\boldsymbol{\Theta}\boldsymbol{\Theta}^{\mathsf{T}}+\tau\mathbf{1}_N\mathbf{1}_N^{\mathsf{T}}-\boldsymbol{d}\mathbf{1}_N^{\mathsf{T}}-\mathbf{1}_N\boldsymbol{d}^{\mathsf{T}},\qquad\operatorname{rank}(\mathbf{M})\le K+2. \tag*{(5)} M = 2 Θ Θ T + τ 1 N 1 N T − d 1 N T − 1 N d T , rank ( M ) ≤ K + 2. (5)
We propose Algorithm 1, which gives an estimator for the signal matrix. This algorithm consists of two stages: the spectral denoising stage gives a fast global denoiser by exploiting the rank-( K + 2 ) (K+2) ( K + 2 ) structure in (4) and (5) ; for nonlinear links, the spectral denoising stage is followed by a one-step refinement, which uses the rescaled right singular vectors as anchors, updates the row space by solving estimating equations for each row i i i , and then updates the column space. For unipartite data, the final symmetrization and rank-( K + 2 ) (K+2) ( K + 2 ) truncation retain the leading signal components while enforcing the one-mode structure. For the bipartite case, we assume without loss of generality that N ≥ J N\ge J N ≥ J .
Remark 5. We discuss the spectral denoising stage. If f ( ⋅ ) f(\cdot) f ( ⋅ ) is the identity function, then we directly apply singular value decomposition to Y \mathbf{Y} Y and select the first K + 2 K+2 K + 2 signals to obtain X ^ \widehat{\mathbf{X}} X , which is formally defined in Algorithm 1. Note that for a general linear function f ( x ) = k x + b f(x)=kx+b f ( x ) = k x + b with k > 0 k>0 k > 0 , for any observed pair corresponding to global objects u u u and v v v ,
f ( τ − ∥ θ u − θ v ∥ 2 ) = ( k τ + b ) − ∥ k θ u − k θ v ∥ 2 , f\left(\tau-\lVert\boldsymbol{\theta}_u-\boldsymbol{\theta}_v\rVert^2\right)=(k\tau+b)-\left\lVert\sqrt{k}\boldsymbol{\theta}_u-\sqrt{k}\boldsymbol{\theta}_v\right\rVert^2, f ( τ − ∥ θ u − θ v ∥ 2 ) = ( k τ + b ) − k θ u − k θ v 2 ,
Algorithm 1 Refined Spectral Denoising
Input: the observation matrix Y \mathbf{Y} Y , latent dimension K K K , link function f ( ⋅ ) f(\cdot) f ( ⋅ ) , truncation parameters a N J a_{NJ} a N J and b N J b_{NJ} b N J , and projection constant C C C . For unipartite data, set J = N \mathcal{J} = N J = N .
Spectral denoising. Define X ^ = ( x ^ i j ) \widehat{\mathbf{X}} = (\widehat{x}_{ij}) X = ( x ij ) by
X ^ = { SVD K + 2 ( Y ) , if f ( x ) = x , ( b N J − a N J ) SVD r ~ ( Y ~ ) + a N J 1 N 1 J T , otherwise, \widehat{\mathbf{X}} =
\begin{cases}
\operatorname{SVD}_{K+2}(\mathbf{Y}), & \text{if } f(x)=x,\\
(b_{NJ}-a_{NJ})\operatorname{SVD}_{\widetilde r}(\widetilde{\mathbf{Y}})+a_{NJ}\mathbf{1}_{N}\mathbf{1}_{J}^{\mathsf{T}}, & \text{otherwise,}
\end{cases} X = { SVD K + 2 ( Y ) , ( b N J − a N J ) SVD r ( Y ) + a N J 1 N 1 J T , if f ( x ) = x , otherwise,
where
Y ~ = ( y ~ i j ) with y ~ i j = { P [ a N J , b N J ] ( y i j ) − a N J } ( b N J − a N J ) , \widetilde{\mathbf{Y}}=(\widetilde{y}_{ij})\text{ with }\widetilde{y}_{ij}=\frac{\{\mathcal{P}_{[a_{NJ},b_{NJ}]}(y_{ij})-a_{NJ}\}}{(b_{NJ}-a_{NJ})}, Y = ( y ij ) with y ij = ( b N J − a N J ) { P [ a N J , b N J ] ( y ij ) − a N J } ,
r ~ = max { K + 2 , max { k : σ k ( Y ~ ) ≥ 1.01 N } } . \widetilde r=\max\left\{K+2,\max\left\{k:\sigma_k(\widetilde{\mathbf{Y}})\ge1.01\sqrt{N}\right\}\right\}. r = max { K + 2 , max { k : σ k ( Y ) ≥ 1.01 N } } .
Let
M ^ S V D = { P [ − C , C ] ( X ^ ) , if f ( x ) = x , f − 1 { P [ f ( − C ) , f ( C ) ] ( X ^ ) } , otherwise. \widehat{\mathbf{M}}^{\mathrm{SVD}}=
\begin{cases}
\mathcal{P}_{[-C,C]}(\widehat{\mathbf{X}}), & \text{if } f(x)=x,\\
f^{-1}\left\{\mathcal{P}_{[f(-C),f(C)]}(\widehat{\mathbf{X}})\right\}, & \text{otherwise.}
\end{cases} M SVD = { P [ − C , C ] ( X ) , f − 1 { P [ f ( − C ) , f ( C )] ( X ) } , if f ( x ) = x , otherwise.
If f ( x ) = x f(x)=x f ( x ) = x , then return M ^ S V D \widehat{\mathbf{M}}^{\mathrm{SVD}} M SVD . Otherwise, perform the one-step refinement below.
One-step refinement for nonlinear link functions. Compute the rank-( K + 2 ) (K+2) ( K + 2 ) SVD of M ^ S V D \widehat{\mathbf{M}}^{\mathrm{SVD}} M SVD and let V ^ ∈ R J × ( K + 2 ) \widehat{\mathbf{V}}\in\mathbb{R}^{J\times(K+2)} V ∈ R J × ( K + 2 ) be its right singular matrix. Set
A ^ = ( α ^ 1 , … , α ^ J ) T = P C , 2 → ∞ ( J V ^ ) . \widehat{\mathbf{A}}=(\widehat{\alpha}_1,\ldots,\widehat{\alpha}_J)^{\mathsf{T}}=\mathcal{P}_{C,2\to\infty}(\sqrt{J}\widehat{\mathbf{V}}). A = ( α 1 , … , α J ) T = P C , 2 → ∞ ( J V ) .
For each row i i i , obtain β ~ i \widetilde{\boldsymbol{\beta}}_i β i by solving
∑ j ∈ J i { y i j − f ( α ^ j T β ~ i ) } α ^ j = 0 K + 2 , (6) \sum_{j\in\mathcal{J}_i}\{y_{ij}-f(\widehat{\boldsymbol{\alpha}}_j^{\mathsf{T}}\widetilde{\boldsymbol{\beta}}_i)\}\widehat{\boldsymbol{\alpha}}_j=\mathbf{0}_{K+2}, \tag*{(6)} j ∈ J i ∑ { y ij − f ( α j T β i )} α j = 0 K + 2 , (6)
where J i = [ J ] \mathcal{J}_i=[J] J i = [ J ] for bipartite data and J i = [ N ] ∖ { i } \mathcal{J}_i=[N]\setminus\{i\} J i = [ N ] ∖ { i } for unipartite data. Then, for each column j j j , obtain α ~ j \widetilde{\boldsymbol{\alpha}}_j α j by solving
∑ i ∈ I j { y i j − f ( α ~ j T β ~ i ) } β ~ i = 0 K + 2 , (7) \sum_{i\in\mathcal{I}_j}\{y_{ij}-f(\widetilde{\boldsymbol{\alpha}}_j^{\mathsf{T}}\widetilde{\boldsymbol{\beta}}_i)\}\widetilde{\boldsymbol{\beta}}_i=\mathbf{0}_{K+2}, \tag*{(7)} i ∈ I j ∑ { y ij − f ( α j T β i )} β i = 0 K + 2 , (7)
where I j = [ N ] \mathcal{I}_j=[N] I j = [ N ] for bipartite data and I j = [ N ] ∖ { j } \mathcal{I}_j=[N]\setminus\{j\} I j = [ N ] ∖ { j } for unipartite data.
Output: For bipartite data, return M ^ R S D = B ~ A ~ T \widehat{\mathbf{M}}^{\mathrm{RSD}}=\widetilde{\mathbf{B}}\widetilde{\mathbf{A}}^{\mathsf{T}} M RSD = B A T , where B ~ = ( β ~ 1 , … , β ~ N ) T \widetilde{\mathbf{B}}=(\widetilde{\boldsymbol{\beta}}_1,\ldots,\widetilde{\boldsymbol{\beta}}_N)^{\mathsf{T}} B = ( β 1 , … , β N ) T and A ~ = ( α ~ 1 , … , α ~ J ) T \widetilde{\mathbf{A}}=(\widetilde{\boldsymbol{\alpha}}_1,\ldots,\widetilde{\boldsymbol{\alpha}}_J)^{\mathsf{T}} A = ( α 1 , … , α J ) T . For unipartite data, return
M ^ R S D = SVD K + 2 { B ~ A ~ T + A ~ B ~ T 2 } . \widehat{\mathbf{M}}^{\mathrm{RSD}}=\operatorname{SVD}_{K+2}\left\{\frac{\widetilde{\mathbf{B}}\widetilde{\mathbf{A}}^{\mathsf{T}}+\widetilde{\mathbf{A}}\widetilde{\mathbf{B}}^{\mathsf{T}}}{2}\right\}. M RSD = SVD K + 2 { 2 B A T + A B T } .
which reduces to the case of the identity link function where the drift becomes k τ + b k\tau+b k τ + b and the embedding vectors are rescaled by k \sqrt{k} k . For nonlinear f ( ⋅ ) f(\cdot) f ( ⋅ ) , instead of directly performing singular value decomposition to Y \mathbf{Y} Y , we first truncate all y i j y_{ij} y ij to [ a N J , b N J ] [a_{NJ},b_{NJ}] [ a N J , b N J ] and rescale them to [ 0 , 1 ] [0,1] [ 0 , 1 ] , after which we apply singular value thresholding Chatterjee [9 ] to the transformed matrix with the threshold 1.01 N 1.01\sqrt{N} 1.01 N . The truncation levels a N J a_{NJ} a N J and b N J b_{NJ} b N J may be chosen according to the response space and tail behavior of y i j y_{ij} y ij . One such choice, together with the resulting convergence rates, is given in Corollary 1. The spectral estimate is then transformed back to the original response scale to obtain X ^ \widehat{\mathbf{X}} X . Finally, for a nonlinear link, the estimate is first projected onto f ( [ − C , C ] ) f([-C,C]) f ([ − C , C ]) and then transformed by f − 1 f^{-1} f − 1 .
Remark 6. We discuss the refinement stage, which is required if f f f is a nonlinear function. Let M ∗ \mathbf{M}^{*} M ∗ denote the true signal matrix, and write its singular value decomposition as M ∗ = U ∗ D ∗ V ∗ ⊤ \mathbf{M}^{*}=\mathbf{U}^{*}\mathbf{D}^{*}\mathbf{V}^{*\top} M ∗ = U ∗ D ∗ V ∗ ⊤ . Further let A ∗ = J V ∗ P ^ \mathbf{A}^{*}=\sqrt{J}\mathbf{V}^{*}\widehat{\mathbf{P}} A ∗ = J V ∗ P and B ∗ = J − 1 / 2 U ∗ D ∗ P ^ \mathbf{B}^{*}=J^{-1/2}\mathbf{U}^{*}\mathbf{D}^{*}\widehat{\mathbf{P}} B ∗ = J − 1/2 U ∗ D ∗ P , where P ^ ∈ arg min P ∈ O K + 2 ∥ V ^ − V ∗ P ∥ F \widehat{\mathbf{P}}\in\arg\min_{\mathbf{P}\in\mathcal{O}_{K+2}}\|\widehat{\mathbf{V}}-\mathbf{V}^{*}\mathbf{P}\|_{F} P ∈ arg min P ∈ O K + 2 ∥ V − V ∗ P ∥ F . If we replace y i j y_{ij} y ij by its expectation f ( m i j ∗ ) f(m_{ij}^{*}) f ( m ij ∗ ) in (6) , then it becomes
∑ j ∈ J i { f ( m i j ∗ ) − f ( α ^ j ⊤ β ~ i ) } α ^ j = 0 K + 2 . \sum_{j\in\mathcal{J}_{i}}\left\{f(m_{ij}^{*})-f\left(\widehat{\boldsymbol{\alpha}}_{j}^{\top}\widetilde{\boldsymbol{\beta}}_{i}\right)\right\}\widehat{\boldsymbol{\alpha}}_{j}=\mathbf{0}_{K+2}. j ∈ J i ∑ { f ( m ij ∗ ) − f ( α j ⊤ β i ) } α j = 0 K + 2 .
Therefore, β ~ i ≈ β i ∗ \widetilde{\boldsymbol{\beta}}_{i}\approx\boldsymbol{\beta}_{i}^{*} β i ≈ β i ∗ as long as A ^ ≈ A ∗ \widehat{\mathbf{A}}\approx\mathbf{A}^{*} A ≈ A ∗ , which is guaranteed by the spectral denoising stage. Similar analysis applies to (7) . Both (6) and (7) involve only K + 2 K+2 K + 2 variables which could be solved efficiently, and there exists a unique solution with probability approaching 1 1 1 , as will be shown in § 4.1 . Moreover, for each i ∈ [ N ] i\in[N] i ∈ [ N ] , (6) does not depend on other β j \boldsymbol{\beta}_{j} β j for j ≠ i j\ne i j = i . Therefore, the procedure could be accelerated by parallel computing, and the same applies to equation (7) .
Configuration Recovery
In this section, we turn to configuration recovery based on an estimate of the signal matrix M ^ \widehat{\mathbf{M}} M . Although M ^ \widehat{\mathbf{M}} M can be taken as the signal estimator returned by Algorithm 1, the recovery procedures apply more generally to any sufficiently accurate estimator of M ∗ \mathbf{M}^{*} M ∗ . Under the regularity and identifiability conditions specified in § 4.2 , uniform consistency for signal estimation translates into uniform consistency for configuration recovery.
Unipartite Configuration Recovery
We first consider configuration recovery for the unipartite case, where the obtained M ^ \widehat{\mathbf{M}} M is supposed to be symmetric. Let Θ ∗ = ( θ 1 ∗ , … , θ N ∗ ) ⊤ \mathbf{\Theta}^{*} = (\theta_{1}^{*}, \ldots, \theta_{N}^{*})^{\top} Θ ∗ = ( θ 1 ∗ , … , θ N ∗ ) ⊤ denote the true latent configuration. Note that 1 2 J N M ∗ J N = ( J N Θ ∗ ) ( J N Θ ∗ ) ⊤ \frac{1}{2}\mathbf{J}_{N}\mathbf{M}^{*}\mathbf{J}_{N} = (\mathbf{J}_{N}\mathbf{\Theta}^{*})(\mathbf{J}_{N}\mathbf{\Theta}^{*})^{\top} 2 1 J N M ∗ J N = ( J N Θ ∗ ) ( J N Θ ∗ ) ⊤ . To estimate Θ ∗ \mathbf{\Theta}^{*} Θ ∗ , let
Eigen K ( J N M ^ J N / 2 ) = ∑ k = 1 K λ ^ k u ^ k u ^ k ⊤ . \operatorname{Eigen}_{K}(\mathbf{J}_{N}\widehat{\mathbf{M}}\mathbf{J}_{N}/2) = \sum_{k=1}^{K}\widehat{\lambda}_{k}\widehat{\mathbf{u}}_{k}\widehat{\mathbf{u}}_{k}^{\top}. Eigen K ( J N M J N /2 ) = k = 1 ∑ K λ k u k u k ⊤ .
Then,
Θ ^ = ( ( λ ^ 1 ) + 1 / 2 u ^ 1 , … , ( λ ^ K ) + 1 / 2 u ^ K ) , \widehat{\mathbf{\Theta}} = \left((\widehat{\lambda}_{1})_{+}^{1/2}\widehat{\mathbf{u}}_{1},\ldots,(\widehat{\lambda}_{K})_{+}^{1/2}\widehat{\mathbf{u}}_{K}\right), Θ = ( ( λ 1 ) + 1/2 u 1 , … , ( λ K ) + 1/2 u K ) ,
where ( λ ^ k ) + = max { 0 , λ ^ k } (\widehat{\lambda}_{k})_{+} = \max\{0,\widehat{\lambda}_{k}\} ( λ k ) + = max { 0 , λ k } for k ∈ [ K ] k \in[K] k ∈ [ K ] .
Bipartite Configuration Recovery
To present the proposed method for bipartite configuration recovery, we first introduce some generic notations. Let Z = ( N U ⊤ , J W ⊤ ) ⊤ ∈ R ( N + J ) × K \mathbf{Z} = \left(\sqrt{N}\mathbf{U}^{\top},\sqrt{J}\mathbf{W}^{\top}\right)^{\top} \in\mathbb{R}^{(N+J)\times K} Z = ( N U ⊤ , J W ⊤ ) ⊤ ∈ R ( N + J ) × K , where U ∈ R N × K \mathbf{U} \in\mathbb{R}^{N\times K} U ∈ R N × K and W ∈ R J × K \mathbf{W} \in\mathbb{R}^{J\times K} W ∈ R J × K . Given an invertible matrix F ∈ R K × K \mathbf{F} \in\mathbb{R}^{K\times K} F ∈ R K × K and a vector b ∈ R K \mathbf{b} \in\mathbb{R}^{K} b ∈ R K , define γ = ( τ , vec ( F ) ⊤ , b ⊤ ) ⊤ ∈ R K ( K + 1 ) + 1 \boldsymbol{\gamma} = (\tau,\operatorname{vec}(\mathbf{F})^{\top},\mathbf{b}^{\top})^{\top} \in\mathbb{R}^{K(K+1)+1} γ = ( τ , vec ( F ) ⊤ , b ⊤ ) ⊤ ∈ R K ( K + 1 ) + 1 , where τ \tau τ is the intercept parameter in (1). Given Z \mathbf{Z} Z and γ \boldsymbol{\gamma} γ , further define the induced configurations as
Θ P 1 ( γ ; Z ) = N U F , Θ P 2 ( γ ; Z ) = J W ( F − 1 ) ⊤ + 1 J b ⊤ . (8) \mathbf{\Theta}_{\mathcal{P}_{1}}(\boldsymbol{\gamma};\mathbf{Z}) = \sqrt{N}\mathbf{U}\mathbf{F}, \qquad\mathbf{\Theta}_{\mathcal{P}_{2}}(\boldsymbol{\gamma};\mathbf{Z}) = \sqrt{J}\mathbf{W}(\mathbf{F}^{-1})^{\top}+\mathbf{1}_{J}\mathbf{b}^{\top}. \tag*{(8)} Θ P 1 ( γ ; Z ) = N UF , Θ P 2 ( γ ; Z ) = J W ( F − 1 ) ⊤ + 1 J b ⊤ . (8)
The corresponding global configuration is
Θ ( γ ; Z ) = ( Θ P 1 ( γ ; Z ) Θ P 2 ( γ ; Z ) ) . \boldsymbol{\Theta}(\gamma;\mathbf{Z}) =
\begin{pmatrix}
\boldsymbol{\Theta}_{\mathcal{P}_1}(\gamma;\mathbf{Z})\\
\boldsymbol{\Theta}_{\mathcal{P}_2}(\gamma;\mathbf{Z})
\end{pmatrix}. Θ ( γ ; Z ) = ( Θ P 1 ( γ ; Z ) Θ P 2 ( γ ; Z ) ) .
We also define the induced signal matrix M ( γ ; Z ) = ( m i j ( γ ; Z ) ) N × J \mathbf{M}(\gamma;\mathbf{Z}) = (m_{ij}(\gamma;\mathbf{Z}))_{N \times J} M ( γ ; Z ) = ( m ij ( γ ; Z ) ) N × J with
m i j ( γ ; Z ) = τ − ∥ θ i ( γ ; Z ) − θ N + j ( γ ; Z ) ∥ 2 , (9) m_{ij}(\gamma;\mathbf{Z}) = \tau- \lVert\boldsymbol{\theta}_i(\gamma;\mathbf{Z}) - \boldsymbol{\theta}_{N+j}(\gamma;\mathbf{Z}) \rVert^2, \tag*{(9)} m ij ( γ ; Z ) = τ − ∥ θ i ( γ ; Z ) − θ N + j ( γ ; Z ) ∥ 2 , (9)
where θ i ( γ ; Z ) \boldsymbol{\theta}_i(\gamma;\mathbf{Z}) θ i ( γ ; Z ) and θ N + j ( γ ; Z ) \boldsymbol{\theta}_{N+j}(\gamma;\mathbf{Z}) θ N + j ( γ ; Z ) are the embedding vectors of global objects i i i and N + j N+j N + j in Θ ( γ ; Z ) \boldsymbol{\Theta}(\gamma;\mathbf{Z}) Θ ( γ ; Z ) , respectively.
The key idea is to profile out the high-dimensional singular spaces, which are estimated directly from the double-centered signal matrix. Conditional on these spaces, summarized by Z \mathbf{Z} Z , configuration recovery only requires fixing a low-dimensional transformation: an invertible matrix F \mathbf{F} F and a shift vector b \mathbf{b} b , together with the drift parameter τ \tau τ . The following flow chart summarizes how these remaining parameters induce a configuration and a signal matrix, which is then aligned with M ^ \widehat{\mathbf{M}} M .
Figure 12 (PDF p. 20). Parameters γ = ( τ , vec ( F ) ⊤ , b ⊤ ) ⊤ \gamma= (\tau, \operatorname{vec}(\mathbf{F})^\top,\mathbf{b}^\top)^\top γ = ( τ , vec ( F ) ⊤ , b ⊤ ) ⊤ ; Induced configuration Θ P 1 ( γ ; Z ) , Θ P 2 ( γ ; Z ) \boldsymbol{\Theta}_{\mathcal{P}_1}(\gamma;\mathbf{Z}),\ \boldsymbol{\Theta}_{\mathcal{P}_2}(\gamma;\mathbf{Z}) Θ P 1 ( γ ; Z ) , Θ P 2 ( γ ; Z ) ; Induced signal M ( γ ; Z ) \mathbf{M}(\gamma;\mathbf{Z}) M ( γ ; Z ) ; Align with M ^ \widehat{\mathbf{M}} M , min γ ∥ M ( γ ; Z ) − M ^ ∥ F 2 \min_{\gamma}\lVert\mathbf{M}(\gamma;\mathbf{Z})-\widehat{\mathbf{M}}\rVert_F^2 min γ ∥ M ( γ ; Z ) − M ∥ F 2 .
We first illustrate the idea for the case where M ∗ \mathbf{M}^* M ∗ is known. Since the distances between embedding vectors are invariant under shifting, we assume Θ P 1 ∗ ⊤ 1 N = 0 \boldsymbol{\Theta}_{\mathcal{P}_1}^{*\top}\mathbf{1}_N=0 Θ P 1 ∗ ⊤ 1 N = 0 without loss of generality. By double-centering M ∗ \mathbf{M}^* M ∗ , we obtain 1 2 J N M ∗ J J = Θ P 1 ∗ ( J J Θ P 2 ∗ ) ⊤ \frac{1}{2}\mathbf{J}_N\mathbf{M}^*\mathbf{J}_J=\boldsymbol{\Theta}_{\mathcal{P}_1}^*(\mathbf{J}_J\boldsymbol{\Theta}_{\mathcal{P}_2}^*)^\top 2 1 J N M ∗ J J = Θ P 1 ∗ ( J J Θ P 2 ∗ ) ⊤ . Let 1 2 J N M ∗ J J = U ∗ Σ ∗ V ∗ ⊤ \frac{1}{2}\mathbf{J}_N\mathbf{M}^*\mathbf{J}_J=\mathbf{U}^*\boldsymbol{\Sigma}^*\mathbf{V}^{*\top} 2 1 J N M ∗ J J = U ∗ Σ ∗ V ∗ ⊤ be the singular value decomposition. To properly scale the singular vectors, we define W ∗ = 1 N J V ∗ Σ ∗ \mathbf{W}^{*} = \frac{1}{\sqrt{NJ}}\mathbf{V}^{*}\mathbf{\Sigma}^{*} W ∗ = N J 1 V ∗ Σ ∗ and construct Z ∗ = ( N U ∗ ⊤ , J W ∗ ⊤ ) ⊤ \mathbf{Z}^{*} = (\sqrt{N}\mathbf{U}^{*\top},\sqrt{J}\mathbf{W}^{*\top})^{\top} Z ∗ = ( N U ∗ ⊤ , J W ∗ ⊤ ) ⊤ . Notice that
Θ P 1 ∗ ( J J Θ P 2 ∗ ) ⊤ = 1 2 J N M ∗ J J = ( N U ∗ ⊤ ) ( J W ∗ ⊤ ) ⊤ . \mathbf{\Theta}_{\mathcal{P}_1}^{*}(\mathbf{J}_{J}\mathbf{\Theta}_{\mathcal{P}_2}^{*})^{\top} = \frac{1}{2}\mathbf{J}_{N}\mathbf{M}^{*}\mathbf{J}_{J} = (\sqrt{N}\mathbf{U}^{*\top})(\sqrt{J}\mathbf{W}^{*\top})^{\top}. Θ P 1 ∗ ( J J Θ P 2 ∗ ) ⊤ = 2 1 J N M ∗ J J = ( N U ∗ ⊤ ) ( J W ∗ ⊤ ) ⊤ .
which guarantees the existence of a vector γ 0 = ( τ 0 , vec ( F 0 ) ⊤ , b 0 ⊤ ) ⊤ \boldsymbol{\gamma}_0 = (\tau_0,\operatorname{vec}(\mathbf{F}_0)^{\top},\mathbf{b}_0^{\top})^{\top} γ 0 = ( τ 0 , vec ( F 0 ) ⊤ , b 0 ⊤ ) ⊤ such that
Θ P 1 ∗ = Θ P 1 ( γ 0 ; Z ∗ ) , Θ P 2 ∗ = Θ P 2 ( γ 0 ; Z ∗ ) . \mathbf{\Theta}_{\mathcal{P}_1}^{*} = \mathbf{\Theta}_{\mathcal{P}_1}(\boldsymbol{\gamma}_0;\mathbf{Z}^{*}), \qquad\mathbf{\Theta}_{\mathcal{P}_2}^{*} = \mathbf{\Theta}_{\mathcal{P}_2}(\boldsymbol{\gamma}_0;\mathbf{Z}^{*}). Θ P 1 ∗ = Θ P 1 ( γ 0 ; Z ∗ ) , Θ P 2 ∗ = Θ P 2 ( γ 0 ; Z ∗ ) .
Equivalently, Θ ∗ = Θ ( γ 0 ; Z ∗ ) \mathbf{\Theta}^{*} = \mathbf{\Theta}(\boldsymbol{\gamma}_0;\mathbf{Z}^{*}) Θ ∗ = Θ ( γ 0 ; Z ∗ ) . Consequently, we have M ∗ = M ( γ 0 ; Z ∗ ) \mathbf{M}^{*} = \mathbf{M}(\boldsymbol{\gamma}_0;\mathbf{Z}^{*}) M ∗ = M ( γ 0 ; Z ∗ ) . If M ∗ \mathbf{M}^{*} M ∗ is known, the true parameter γ 0 \boldsymbol{\gamma}_0 γ 0 could be recovered by minimizing the Frobenius norm ∥ M ( γ ; Z ∗ ) − M ∗ ∥ F 2 \lVert\mathbf{M}(\boldsymbol{\gamma};\mathbf{Z}^{*})-\mathbf{M}^{*}\rVert_{F}^{2} ∥ M ( γ ; Z ∗ ) − M ∗ ∥ F 2 .
For parameter estimation, the true M ∗ \mathbf{M}^{*} M ∗ is unknown, and we rely on the perturbed signal matrix M ^ \widehat{\mathbf{M}} M . For any γ \boldsymbol{\gamma} γ , Z \mathbf{Z} Z and M \mathbf{M} M , we measure the reconstruction error based on γ , Z \boldsymbol{\gamma},\mathbf{Z} γ , Z from M \mathbf{M} M by the following objective function
l ( γ ; Z , M ) = ∥ M ( γ ; Z ) − M ∥ F 2 . l(\boldsymbol{\gamma};\mathbf{Z},\mathbf{M}) = \lVert\mathbf{M}(\boldsymbol{\gamma};\mathbf{Z})-\mathbf{M}\rVert_{F}^{2}. l ( γ ; Z , M ) = ∥ M ( γ ; Z ) − M ∥ F 2 .
where M ( γ ; Z ) \mathbf{M}(\boldsymbol{\gamma};\mathbf{Z}) M ( γ ; Z ) is the signal matrix defined in (9) . The pair ( γ 0 , Z ∗ ) (\boldsymbol{\gamma}_0,\mathbf{Z}^{*}) ( γ 0 , Z ∗ ) exactly reproduces the true drifted distance matrix M ∗ \mathbf{M}^{*} M ∗ , yielding l ( γ 0 ; Z ∗ , M ∗ ) = 0 l(\boldsymbol{\gamma}_0;\mathbf{Z}^{*},\mathbf{M}^{*}) = 0 l ( γ 0 ; Z ∗ , M ∗ ) = 0 . We proceed by computing the rank-K K K singular value decomposition as SVD K ( J N M ^ J J / 2 ) = U ^ Σ ^ V ^ ⊤ \operatorname{SVD}_{K}\left(\mathbf{J}_{N}\widehat{\mathbf{M}}\mathbf{J}_{J}/2\right) = \widehat{\mathbf{U}}\widehat{\mathbf{\Sigma}}\widehat{\mathbf{V}}^{\top} SVD K ( J N M J J /2 ) = U Σ V ⊤ , where U ^ ∈ R N × K \widehat{\mathbf{U}} \in\mathbb{R}^{N\times K} U ∈ R N × K , V ^ ∈ R J × K \widehat{\mathbf{V}} \in\mathbb{R}^{J\times K} V ∈ R J × K , and Σ ^ = diag ( σ ^ 1 , … , σ ^ K ) \widehat{\mathbf{\Sigma}} = \operatorname{diag}(\widehat{\sigma}_1,\ldots,\widehat{\sigma}_K) Σ = diag ( σ 1 , … , σ K ) . Similar to the case of M ∗ \mathbf{M}^{*} M ∗ , we define W ^ = 1 N J V ^ Σ ^ \widehat{\mathbf{W}} = \frac{1}{\sqrt{NJ}}\widehat{\mathbf{V}}\widehat{\mathbf{\Sigma}} W = N J 1 V Σ and construct the matrix Z ^ = ( N U ^ ⊤ , J W ^ ⊤ ) ⊤ \widehat{\mathbf{Z}} = (\sqrt{N}\widehat{\mathbf{U}}^{\top},\sqrt{J}\widehat{\mathbf{W}}^{\top})^{\top} Z = ( N U ⊤ , J W ⊤ ) ⊤ . To estimate γ 0 \boldsymbol{\gamma}_0 γ 0 , we solve the following constrained optimization problem:
γ ^ ∈ arg min γ ∈ Γ l ( γ ; Z ^ , M ^ ) with l ( γ ; Z ^ , M ^ ) = ∥ M ( γ ; Z ^ ) − M ^ ∥ F 2 , Γ = { γ = ( τ , vec ( F ) ⊤ , b ⊤ ) ⊤ ∈ R K ( K + 1 ) + 1 : ∥ γ ∥ ≤ C , det ( F ) ≠ 0 , and F ⊤ F is diagonal with non-increasing entries } . (10) \begin{aligned}
\widehat{\boldsymbol{\gamma}} &\in\underset{\boldsymbol{\gamma}\in\Gamma}{\arg\min}\; l(\boldsymbol{\gamma};\widehat{\mathbf{Z}},\widehat{\mathbf{M}}) \quad\text{with} \quad l(\boldsymbol{\gamma};\widehat{\mathbf{Z}},\widehat{\mathbf{M}}) = \lVert\mathbf{M}(\boldsymbol{\gamma};\widehat{\mathbf{Z}})-\widehat{\mathbf{M}}\rVert_{F}^{2},\\
\Gamma&= \left\{\boldsymbol{\gamma} = (\tau,\operatorname{vec}(\mathbf{F})^{\top},\mathbf{b}^{\top})^{\top}\in\mathbb{R}^{K(K+1)+1}: \lVert\boldsymbol{\gamma}\rVert\le C,\ \det(\mathbf{F})\ne0,\right.\\
&\hspace{3cm}\left.\text{and }\mathbf{F}^{\top}\mathbf{F}\text{ is diagonal with non-increasing entries}\right\}. \tag*{(10)}
\end{aligned} γ Γ ∈ γ ∈ Γ arg min l ( γ ; Z , M ) with l ( γ ; Z , M ) = ∥ M ( γ ; Z ) − M ∥ F 2 , = { γ = ( τ , vec ( F ) ⊤ , b ⊤ ) ⊤ ∈ R K ( K + 1 ) + 1 : ∥ γ ∥ ≤ C , det ( F ) = 0 , and F ⊤ F is diagonal with non-increasing entries } . (10)
for a sufficiently large fixed constant C > 0 C > 0 C > 0 .
It is worth mentioning that the optimization problem (10) involves only K ( K + 1 ) + 1 K(K+1)+1 K ( K + 1 ) + 1 parameters, so it is computationally fast. This is in contrast to the SMACOF method [15 ] and likelihood-based optimization [13 ] , which involve at least ( N + J ) K (N+J)K ( N + J ) K parameters. We summarize the complete estimation procedure in Algorithm 2.
Algorithm 2 Bipartite configuration recovery
Input: Perturbed signal matrix M ^ \widehat{\mathbf{M}} M .
Compute SVD K ( J N M ^ J J / 2 ) = U ^ Σ ^ V ^ T \operatorname{SVD}_{K}(\mathbf{J}_{N}\widehat{\mathbf{M}}\mathbf{J}_{J}/2)=\widehat{\mathbf{U}}\widehat{\mathbf{\Sigma}}\widehat{\mathbf{V}}^{\mathsf{T}} SVD K ( J N M J J /2 ) = U Σ V T , where the signs of each singular-vector pair are chosen so that the first nonzero entry of each column of U ^ \widehat{\mathbf{U}} U is positive. Define W ^ = V ^ Σ ^ / N J \widehat{\mathbf{W}}=\widehat{\mathbf{V}}\widehat{\mathbf{\Sigma}}/\sqrt{NJ} W = V Σ / N J and Z ^ = ( N U ^ T , J W ^ T ) T \widehat{\mathbf{Z}}=(\sqrt{N}\widehat{\mathbf{U}}^{\mathsf{T}},\sqrt{J}\widehat{\mathbf{W}}^{\mathsf{T}})^{\mathsf{T}} Z = ( N U T , J W T ) T .
Solve γ ^ ∈ arg min γ ∈ Γ l ( γ ; Z ^ , M ^ ) \widehat{\gamma}\in\operatorname*{arg\,min}_{\gamma\in\Gamma}l(\gamma;\widehat{\mathbf{Z}},\widehat{\mathbf{M}}) γ ∈ arg min γ ∈ Γ l ( γ ; Z , M ) , where l ( γ ; Z ^ , M ^ ) l(\gamma;\widehat{\mathbf{Z}},\widehat{\mathbf{M}}) l ( γ ; Z , M ) and Γ \Gamma Γ are given in (10).
Output:
Θ ^ P 1 = Θ P 1 ( γ ^ ; Z ^ ) = N U ^ F ^ \widehat{\mathbf{\Theta}}_{P1}=\mathbf{\Theta}_{P1}(\widehat{\gamma};\widehat{\mathbf{Z}})=\sqrt{N}\widehat{\mathbf{U}}\widehat{\mathbf{F}} Θ P 1 = Θ P 1 ( γ ; Z ) = N U F ,
Θ ^ P 2 = Θ P 2 ( γ ^ ; Z ^ ) = J W ^ ( F ^ − 1 ) T + 1 J b ^ T \widehat{\mathbf{\Theta}}_{P2}=\mathbf{\Theta}_{P2}(\widehat{\gamma};\widehat{\mathbf{Z}})=\sqrt{J}\widehat{\mathbf{W}}(\widehat{\mathbf{F}}^{-1})^{\mathsf{T}}+\mathbf{1}_{J}\widehat{\mathbf{b}}^{\mathsf{T}} Θ P 2 = Θ P 2 ( γ ; Z ) = J W ( F − 1 ) T + 1 J b T ,
and Θ ^ = ( Θ ^ P 1 Θ ^ P 2 ) \widehat{\mathbf{\Theta}}=\begin{pmatrix}\widehat{\mathbf{\Theta}}_{P1}\\\widehat{\mathbf{\Theta}}_{P2}\end{pmatrix} Θ = ( Θ P 1 Θ P 2 ) .
Theoretical Guarantees
Throughout this section, we take the latent dimension K K K to be fixed, and suppose J ≤ N = J O ( 1 ) J \le N = J^{O(1)} J ≤ N = J O ( 1 ) . For unipartite data, we set J = N J = N J = N .
Signal Matrix Estimation
We study the asymptotic properties of the signal estimators produced by Algorithm 1. The purpose of this section is to show that Algorithm 1 produces an entrywise accurate estimate of the signal matrix M ∗ \mathbf{M}^{*} M ∗ . This entrywise control will serve as the input condition for the configuration recovery analysis in § 4.2 .
Let Θ ∗ = ( θ u ∗ ) u ∈ P \boldsymbol{\Theta}^{*} = (\boldsymbol{\theta}^{*}_{u})_{u\in\mathcal{P}} Θ ∗ = ( θ u ∗ ) u ∈ P be the true global latent configuration, M ∗ \mathbf{M}^{*} M ∗ be the true signal matrix, and τ 0 \tau_{0} τ 0 be the true value of the drift parameter τ \tau τ . For unipartite data, P = [ N ] \mathcal{P} = [N] P = [ N ] and Θ ∗ = ( θ 1 ∗ , … , θ N ∗ ) ⊤ \boldsymbol{\Theta}^{*} = (\boldsymbol{\theta}^{*}_{1},\ldots,\boldsymbol{\theta}^{*}_{N})^{\top} Θ ∗ = ( θ 1 ∗ , … , θ N ∗ ) ⊤ . For bipartite data, P = [ N + J ] \mathcal{P} = [N+J] P = [ N + J ] , with configurations
Θ P 1 ∗ = ( θ 1 ∗ , … , θ N ∗ ) ⊤ , Θ P 2 ∗ = ( θ N + 1 ∗ , … , θ N + J ∗ ) ⊤ . \boldsymbol{\Theta}^{*}_{\mathcal{P}_{1}} = (\boldsymbol{\theta}^{*}_{1},\ldots,\boldsymbol{\theta}^{*}_{N})^{\top}, \qquad\boldsymbol{\Theta}^{*}_{\mathcal{P}_{2}} = (\boldsymbol{\theta}^{*}_{N+1},\ldots,\boldsymbol{\theta}^{*}_{N+J})^{\top}. Θ P 1 ∗ = ( θ 1 ∗ , … , θ N ∗ ) ⊤ , Θ P 2 ∗ = ( θ N + 1 ∗ , … , θ N + J ∗ ) ⊤ .
Let M ∗ = U ∗ D ∗ V ∗ ⊤ \mathbf{M}^{*} = \mathbf{U}^{*}\mathbf{D}^{*}\mathbf{V}^{*\top} M ∗ = U ∗ D ∗ V ∗ ⊤ be a rank-( K + 2 ) (K+2) ( K + 2 ) singular value decomposition of M ∗ \mathbf{M}^{*} M ∗ . For the spectral estimator M ^ S V D \widehat{\mathbf{M}}^{\mathrm{SVD}} M SVD , let U ^ D ^ V ^ ⊤ : = SVD K + 2 ( M ^ S V D ) \widehat{\mathbf{U}}\widehat{\mathbf{D}}\widehat{\mathbf{V}}^{\top} := \operatorname{SVD}_{K+2}(\widehat{\mathbf{M}}^{\mathrm{SVD}}) U D V ⊤ := SVD K + 2 ( M SVD ) denote its rank-( K + 2 ) (K+2) ( K + 2 ) truncated singular value decomposition.
To derive the theoretical results, we impose several regularity conditions on the boundedness of the configuration and signal matrix, the tail behavior of the random noise, the smoothness and monotonicity of the link function, and the spectral structure of the signal matrix.
Assumption 1. Suppose the following conditions hold.
1.1. max u ∈ P ∥ θ u ∗ ∥ ≤ C \max_{u\in\mathcal{P}}\|\boldsymbol{\theta}^{*}_{u}\| \le C max u ∈ P ∥ θ u ∗ ∥ ≤ C and ∥ M ∗ ∥ max ≤ C \|\mathbf{M}^{*}\|_{\max} \le C ∥ M ∗ ∥ m a x ≤ C for some constant C > 0 C > 0 C > 0 .
1.2. max i ∈ [ N ] , j ∈ [ J ] ∥ y i j − f ( m i j ∗ ) ∥ ψ 1 = O ( 1 ) \max_{i\in[N],\,j\in[J]}\|y_{ij} - f(m^{*}_{ij})\|_{\psi_{1}} = O(1) max i ∈ [ N ] , j ∈ [ J ] ∥ y ij − f ( m ij ∗ ) ∥ ψ 1 = O ( 1 ) as N , J → ∞ N,J \to\infty N , J → ∞ .
1.3. The link function f f f is twice continuously differentiable and f ′ ( x ) > 0 f'(x)>0 f ′ ( x ) > 0 for x ∈ R x\in\mathbb{R} x ∈ R .
1.4. The singular vectors of M ∗ \mathbf{M}^* M ∗ satisfy ∥ U ∗ ∥ 2 → ∞ ≲ N − 1 / 2 \lVert\mathbf{U}^*\rVert_{2\to\infty}\lesssim N^{-1/2} ∥ U ∗ ∥ 2 → ∞ ≲ N − 1/2 and ∥ V ∗ ∥ 2 → ∞ ≲ J − 1 / 2 \lVert\mathbf{V}^*\rVert_{2\to\infty}\lesssim J^{-1/2} ∥ V ∗ ∥ 2 → ∞ ≲ J − 1/2 .
1.5. The singular values of M ∗ \mathbf{M}^* M ∗ satisfy σ K + 2 ( M ∗ ) ≳ N J \sigma_{K+2}(\mathbf{M}^*)\gtrsim\sqrt{NJ} σ K + 2 ( M ∗ ) ≳ N J .
Remark 8. Assumption 1.1 requires that the global latent configuration lies in a compact subset of R K \mathbb{R}^{K} R K , which is a standard condition in the literature of low-rank matrix estimation and multidimensional unfolding [13 , 43 ] . Assumption 1.2 controls the tail behavior of the random noises, which holds for many model types in (1), including linear, logistic, Probit, and Poisson models. Assumption 1.3 imposes smoothness and monotonicity conditions on the link function. Since f ′ f' f ′ is continuous and strictly positive, it is bounded away from zero on any fixed compact interval. In particular, under Assumption 1.1, sup y ∈ [ f ( − C ) , f ( C ) ] ∣ ( f − 1 ) ′ ( y ) ∣ = sup ∣ m ∣ ≤ C 1 f ′ ( m ) < ∞ \sup_{y\in[f(-C),f(C)]}\lvert(f^{-1})'(y)\rvert=\sup_{\lvert m\rvert\le C}\frac{1}{f'(m)}<\infty sup y ∈ [ f ( − C ) , f ( C )] ∣( f − 1 ) ′ ( y )∣ = sup ∣ m ∣ ≤ C f ′ ( m ) 1 < ∞ . Moreover, the continuity of f ′ f' f ′ and f ′ ′ f'' f ′′ provides the derivative bounds required in the Taylor expansions of the refinement analysis. Assumptions 1.4 and 1.5 are standard incoherence and signal strength conditions that prevent overly concentrated singular vectors and degenerate singular values.
Error Bounds for Spectral Denoising
We next establish error bounds for the signal matrix estimator M ^ S V D \widehat{\mathbf{M}}^{\mathrm{SVD}} M SVD obtained from the spectral denoising stage of Algorithm 1.
Theorem 1. For M ^ S V D \widehat{\mathbf{M}}^{\mathrm{SVD}} M SVD , the following error bounds hold.
(Identity link ) Suppose Assumption 1.1 and 1.2 hold. If f ( x ) = x f(x)=x f ( x ) = x , then
1 N J ∥ M ^ S V D − M ∗ ∥ F 2 = O p ( J − 1 ) . \frac{1}{NJ}\left\lVert\widehat{\mathbf{M}}^{\mathrm{SVD}}-\mathbf{M}^*\right\rVert_F^2=O_p(J^{-1}). N J 1 M SVD − M ∗ F 2 = O p ( J − 1 ) .
If, in addition, Assumptions 1.4 and 1.5 hold, then
∥ M ^ S V D − M ∗ ∥ max = O p ( J − 1 / 2 log ( N J ) ) . \left\|\widehat{\mathbf{M}}^{\mathrm{SVD}}-\mathbf{M}^{*}\right\|_{\max}=O_p\left(J^{-1/2}\sqrt{\log(NJ)}\right). M SVD − M ∗ m a x = O p ( J − 1/2 log ( N J ) ) .
(Nonlinear link) Suppose Assumption 1.1 and 1.3 hold. Let a N J < b N J a_{NJ}<b_{NJ} a N J < b N J be deterministic truncation levels and define R N J = b N J − a N J R_{NJ}=b_{NJ}-a_{NJ} R N J = b N J − a N J . Further suppose R N J + r N J = o ( J ) R_{NJ}+r_{NJ}=o(\sqrt{J}) R N J + r N J = o ( J ) , where
r N J = { max i ∈ [ N ] , j ∈ [ J ] ∣ E { P [ a N J , b N J ] ( y i j ) − f ( m i j ∗ ) } ∣ , bipartite , max i ≠ j ∣ E { P [ a N J , b N J ] ( y i j ) − f ( m i j ∗ ) } ∣ , unipartite . r_{NJ}=
\begin{cases}
\max_{i\in[N],\,j\in[J]}\left|\mathbb{E}\left\{\mathcal{P}_{[a_{NJ},b_{NJ}]}(y_{ij})-f(m_{ij}^{*})\right\}\right|, & \text{bipartite},\\[1.2em]
\max_{i\ne j}\left|\mathbb{E}\left\{\mathcal{P}_{[a_{NJ},b_{NJ}]}(y_{ij})-f(m_{ij}^{*})\right\}\right|, & \text{unipartite}.
\end{cases} r N J = ⎩ ⎨ ⎧ max i ∈ [ N ] , j ∈ [ J ] E { P [ a N J , b N J ] ( y ij ) − f ( m ij ∗ ) } , max i = j E { P [ a N J , b N J ] ( y ij ) − f ( m ij ∗ ) } , bipartite , unipartite .
For bipartite data, the normalized Frobenius error satisfies
1 N J ∥ M ^ S V D − M ∗ ∥ F 2 = O p ( R N J { ( R N J + r N J ) 2 J } 1 K + 2 + r N J ( R N J + r N J ) ) . \frac{1}{NJ}\left\|\widehat{\mathbf{M}}^{\mathrm{SVD}}-\mathbf{M}^{*}\right\|_{F}^{2}
=O_p\left(R_{NJ}\left\{\frac{(R_{NJ}+r_{NJ})^{2}}{J}\right\}^{\frac{1}{K+2}}+r_{NJ}(R_{NJ}+r_{NJ})\right). N J 1 M SVD − M ∗ F 2 = O p ( R N J { J ( R N J + r N J ) 2 } K + 2 1 + r N J ( R N J + r N J ) ) .
The same bound holds for unipartite data, provided that f ( τ 0 ) ∈ [ a N N , b N N ] f(\tau_0)\in[a_{NN},b_{NN}] f ( τ 0 ) ∈ [ a N N , b N N ] .
Remark 9. For the identity link, the rate O p ( J − 1 ) O_p(J^{-1}) O p ( J − 1 ) agrees with the minimax order for fixed-rank matrix estimation under additive noise and full observation [2 ] . Under the additional incoherence and signal strength conditions in Assumption 1, the same spectral estimator is also consistent in max-norm. The logarithmic factor reflects the cost of controlling all entries simultaneously, relative to the J − 1 / 2 J^{-1/2} J − 1/2 scale, which is consistent with existing literature on entrywise perturbation for low-rank spectral estimators [1 ] . Consequently, for the identity link, the spectral estimator itself already provides the uniform signal accuracy required for subsequent configuration recovery, and no refinement step is needed. For nonlinear links, Theorem 1 provides a rate for the proposed spectral estimator under a mean formulation without requiring a fully specified likelihood. Under stronger distributional assumptions, results for related generalized latent factor models suggest the parametric benchmark O p ( J − 1 ) O_p(J^{-1}) O p ( J − 1 ) under the normalized Frobenius loss for fixed K K K and N ≥ J N \ge J N ≥ J [12 ] . When a full likelihood is available, the first-stage signal estimator may instead be constructed using, for example, a low-rank constrained maximum likelihood estimator or a nuclear-norm penalized likelihood estimator. The subsequent refinement, when applicable, and configuration recovery procedures are modular and can be combined with such alternative signal estimators provided that the required initial estimation accuracy is satisfied.
Corollary 1. Suppose Assumptions 1.1, 1.2, and 1.3 hold, and let f f f be nonlinear. Let Y N J \mathcal{Y}_{NJ} Y N J denote the closure of the union of the supports of all response variables y i j y_{ij} y ij under consideration, and suppose that inf Y N J < sup Y N J \inf\mathcal{Y}_{NJ} < \sup\mathcal{Y}_{NJ} inf Y N J < sup Y N J for all sufficiently large N N N and J J J . Then there exists a fixed constant κ > 0 \kappa> 0 κ > 0 such that, taking a N J = max { inf Y N J , − κ log J } a_{NJ} = \max\{\inf\mathcal{Y}_{NJ},-\kappa\log J\} a N J = max { inf Y N J , − κ log J } and b N J = min { sup Y N J , κ log J } b_{NJ} = \min\{\sup\mathcal{Y}_{NJ},\kappa\log J\} b N J = min { sup Y N J , κ log J } , we have
1 N J ∥ M ^ S V D − M ∗ ∥ F 2 = O p ( J − 1 K + 2 ( log J ) K + 4 K + 2 ) . \frac{1}{NJ}\left\|\widehat{\mathbf{M}}^{\mathrm{SVD}}-\mathbf{M}^{*}\right\|_{F}^{2}
=O_p\left(J^{-\frac{1}{K+2}}(\log J)^{\frac{K+4}{K+2}}\right). N J 1 M SVD − M ∗ F 2 = O p ( J − K + 2 1 ( log J ) K + 2 K + 4 ) .
If, in addition, the response supports are uniformly bounded, that is, sup N , J sup y ∈ Y N J ∣ y ∣ < ∞ \sup_{N,J}\sup_{y\in\mathcal{Y}_{NJ}}|y|<\infty sup N , J sup y ∈ Y N J ∣ y ∣ < ∞ , then, for all sufficiently large J J J , a N J = inf Y N J a_{NJ}=\inf\mathcal{Y}_{NJ} a N J = inf Y N J and b N J = sup Y N J b_{NJ}=\sup\mathcal{Y}_{NJ} b N J = sup Y N J . Consequently,
1 N J ∥ M ^ S V D − M ∗ ∥ F 2 = O p ( J − 1 K + 2 ) . \frac{1}{NJ}\left\|\widehat{\mathbf{M}}^{\mathrm{SVD}}-\mathbf{M}^{*}\right\|_{F}^{2}
=O_p\left(J^{-\frac{1}{K+2}}\right). N J 1 M SVD − M ∗ F 2 = O p ( J − K + 2 1 ) .
The same conclusions hold for unipartite data with J = N J=N J = N , provided that f ( τ 0 ) ∈ [ a N N , b N N ] f(\tau_0)\in[a_{NN},b_{NN}] f ( τ 0 ) ∈ [ a N N , b N N ] for all sufficiently large N N N .
Remark 10. We illustrate the preceding results with several common response distributions. For Gaussian responses with the identity link, Theorem 1 gives
1 N J ∥ M ^ S V D − M ∗ ∥ F 2 = O p ( J − 1 ) . \frac{1}{NJ}\left\|\widehat{\mathbf{M}}^{\mathrm{SVD}}-\mathbf{M}^{*}\right\|_{F}^{2}=O_p(J^{-1}). N J 1 M SVD − M ∗ F 2 = O p ( J − 1 ) .
For binomial responses y i j ∼ Binomial ( L , σ ( m i j ∗ ) ) y_{ij} \sim\operatorname{Binomial}(L,\sigma(m^*_{ij})) y ij ∼ Binomial ( L , σ ( m ij ∗ )) with fixed L L L and σ ( m ) = ( 1 + e − m ) − 1 \sigma(m) = (1 + e^{-m})^{-1} σ ( m ) = ( 1 + e − m ) − 1 , the link function is f ( m ) = L σ ( m ) f(m) = L\sigma(m) f ( m ) = L σ ( m ) and the response space is Y = { 0 , … , L } \mathcal{Y} = \{0,\ldots,L\} Y = { 0 , … , L } . Taking a N J = 0 a_{NJ} = 0 a N J = 0 and b N J = L b_{NJ} = L b N J = L gives R N J = L R_{NJ} = L R N J = L and r N J = 0 r_{NJ} = 0 r N J = 0 , and hence
1 N J ∥ M ^ S V D − M ∗ ∥ F 2 = O p ( J − 1 K + 2 ) . \frac{1}{NJ}\|\widehat{\mathbf{M}}^{\mathrm{SVD}}-\mathbf{M}^*\|_F^2 = O_p\left(J^{-\frac{1}{K+2}}\right). N J 1 ∥ M SVD − M ∗ ∥ F 2 = O p ( J − K + 2 1 ) .
The Bernoulli model corresponds to L = 1 L = 1 L = 1 . For Poisson responses y i j ∼ Poisson ( exp ( m i j ∗ ) ) y_{ij} \sim\operatorname{Poisson}(\exp(m^*_{ij})) y ij ∼ Poisson ( exp ( m ij ∗ )) , the response space is Y = { 0 , 1 , … } \mathcal{Y} = \{0,1,\ldots\} Y = { 0 , 1 , … } . Taking a N J = 0 a_{NJ} = 0 a N J = 0 and b N J = κ log J b_{NJ} = \kappa\log J b N J = κ log J for any fixed κ > 0 \kappa> 0 κ > 0 gives R N J ≍ log J R_{NJ} \asymp\log J R N J ≍ log J , while r N J r_{NJ} r N J vanishes faster than any polynomial in J J J . Therefore,
1 N J ∥ M ^ S V D − M ∗ ∥ F 2 = O p ( J − 1 K + 2 ( log J ) K + 4 K + 2 ) . \frac{1}{NJ}\|\widehat{\mathbf{M}}^{\mathrm{SVD}}-\mathbf{M}^*\|_F^2 = O_p\left(J^{-\frac{1}{K+2}}(\log J)^{\frac{K+4}{K+2}}\right). N J 1 ∥ M SVD − M ∗ ∥ F 2 = O p ( J − K + 2 1 ( log J ) K + 2 K + 4 ) .
Entrywise Consistency for One-Step Refinement
For the identity link, Theorem 1 shows that the spectral estimator M ^ S V D \widehat{\mathbf{M}}^{\mathrm{SVD}} M SVD is already uniformly consistent under the max norm, so no refinement is required. For nonlinear links, the spectral denoising results above provide normalized Frobenius consistency but do not directly yield entrywise control. We therefore apply the one-step refinement to upgrade a Frobenius-norm accurate initializer to a max-norm accurate estimator. The following generic refinement result applies to both observation structures. Throughout the refinement analysis, the projection radius C C C is chosen as a sufficiently large fixed constant such that ∥ A ∗ ∥ 2 → ∞ ≤ C \|\mathbf{A}^*\|_{2\to\infty} \le C ∥ A ∗ ∥ 2 → ∞ ≤ C , with J = N J = N J = N for unipartite data.
Theorem 2. Suppose that f f f is nonlinear and Assumption 1 holds. Let M i n i t \mathbf{M}^{\mathrm{init}} M init be an initial estimator of M ∗ \mathbf{M}^* M ∗ satisfying ∥ M i n i t − M ∗ ∥ F = O p ( e M , F ) \|\mathbf{M}^{\mathrm{init}}-\mathbf{M}^*\|_F = O_p(e_{M,F}) ∥ M init − M ∗ ∥ F = O p ( e M , F ) for some deterministic rate sequence e M , F e_{M,F} e M , F such that
N ≲ e M , F ≪ N J ( log ( N J ) ) − 2 . \sqrt{N} \lesssim e_{M,F} \ll\sqrt{NJ}(\log(NJ))^{-2}. N ≲ e M , F ≪ N J ( log ( N J ) ) − 2 .
Apply the one-step refinement stage of Algorithm 1 with initialization M i n i t \mathbf{M}^{\mathrm{init}} M init . Then, with probability tending to one, all row- and column-wise estimating equations admit unique solutions.
Let M ^ r e f \widehat{\mathbf{M}}^{\mathrm{ref}} M ref denote the resulting refined estimator. Then
∥ M ^ r e f − M ∗ ∥ max = O p ( ( N J ) − 1 / 2 ( log ( N J ) ) 2 e M , F ) . \left\|\widehat{\mathbf{M}}^{\mathrm{ref}}-\mathbf{M}^{*}\right\|_{\max}=O_{p}\left((NJ)^{-1/2}(\log(NJ))^{2}e_{\mathrm{M},F}\right). M ref − M ∗ m a x = O p ( ( N J ) − 1/2 ( log ( N J ) ) 2 e M , F ) .
Remark 11. Theorem 2 shows that the one-step refinement stage upgrades a Frobenius-norm accurate initializer into an entrywise accurate estimator while preserving its normalized Frobenius convergence rate up to a squared logarithmic factor. The intuition can be seen under the factor representation M ∗ = B ∗ A ∗ ⊤ \mathbf{M}^{*}=\mathbf{B}^{*}\mathbf{A}^{*\top} M ∗ = B ∗ A ∗ ⊤ . The initial estimator gives an accurate estimate of the leading right singular subspace and hence provides an anchor matrix A ^ \widehat{\mathbf{A}} A close to A ∗ \mathbf{A}^{*} A ∗ . Given the anchor, the estimation errors for B ~ \widetilde{\mathbf{B}} B from the row-wise estimating equations come from both the random noise and the plug-in error of A ^ \widehat{\mathbf{A}} A . The column-wise estimating equations then use B ~ \widetilde{\mathbf{B}} B as the anchor matrix and similarly refine A ^ \widehat{\mathbf{A}} A . These two factor-level refinements yield two-to-infinity norm error controls for both B ~ \widetilde{\mathbf{B}} B and A ~ \widetilde{\mathbf{A}} A , and hence yield an entrywise error bound for M ^ r e f \widehat{\mathbf{M}}^{\mathrm{ref}} M ref .
Corollary 2. Suppose that f f f is nonlinear and Assumption 1 holds, with J = N J=N J = N for unipartite data. Let a N J a_{NJ} a N J and b N J b_{NJ} b N J be chosen as in Corollary 1. For unipartite data, additionally suppose that f ( τ 0 ) ∈ [ a N N , b N N ] f(\tau_{0})\in[a_{NN},b_{NN}] f ( τ 0 ) ∈ [ a N N , b N N ] for all sufficiently large N N N . Then
∥ M ^ R S D − M ∗ ∥ max = O p ( ( log ( N J ) ) 2 J − 1 2 ( K + 2 ) ( log J ) K + 4 2 ( K + 2 ) ) . \left\|\widehat{\mathbf{M}}^{\mathrm{RSD}}-\mathbf{M}^{*}\right\|_{\max}
=O_{p}\left((\log(NJ))^{2}J^{-\frac{1}{2(K+2)}}(\log J)^{\frac{K+4}{2(K+2)}}\right). M RSD − M ∗ m a x = O p ( ( log ( N J ) ) 2 J − 2 ( K + 2 ) 1 ( log J ) 2 ( K + 2 ) K + 4 ) . If the response supports are uniformly bounded, the factor ( log J ) K + 4 2 ( K + 2 ) (\log J)^{\frac{K+4}{2(K+2)}} ( log J ) 2 ( K + 2 ) K + 4 can be omitted.
Theorem 1 and Corollary 2 together show that Algorithm 1 yields uniform entrywise consistency under both link regimes. For the identity link, this property is already achieved by the spectral estimator and no refinement is needed; for nonlinear links, the one-step refinement upgrades a Frobenius-consistent spectral initializer under the stated truncation choices to uniform entrywise consistency. The resulting max-norm bounds provide the uniform signal estimation accuracy required for the subsequent configuration recovery in both the unipartite and bipartite settings.
Configuration Recovery
In this section, let M ^ \widehat{\mathbf{M}} M be a generic estimator of M ∗ \mathbf{M}^{*} M ∗ such that ξ = ∥ M ^ − M ∗ ∥ max = o ( 1 ) \xi= \lVert\widehat{\mathbf{M}}-\mathbf{M}^{*}\rVert_{\max}=o(1) ξ = ∥ M − M ∗ ∥ m a x = o ( 1 ) .
Unipartite Configuration Recovery
We first establish the consistency of the unipartite configuration recovery procedure introduced in § 3.2 . The next theorem gives the configuration recovery error based on a generic uniformly consistent estimator M ^ \widehat{\mathbf{M}} M .
Theorem 3. Suppose Assumption 1.1 holds, 1 N ⊤ Θ ∗ = 0 \mathbf{1}_{N}^{\top}\mathbf{\Theta}^{*}=0 1 N ⊤ Θ ∗ = 0 , and
lim inf N → ∞ λ min ( N − 1 Θ ∗ ⊤ Θ ∗ ) > 0. \liminf_{N\to\infty}\lambda_{\min}\left(N^{-1}\mathbf{\Theta}^{*\top}\mathbf{\Theta}^{*}\right)>0. N → ∞ lim inf λ m i n ( N − 1 Θ ∗ ⊤ Θ ∗ ) > 0. Let M ^ \widehat{\mathbf{M}} M be symmetric and let Θ ^ \widehat{\mathbf{\Theta}} Θ be the estimator obtained from the unipartite configuration recovery procedure in § 3.2 . Then d c ( Θ ^ , Θ ∗ ) ≲ ξ d_{c}(\widehat{\mathbf{\Theta}},\mathbf{\Theta}^{*})\lesssim\xi d c ( Θ , Θ ∗ ) ≲ ξ .
According to Theorem 3, for unipartite data, a uniformly accurate estimator of the signal matrix translates directly into uniform recovery of the latent configuration, without further deterioration in the convergence rate.
Bipartite Configuration Recovery
For γ = ( τ , vec ( F ) ⊤ , b ⊤ ) ⊤ \gamma=(\tau,\operatorname{vec}(\mathbf{F})^{\top},\mathbf{b}^{\top})^{\top} γ = ( τ , vec ( F ) ⊤ , b ⊤ ) ⊤ , define its sign-equivalence class by
[ γ ] : = { ( τ , vec ( F Q ) ⊤ , ( Q ⊤ b ) ⊤ ) : Q ∈ S K } . [\gamma]:=\left\{(\tau,\operatorname{vec}(\mathbf{F}\mathbf{Q})^{\top},(\mathbf{Q}^{\top}\mathbf{b})^{\top}):\mathbf{Q}\in\mathcal{S}_{K}\right\}. [ γ ] := { ( τ , vec ( FQ ) ⊤ , ( Q ⊤ b ) ⊤ ) : Q ∈ S K } .
Thus, two parameter vectors are sign-equivalent if they differ only by column-wise sign changes of F \mathbf{F} F and the corresponding sign changes of b \mathbf{b} b . To ensure the statistical consistency of the recovered latent configurations, we impose the following regularity conditions on the true parameters.
Assumption 2. Suppose γ 0 ∈ Γ \gamma_0 \in\Gamma γ 0 ∈ Γ and the following conditions hold.
2.1. 1 N T Θ P 1 ∗ = 0 \mathbf{1}_N^{\mathsf{T}}\boldsymbol{\Theta}^{*}_{\mathcal{P}_1} = 0 1 N T Θ P 1 ∗ = 0 and N − 1 Θ P 1 ∗ T Θ P 1 ∗ N^{-1}\boldsymbol{\Theta}^{*\mathsf{T}}_{\mathcal{P}_1}\boldsymbol{\Theta}^{*}_{\mathcal{P}_1} N − 1 Θ P 1 ∗ T Θ P 1 ∗ is diagonal and converges to a matrix with strictly decreasing positive diagonal entries. Further, min 1 ≤ k ≤ K { σ k ( J N M ∗ J J ) − σ k + 1 ( J N M ∗ J J ) } = Ω ( N J ) \min_{1\leq k\leq K}\left\{\sigma_k(\mathbf{J}_N\mathbf{M}^{*}\mathbf{J}_J)-\sigma_{k+1}(\mathbf{J}_N\mathbf{M}^{*}\mathbf{J}_J)\right\}=\Omega(\sqrt{NJ}) min 1 ≤ k ≤ K { σ k ( J N M ∗ J J ) − σ k + 1 ( J N M ∗ J J ) } = Ω ( N J ) , and the first nonzero entry of each left singular vector u k ∗ \mathbf{u}^{*}_k u k ∗ of J N M ∗ J J \mathbf{J}_N\mathbf{M}^{*}\mathbf{J}_J J N M ∗ J J is positive.
2.2. There exists a diverging sequence q N J q_{NJ} q N J such that
lim inf ( N , J ) → ∞ q N J − 1 σ ( K + 1 ) ( K + 2 ) 2 ( ∂ M ( γ 0 ; Z ∗ ) ) > 0 , \liminf_{(N,J)\to\infty}q_{NJ}^{-1}\sigma_{\frac{(K+1)(K+2)}{2}}\left(\partial\mathbf{M}(\gamma_0;\mathbf{Z}^{*})\right)>0, ( N , J ) → ∞ lim inf q N J − 1 σ 2 ( K + 1 ) ( K + 2 ) ( ∂ M ( γ 0 ; Z ∗ ) ) > 0 ,
where ∂ M ( γ ; Z ) ∈ R N J × ( K ( K + 1 ) + 1 ) \partial\mathbf{M}(\gamma;\mathbf{Z})\in\mathbb{R}^{NJ\times(K(K+1)+1)} ∂ M ( γ ; Z ) ∈ R N J × ( K ( K + 1 ) + 1 ) is the Jacobian matrix of the mapping γ ↦ vec ( M ( γ ; Z ) ) \gamma\mapsto\operatorname{vec}(\mathbf{M}(\gamma;\mathbf{Z})) γ ↦ vec ( M ( γ ; Z )) .
2.3. There exist positive deterministic sequences ρ N J \rho_{NJ} ρ N J and c N J c_{NJ} c N J such that
lim inf ( N , J ) → ∞ c N J − 1 inf γ ∈ Γ : inf γ ~ ∈ [ γ 0 ] ∥ γ − γ ~ ∥ ≥ ρ N J ( N J ) − 1 l ( γ ; Z ∗ , M ∗ ) > 0. \liminf_{(N,J)\to\infty}c_{NJ}^{-1}\inf_{\substack{\gamma\in\Gamma:\ \inf_{\widetilde{\gamma}\in[\gamma_0]}\|\gamma-\widetilde{\gamma}\|\geq\rho_{NJ}}}(NJ)^{-1}l(\gamma;\mathbf{Z}^{*},\mathbf{M}^{*})>0. ( N , J ) → ∞ lim inf c N J − 1 γ ∈ Γ : i n f γ ∈ [ γ 0 ] ∥ γ − γ ∥ ≥ ρ N J inf ( N J ) − 1 l ( γ ; Z ∗ , M ∗ ) > 0.
Remark 12. As discussed in § 3.2 , Assumption 2.1 fixes the shift and rotational ambiguities of the configuration up to column-wise sign changes. In particular, F 0 T F 0 = Θ P 1 ∗ T Θ P 1 ∗ / N \mathbf{F}_0^{\mathsf{T}}\mathbf{F}_0=\boldsymbol{\Theta}^{*\mathsf{T}}_{\mathcal{P}_1}\boldsymbol{\Theta}^{*}_{\mathcal{P}_1}/N F 0 T F 0 = Θ P 1 ∗ T Θ P 1 ∗ / N is diagonal with asymptotically distinct positive diagonal entries, which motivates the diagonal constraint on F T F \mathbf{F}^{\mathsf{T}}\mathbf{F} F T F in Γ \Gamma Γ in (10). The singular-value gap condition ensures the stability of the SVD-based recovery, while the sign normalization on U ∗ \mathbf{U}^{*} U ∗ fixes the representation of the population singular vectors. The quantity q N J q_{NJ} q N J measures the local identification strength of the configuration. As shown in Theorem 4 below, N J / q N J \sqrt{NJ}/q_{NJ} N J / q N J acts as an amplification factor that determines how estimation error in the signal matrix propagates to the recovered configuration. Assumption 2.2 quantifies local identification through the Jacobian ∂ M ( γ 0 ; Z ∗ ) \partial\mathbf{M}(\gamma_0;\mathbf{Z}^{*}) ∂ M ( γ 0 ; Z ∗ ) : its first ( K + 1 ) ( K + 2 ) / 2 (K+1)(K+2)/2 ( K + 1 ) ( K + 2 ) /2 singular values are bounded below at scale q N J q_{NJ} q N J , while the remaining K ( K − 1 ) / 2 K(K-1)/2 K ( K − 1 ) /2 directions correspond to rotational invariance of the unconstrained parameterization. Assumption 2.3 complements this local condition by requiring separation, at scale c N J c_{NJ} c N J , outside a ρ N J \rho_{NJ} ρ N J -neighborhood of the sign-equivalence class [ γ 0 ] [\gamma_0] [ γ 0 ] . Together, these conditions characterize local identification and global separation of the true parameter modulo column-wise sign changes. Unlike the anchor-point conditions used in Chen et al. [13 ] , they formulate configuration identifiability directly through the local Jacobian and the population loss, without requiring particular anchor points.
Theorem 4. Suppose Assumptions 1.1 and 2 hold, and suppose further that ξ 2 = o ( c N J ) \xi^2=o(c_{NJ}) ξ 2 = o ( c N J ) and ρ N J = o ( q N J / N J ) \rho_{NJ}=o\left(q_{NJ}/\sqrt{NJ}\right) ρ N J = o ( q N J / N J ) . Then, for all sufficiently large N N N and J J J , the minimum in (10) is attained. Let γ ^ \widehat{\gamma} γ be any global minimizer of (10) . Then
d c ( Θ ( γ ^ ; Z ^ ) , Θ ∗ ) ≲ N J q N J ξ . d_c\left(\mathbf{\Theta}(\widehat{\gamma};\widehat{\mathbf{Z}}),\mathbf{\Theta}^{*}\right)\lesssim\frac{\sqrt{NJ}}{q_{NJ}}\xi. d c ( Θ ( γ ; Z ) , Θ ∗ ) ≲ q N J N J ξ .
Remark 13. The additional rate conditions in Theorem 4 have distinct roles in the perturbation analysis. The condition ξ 2 = o ( c N J ) \xi^2=o(c_{NJ}) ξ 2 = o ( c N J ) ensures that the squared perturbation of the signal matrix is asymptotically smaller than the global separation in Assumption 2.3, so that any global minimizer is localized within a ρ N J \rho_{NJ} ρ N J -neighborhood of [ γ 0 ] [\gamma_0] [ γ 0 ] . The condition ρ N J = o ( q N J / N J ) \rho_{NJ}=o(q_{NJ}/\sqrt{NJ}) ρ N J = o ( q N J / N J ) ensures that, within this neighborhood, the linear term of the mapping γ ↦ vec { M ( γ ; Z ∗ ) } \gamma\mapsto\operatorname{vec}\{\mathbf{M}(\gamma;\mathbf{Z}^{*})\} γ ↦ vec { M ( γ ; Z ∗ )} governed by the Jacobian ∂ M ( γ 0 ; Z ∗ ) \partial\mathbf{M}(\gamma_0;\mathbf{Z}^{*}) ∂ M ( γ 0 ; Z ∗ ) dominates the higher-order remainder. Given the conditions, the uniform error rate for configuration recovery is N J ξ / q N J \sqrt{NJ}\xi/q_{NJ} N J ξ / q N J , which is determined by both the estimation error ξ \xi ξ for the signal matrix and the local identification strength q N J q_{NJ} q N J of the true configuration. In the standard well-conditioned regime q N J ≲ N J q_{NJ}\lesssim\sqrt{NJ} q N J ≲ N J , the configuration recovery error has the same order as the signal estimation error.
Corollary 3. Suppose Assumption 1 holds, together with the additional conditions of Theorem 3 for unipartite data or Assumption 2 for bipartite data. Let M ^ \widehat{\mathbf{M}} M be the output of Algorithm 1, that is,
M ^ = { M ^ S V D , f ( x ) = x , M ^ R S D , f is nonlinear . \widehat{\mathbf{M}} =
\begin{cases}
\widehat{\mathbf{M}}^{\mathrm{SVD}}, & f(x)=x,\\
\widehat{\mathbf{M}}^{\mathrm{RSD}}, & f\text{ is nonlinear}.
\end{cases} M = { M SVD , M RSD , f ( x ) = x , f is nonlinear . where, in the nonlinear case, a N J a_{NJ} a N J and b N J b_{NJ} b N J are chosen as in Corollary 1. For nonlinear unipartite data, additionally suppose that f ( τ 0 ) ∈ [ a N N , b N N ] f(\tau_0) \in[a_{NN},b_{NN}] f ( τ 0 ) ∈ [ a N N , b N N ] for all sufficiently large N N N .
Define
δ N J = { log ( N J ) J − 1 / 2 , f ( x ) = x , ( log ( N J ) ) 2 J − 1 2 ( K + 2 ) ( log J ) K + 4 2 ( K + 2 ) , f is nonlinear . \delta_{NJ} =
\begin{cases}
\sqrt{\log(NJ)}J^{-1/2}, & f(x)=x,\\
(\log(NJ))^2J^{-\frac{1}{2(K+2)}}(\log J)^{\frac{K+4}{2(K+2)}}, & f\text{ is nonlinear}.
\end{cases} δ N J = { log ( N J ) J − 1/2 , ( log ( N J ) ) 2 J − 2 ( K + 2 ) 1 ( log J ) 2 ( K + 2 ) K + 4 , f ( x ) = x , f is nonlinear . For bipartite data, suppose further that δ N J 2 = o ( c N J ) \delta_{NJ}^2=o(c_{NJ}) δ N J 2 = o ( c N J ) and ρ N J = o ( q N J / N J ) \rho_{NJ}=o\left(q_{NJ}/\sqrt{NJ}\right) ρ N J = o ( q N J / N J ) . Let Θ ^ \widehat{\Theta} Θ denote the configuration estimator obtained by applying the corresponding configuration recovery procedure to $\widehat{\mathbf{M}}.
d c ( Θ ^ , Θ ∗ ) = { O p ( δ N J ) , for unipartite data , O p ( N J q N J δ N J ) , for bipartite data . d_c(\widehat{\Theta},\Theta^*) =
\begin{cases}
O_p(\delta_{NJ}), & \text{for unipartite data},\\
O_p\left(\dfrac{\sqrt{NJ}}{q_{NJ}}\delta_{NJ}\right), & \text{for bipartite data}.
\end{cases} d c ( Θ , Θ ∗ ) = ⎩ ⎨ ⎧ O p ( δ N J ) , O p ( q N J N J δ N J ) , for unipartite data , for bipartite data . If the response supports are uniformly bounded, the factor ( log J ) K + 4 2 ( K + 2 ) (\log J)^{\frac{K+4}{2(K+2)}} ( log J ) 2 ( K + 2 ) K + 4 in δ N J \delta_{NJ} δ N J can be omitted.
Remark 14. Corollary 3 follows by combining the identity-link max-norm bound in Theorem 1 and the nonlinear-link max-norm bound in Corollary 2 with the configuration recovery bounds in Theorems 3 and 4. For unipartite data, the configuration recovery step preserves the rate of the signal estimator returned by Algorithm 1. For bipartite data, the recovery rate additionally depends on the local identification strength through the factor N J / q N J \sqrt{NJ}/q_{NJ} N J / q N J ; when q N J ≍ N J q_{NJ}\asymp\sqrt{NJ} q N J ≍ N J , it has the same order as the corresponding signal estimator. For nonlinear responses with uniformly bounded response supports, the sharper rate follows by omitting the additional logarithmic factor in Corollary 2.
Simulation Study
In this section, we conduct simulation studies to examine the finite-sample performance of the proposed method.
Simulation Setup
We consider latent dimensions K ∈ { 3 , 5 } K \in\{3,5\} K ∈ { 3 , 5 } . For the bipartite setting, we set N = 2 J N = 2J N = 2 J with J ∈ { 300 , 500 , 1000 , 2000 } J \in\{300,500,1000,2000\} J ∈ { 300 , 500 , 1000 , 2000 } . For the unipartite setting, we set N = J N = J N = J and consider symmetric N × N N \times N N × N matrices with N ∈ { 300 , 500 , 1000 , 2000 } N \in\{300,500,1000,2000\} N ∈ { 300 , 500 , 1000 , 2000 } . Each setting is replicated 50 times.
For each case, Θ P 1 ∗ \boldsymbol{\Theta}^{*}_{\mathcal{P}_1} Θ P 1 ∗ is constructed from an N × K N \times K N × K matrix with i.i.d. Uniform( − 1 , 1 ) (-1,1) ( − 1 , 1 ) entries. We first center each column and orthogonalize the centered columns using a QR decomposition, and then rescale each column. Specifically, we rescale the columns so that
1 N ( Θ P 1 ∗ ) T Θ P 1 ∗ = diag ( λ ( K ) ) , \frac{1}{N}(\boldsymbol{\Theta}^{*}_{\mathcal{P}_1})^{\mathsf{T}}\boldsymbol{\Theta}^{*}_{\mathcal{P}_1} = \operatorname{diag}(\boldsymbol{\lambda}^{(K)}), N 1 ( Θ P 1 ∗ ) T Θ P 1 ∗ = diag ( λ ( K ) ) ,
where λ ( 3 ) = ( 0.20 , 0.18 , 0.16 ) T \boldsymbol{\lambda}^{(3)} = (0.20,0.18,0.16)^{\mathsf{T}} λ ( 3 ) = ( 0.20 , 0.18 , 0.16 ) T and λ ( 5 ) = ( 0.20 , 0.18 , 0.16 , 0.14 , 0.12 ) T \boldsymbol{\lambda}^{(5)} = (0.20,0.18,0.16,0.14,0.12)^{\mathsf{T}} λ ( 5 ) = ( 0.20 , 0.18 , 0.16 , 0.14 , 0.12 ) T . The decreasing entries of λ ( K ) \boldsymbol{\lambda}^{(K)} λ ( K ) give distinct, nondegenerate latent directions, which is consistent with the normalization and nondegeneracy conditions in Assumption 2.1. For the bipartite setting, Θ P 2 ∗ \boldsymbol{\Theta}^{*}_{\mathcal{P}_2} Θ P 2 ∗ is independently generated from a J × K J \times K J × K matrix with i.i.d. Uniform( − 1 , 1 ) (-1,1) ( − 1 , 1 ) entries and rescaled column-wise so that the mean squared entry of each column matches that of the corresponding column of Θ P 1 ∗ \boldsymbol{\Theta}^{*}_{\mathcal{P}_1} Θ P 1 ∗ . For the unipartite setting, P 1 = P 2 = [ N ] \mathcal{P}_1 = \mathcal{P}_2 = [N] P 1 = P 2 = [ N ] .
Recall that
m i j ∗ = { τ 0 − ∥ θ i ∗ − θ N + j ∗ ∥ 2 , for the bipartite setting , τ 0 − ∥ θ i ∗ − θ j ∗ ∥ 2 , for the unipartite setting . m_{ij}^{*} =
\begin{cases}
\tau_0 - \lVert\theta_i^{*} - \theta_{N+j}^{*} \rVert^2, & \text{for the bipartite setting}, \\
\tau_0 - \lVert\theta_i^{*} - \theta_j^{*} \rVert^2, & \text{for the unipartite setting}.
\end{cases} m ij ∗ = { τ 0 − ∥ θ i ∗ − θ N + j ∗ ∥ 2 , τ 0 − ∥ θ i ∗ − θ j ∗ ∥ 2 , for the bipartite setting , for the unipartite setting .
We use the same data-generating process for the latent configurations across the Gaussian, Binary, and Poisson observation models. We choose τ 0 \tau_0 τ 0 so that the average of the entries of M ∗ \mathbf{M}^{*} M ∗ equals 0 for the Gaussian and Binary models. For the Poisson model, we instead center the entries of M ∗ \mathbf{M}^{*} M ∗ to 0.5, yielding larger expected counts and thereby reducing the prevalence of zero observations. Thus, all three observation models share the same latent geometry, while the Poisson natural-parameter matrix includes an additional intercept shift of 0.5. We then generate observations according to the corresponding observation model. For unipartite data, observations are generated for i < j i < j i < j , and we set y j i = y i j y_{ji}=y_{ij} y j i = y ij and y i i = 0 y_{ii}=0 y ii = 0 .
Homoskedastic Gaussian model. Let y i j ∼ N ( m i j ∗ , 1 ) y_{ij} \sim N(m_{ij}^{*},1) y ij ∼ N ( m ij ∗ , 1 ) .
Heteroskedastic Gaussian model. Let y i j ∼ N ( m i j ∗ , σ i j 2 ) y_{ij} \sim N(m_{ij}^{*},\sigma_{ij}^{2}) y ij ∼ N ( m ij ∗ , σ ij 2 ) , where σ i j 2 ∼ i . i . d . Uniform ( 0.4 , 1.6 ) \sigma_{ij}^{2} \overset{\mathrm{i.i.d.}}{\sim} \operatorname{Uniform}(0.4,1.6) σ ij 2 ∼ i.i.d. Uniform ( 0.4 , 1.6 ) .
Binary model. Let y i j ∼ Bernoulli ( p i j ) y_{ij} \sim\operatorname{Bernoulli}(p_{ij}) y ij ∼ Bernoulli ( p ij ) , where p i j = 1 / { 1 + exp ( − m i j ∗ ) } p_{ij}=1/\{1+\exp(-m_{ij}^{*})\} p ij = 1/ { 1 + exp ( − m ij ∗ )} .
Poisson model. Let y i j ∼ Poisson ( λ i j ) y_{ij} \sim\operatorname{Poisson}(\lambda_{ij}) y ij ∼ Poisson ( λ ij ) , where λ i j = exp ( m i j ∗ ) \lambda_{ij}=\exp(m_{ij}^{*}) λ ij = exp ( m ij ∗ ) .
For the nonlinear-link models, we set ( a N J , b N J ) = ( 0 , 1 ) (a_{NJ},b_{NJ})=(0,1) ( a N J , b N J ) = ( 0 , 1 ) for the binary model and ( a N J , b N J ) = ( 0 , 3 log J ) (a_{NJ},b_{NJ})=(0,3\log J) ( a N J , b N J ) = ( 0 , 3 log J ) for the Poisson model. We use the projection constant C = 7 C=7 C = 7 in the spectral-denoising stage for all observation models and also in the one-step-refinement stage for the binary and Poisson models.
Results
To assess the effect of refinement, we compare M ^ S V D \widehat{\mathbf{M}}^{\mathrm{SVD}} M SVD and M ^ R S D \widehat{\mathbf{M}}^{\mathrm{RSD}} M RSD under the max-norm loss. For the Gaussian models, M ^ R S D \widehat{\mathbf{M}}^{\mathrm{RSD}} M RSD is computed only as a diagnostic comparison, since Algorithm 1 returns M ^ S V D \widehat{\mathbf{M}}^{\mathrm{SVD}} M SVD under the identity link. Configuration recovery is always based on the final output of Algorithm 1, namely M ^ S V D \widehat{\mathbf{M}}^{\mathrm{SVD}} M SVD for the Gaussian models and M ^ R S D \widehat{\mathbf{M}}^{\mathrm{RSD}} M RSD for the binary and Poisson models. For r ∈ { S V D , R S D } r \in\{\mathrm{SVD},\mathrm{RSD}\} r ∈ { SVD , RSD } , we denote E max ( r ) = ∥ M ^ r − M ∗ ∥ max \mathcal{E}_{\max}^{(r)} = \lVert\widehat{\mathbf{M}}^{r} - \mathbf{M}^{*} \rVert_{\max} E m a x ( r ) = ∥ M r − M ∗ ∥ m a x . For configuration recovery, we report d c ( Θ ^ , Θ ∗ ) d_{c}(\widehat{\boldsymbol{\Theta}}, \boldsymbol{\Theta}^{*}) d c ( Θ , Θ ∗ ) . We also report ∣ τ ^ − τ 0 ∣ \lvert\widehat{\tau} - \tau_{0} \rvert ∣ τ − τ 0 ∣ under bipartite settings. Results for K = 3 K = 3 K = 3 are presented below, while those for K = 5 K = 5 K = 5 are provided in Appendix § 8 .
Figure 2 –Figure 3 show the results for the unipartite setting, while Figure 4 –Figure 5 show the corresponding results for the bipartite setting, together with the drift parameter error. Across both observation structures, the one-step refinement substantially reduces the entrywise maximum error for the binary and Poisson models, and the refined max-norm error decreases steadily as the matrix size increases. In contrast, the max-norm error of the spectral estimator remains comparatively large for these nonlinear models. This pattern is consistent with the role of refinement in our theory: the spectral stage provides a Frobenius-consistent initializer, while the refinement yields uniform entrywise control. For the Gaussian models, Algorithm 1 returns M ^ S V D \widehat{\mathbf{M}}^{\mathrm{SVD}} M SVD , which is already max-norm consistent by Theorem 1. In the bipartite setting, the additional refinement has little empirical effect. In the unipartite setting, a small finite-sample deterioration is observed after refinement, possibly because the spectral estimator directly exploits the symmetric one-mode structure, whereas refinement temporarily estimates the row and column factors separately before reimposing symmetry at the final step. Across all four models, the configuration recovery error decreases with the matrix size; the bipartite drift estimation error shows the same overall trend.
Figure 2. Max-norm errors of M ^ S V D \widehat{\mathbf{M}}^{\mathrm{SVD}} M SVD and M ^ R S D \widehat{\mathbf{M}}^{\mathrm{RSD}} M RSD for the unipartite model with K = 3 K = 3 K = 3 .
Figure 3. Configuration recovery error d c ( Θ ^ , Θ ∗ ) d_c(\widehat{\boldsymbol{\Theta}},\boldsymbol{\Theta}^{*}) d c ( Θ , Θ ∗ ) for the unipartite model with K = 3 K=3 K = 3 .
Figure 4. Max-norm errors of M ^ S V D \widehat{\mathbf{M}}^{\mathrm{SVD}} M SVD and M ^ R S D \widehat{\mathbf{M}}^{\mathrm{RSD}} M RSD for the bipartite model with K = 3 K=3 K = 3 .
Figure 5. ∣ τ ^ − τ 0 ∣ |\widehat{\tau}-\tau_0| ∣ τ − τ 0 ∣ and d c ( Θ ^ , Θ ∗ ) d_c(\widehat{\boldsymbol{\Theta}},\boldsymbol{\Theta}^{*}) d c ( Θ , Θ ∗ ) for the bipartite model with K = 3 K=3 K = 3 .
Table 1 reports the computational time of the proposed estimation procedure.1 For each setting, we record the wall-clock time of the complete estimation pipeline, including spectral denoising and configuration recovery for the Gaussian models, and spectral denoising, one-step refinement, and configuration recovery for the binary and Poisson models. Each experiment is repeated 50 times, and the mean runtime is reported. As shown in Table 1 , the runtime increases with the matrix size, as expected, while the difference between K = 3 K = 3 K = 3 and K = 5 K = 5 K = 5 is generally moderate. The Gaussian models are computationally less demanding than the binary and Poisson models, for which the refinement step accounts for most of the computation. Even for the largest setting considered, with J = 2000 J = 2000 J = 2000 , the mean runtime remains below 20 seconds across all cases. These results demonstrate that the proposed procedure scales efficiently to large relational matrices, completing the full estimation pipeline within seconds even at the largest simulated sizes.
Table 1. Mean runtime (seconds), with standard errors in parentheses, over 50 replications.
Comparison with Likelihood-Based Estimation
We further compare the proposed procedure with likelihood-based alternatives in the bipartite binary setting, as considered in § 5.1 . The bipartite binary setting is most directly connected to existing likelihood-based probabilistic unfolding methods, including the approach of Chen et al. [13 ] , and therefore provides a natural benchmark for our procedure.
We compare RSD + CR with two likelihood-based alternatives designed to separate the effects of likelihood-based signal estimation and direct enforcement of the Euclidean distance geometry. In particular, we compare three methods: (i) the proposed RSD + CR procedure, consisting of spectral denoising, one-step refinement, and configuration recovery; (ii) a two-stage likelihood-based procedure (low-rank MLE + CR), which first estimates the low-rank signal matrix by maximum likelihood and then applies the same configuration recovery procedure as RSD + CR; (iii) the joint maximum likelihood estimator (joint MLE), which directly estimates the latent configurations and the intercept by maximizing the Bernoulli likelihood using alternating optimization. The joint maximum likelihood estimator is similar to the estimator in Chen et al. [13 ] , except that we have an additional drift parameter τ \tau τ here.
Table 2 shows a clear statistical–computational tradeoff among the three procedures. For configuration recovery, joint MLE achieves the smallest errors throughout, while low-rank MLE + CR generally improves modestly over RSD + CR and lies between the other two methods. The advantage of joint MLE is expected under the correctly specified model, since it imposes the K K K -dimensional Euclidean distance structure throughout likelihood optimization, whereas low-rank MLE + CR first estimates a more general rank-( K + 2 ) (K + 2) ( K + 2 ) signal matrix and only enforces the distance geometry in the subsequent recovery step; the two additional signal dimensions introduce extra nuisance flexibility and can lead to some loss of statistical efficiency. For the drift parameter, joint MLE is also consistently the most accurate, while low-rank MLE + CR provides only a modest improvement over RSD + CR.
Table 2. Comparison of RSD + CR, low-rank MLE + CR, and joint MLE in the nonlinear bipartite binary setting with K = 3 K = 3 K = 3 and N = 2 J N = 2J N = 2 J . Each entry reports the mean, with standard errors in parentheses, over 50 replications.
In contrast, RSD + CR is substantially faster than both likelihood-based alternatives. Across the four sample sizes considered, it is approximately 11–28 times faster than low-rank MLE
CR and 13–34 times faster than joint MLE. This substantial computational advantage arises because RSD + CR relies on spectral denoising, one-step low-dimensional row- and column-wise refinements, and a final configuration-recovery optimization whose dimension depends only on K K K , whereas both likelihood-based alternatives require repeated iterative optimization.
Real Data Analysis
For the real-data analyses, we select the latent dimension by inspecting the leading spectral values of the double-centered estimated signal matrix and choose K K K at the dominant spectral elbow. In both applications, the leading spectral values indicate that a two-dimensional representation captures the main geometric structure, and we therefore set K = 2 K = 2 K = 2 for visualization.
Analysis of a Statistician Citation Network
We apply the proposed method to a real-world citation network of statisticians. The data set, collected by Ji et al. (2021), contains publication records from 36 statistical journals between 1975 and 2015. To obtain a clear and interpretable visualization, we restrict the analysis to the top 0.3% most active authors, resulting in a set of 142 statisticians. Based on their citation relationships, we construct an undirected intellectual network with adjacency matrix Y = ( y i j ) 142 × 142 \mathbf{Y} = (y_{ij})_{142 \times142} Y = ( y ij ) 142 × 142 . Specifically, the edge weight between statisticians i i i and j j j is defined as the symmetrized citation frequency after a log transformation to reduce the influence of large citation counts, y i j = log ( 1 + ( C i j + C j i ) / 2 ) y_{ij} = \log(1 + (C_{ij} + C_{ji})/2) y ij = log ( 1 + ( C ij + C j i ) /2 ) , where C i j C_{ij} C ij denotes the number of citations from statistician i i i to statistician j j j . This symmetrization procedure converts the directed citation relationship into an undirected measure of intellectual connection strength. We apply the proposed unipartite method to Y \mathbf{Y} Y under a Gaussian observation model with the identity link.
Figure 6 presents the estimated two-dimensional latent configuration of the 142 statisticians. By applying K-means clustering with five clusters to these estimated embedding vectors, we partition the network into five distinct research communities. In the visualization, we manually select representative statisticians from the five K-means clusters and highlight them in dark red. The blue dashed contours, generated based on the K-means partition, delineate the spatial boundaries of these communities. A legend listing the five research groups is provided in the top-right corner of the plot. The resulting visualization reveals clear separation among research communities in the statistician citation network.
Figure 6. Two-dimensional visualization of the estimated embedding vectors for 142 statisticians.
To interpret the five K-means clusters, we examine the main research areas of the statisticians in each cluster. The clusters can be broadly characterized as follows:
Statistical Learning and High-Dimensional Inference: this cluster includes researchers working on nonparametric methods, regularization, resampling, and high-
dimensional statistical learning. Representative figures include Jianqing Fan, Peter Hall, Robert Tibshirani, Trevor Hastie, Bradley Efron, and Peter Bickel.
Epidemiology, Survival Analysis, and Causal Inference: this cluster centers on statisticians known for their foundational contributions to epidemiological studies and survival analysis. Notable statisticians in this group include James Robins, Ross Prentice, Lee-jen Wei, Norman Breslow, and Scott Zeger.
Missing Data, EM Algorithm, and Potential Outcomes: this cluster comprises researchers renowned for their work on causal inference under the potential outcomes framework and missing-data methodologies. Key members include Donald B. Rubin, Nan Laird, Roderick Little, Thomas A. Louis, Joseph Ibrahim, and Arthur P. Dempster.
Biostatistics and Semiparametric/Nonparametric Regression: this cluster brings together statisticians who have made significant advances in biostatistics, measurement error models, and semiparametric regression. Representative authors include Xihong Lin, Raymond Carroll, Xuming He, Bruce Lindsay, and David Ruppert.
Computational Bayes, MCMC, and Graphical Models: this cluster consists of leading contributors to Markov chain Monte Carlo methods, computational Bayesian inference, and graphical models. Prominent names include Alan Gelfand, Adrian Smith, Christian Robert, David Dunson, Andrew Gelman, George Casella, and Peter Green.
The K-means clustering identifies several interpretable research communities without utilizing prior research-field labels in either model estimation or clustering. Moreover, the estimated embedding vectors provide geometric insights beyond discrete groupings. Statisticians whose work spans several research areas tend to appear near the cluster boundaries. For instance, Bradley Efron is positioned near the intersection of the statistical learning and biostatistics groups, which reflects his broad expertise spanning bootstrap resampling, empirical Bayes methods, and classical statistical inference. Thus, the recovered latent space effectively captures both the cohesive community structure of the citation network and the gradual, continuous relationships bridging different subfields of statistics.
Analysis of the European Parliament Voting Data
We further evaluate the proposed method using roll-call voting data from the European Parliament. The data set is obtained from the HowTheyVote.eu repository,2 which provides voting records for Members of the European Parliament (MEPs). Our analysis focuses on the 9th parliamentary term, covering the period from July 2, 2019, to July 15, 2024.
To formalize the analysis, we construct an observation matrix in which each row corresponds to a specific MEP and each column corresponds to a legislative procedure. The observed voting outcomes fall into three categories: votes in favor, votes against, and abstentions. We code votes in favor, votes against, and abstentions as 1, −1, and 0, respectively. Our analysis includes all 867 MEPs recorded in the data set who cast at least one vote during this term, and retains 1650 distinct legislative procedures that contain at least one valid vote. Missing entries, including non-participation records, are imputed with 0, treating them as a neutral midpoint on the coded voting scale and yielding a complete data matrix of size 867 × 1650 867 \times1650 867 × 1650 . Let x i j x_{ij} x ij denote the voting outcome for the i i i -th MEP on the j j j -th legislative procedure. Let y i j = x i j + 1 ∈ { 0 , 1 , 2 } y_{ij} = x_{ij} + 1 \in\{0, 1, 2\} y ij = x ij + 1 ∈ { 0 , 1 , 2 } , and one possible specification is that y i j ∼ Binomial ( 2 , σ ( τ − ∥ θ i − a j ∥ 2 ) ) y_{ij} \sim\operatorname{Binomial}(2, \sigma(\tau- \lVert\theta_i - a_j \rVert^2)) y ij ∼ Binomial ( 2 , σ ( τ − ∥ θ i − a j ∥ 2 )) with σ ( ⋅ ) \sigma(\cdot) σ ( ⋅ ) being the sigmoid function. The induced link is f ( m ) = 2 / ( 1 + exp ( − m ) ) f(m) = 2/(1 + \exp(-m)) f ( m ) = 2/ ( 1 + exp ( − m )) .
As shown in Figure 7 (left), the recovered latent configurations exhibit a strong alignment with the official political group affiliations of the MEPs, which is highly consistent with the well-known two-dimensional model of the European Parliament proposed in political science literature [23 , 20 ] . Moreover, the two resolved dimensions can be broadly interpreted in terms of attitudes toward European integration and traditional socioeconomic ideologies. The first latent dimension appears to capture a cleavage related to attitudes toward European integration. The signs of the displayed axes are chosen for interpretability and have no effect on the underlying Euclidean distances. As shown in Figure 7 (left) and summarized in Table 3 , EPP, RENEW, and SD are located on the positive side of this dimension, whereas ID, ECR, and GUE_NGL are located on the negative side. This ordering is broadly consistent with the distinction between groups that are generally more supportive of European integration and groups that exhibit more critical attitudes toward supranational integration. The second latent dimension appears to capture the conventional left–right ideological cleavage in the European Parliament. As shown in Figure 7 (left) and summarized in Table 3 , GREEN_EFA and GUE_NGL are located on the negative side of this dimension, followed by SD and RENEW near the center, whereas EPP, ECR, and ID occupy more positive positions. Importantly, this dimension is distinct from the European integration dimension: for example, ECR and GUE_NGL are located at similar positions along the first dimension but differ substantially along the second dimension.
Figure 7. Two views of the recovered European Parliament latent configuration. Left: estimated embedding vectors of 867 MEPs, colored by official political group affiliation. Right: joint representation of MEPs and legislative procedures in the recovered bipartite configuration.
Table 3. Coordinate statistics of political groups on the first and second latent dimensions.
A key advantage of the bipartite framework is that MEPs and legislative procedures are represented jointly in the same Euclidean space. Figure 7 (right) illustrates this joint representation, allowing legislative procedures to be interpreted relative to the locations of political groups in the recovered configuration. For illustration, for each political group we identify the two distinct roll-call issues closest to the centroid of its MEP embedding vectors. For example, the procedures nearest to the SD center concern humanitarian assistance in Gaza and the ILO Violence and Harassment Convention, while those near the GREEN_EFA center involve the role of tax policy during crises and the establishment of the EU ethics body. The ECR center lies close to procedures concerning migration-related flexibility instruments and additional technical criteria for climate-related economic activities. The complete set of group–procedure associations is reported in Table 4 in Appendix § 8 .
Table 4. Top-2 closest legislative procedures to each political group center.
Discussion
This paper develops a computationally efficient framework for recovering Euclidean configurations from unipartite and bipartite relational data under a general mean formulation. The proposed approach separates low-rank signal estimation from geometric recovery, and in the bipartite setting reduces the remaining ambiguity to a low-dimensional alignment problem. The recovery step is modular and can be combined with alternative sufficiently accurate signal estimators. The theory establishes uniform configuration recovery for both settings and formulates bipartite identifiability through local identification strength and global separation, without requiring explicit anchor-point conditions.
Several directions merit further study. A natural extension is to multipartite relational data with structured blockwise missingness, where global identifiability may depend jointly on the observed block pattern and the latent geometry. Such extensions would broaden the applicability of the framework to complex relational systems involving multiple types of entities and interactions. Another direction is to develop statistically efficient first-stage estimators for nonlinear-link models. Replacing the spectral estimator with likelihood-based or other efficient estimators may improve the signal estimation accuracy while retaining the modular configuration recovery procedure developed here. This combination could provide a foundation for establishing asymptotic distributions and uncertainty quantification for recovered embeddings, pairwise distances, and other geometrically interpretable quantities. A further direction is to extend the framework to dynamic relational systems, where latent configurations evolve over time. Incorporating temporal dependence and developing scalable procedures for tracking changes in latent geometry would enable applications to longitudinal network and interaction data.
Additional Simulation and Real-Data Results
This appendix reports additional simulation and real-data results that complement the numerical analyses in the main text.
Additional Simulation Results for K = 5 K = 5 K = 5
This section reports the simulation results for latent dimension K = 5 K = 5 K = 5 . The simulation settings and evaluation criteria are the same as those in § 5.2 of the main text. Overall, the results are consistent with those for K = 3 K = 3 K = 3 and exhibit similar patterns as the matrix size increases.
Figure 8 –Figure 9 show the results for the unipartite setting, while Figure 10 –Figure 11 show the corresponding results for the bipartite setting, together with the drift parameter error.
Figure 8. Max-norm errors of M ^ S V D \widehat{\mathbf{M}}^{\mathrm{SVD}} M SVD and M ^ R S D \widehat{\mathbf{M}}^{\mathrm{RSD}} M RSD for the unipartite model with K = 5 K = 5 K = 5 .
Figure 9. Configuration recovery error d c ( Θ ^ , Θ ∗ ) d_c(\widehat{\Theta}, \Theta^*) d c ( Θ , Θ ∗ ) for the unipartite model with K = 5 K = 5 K = 5 .
Figure 10. Max-norm errors of M ^ S V D \widehat{\mathbf{M}}^{\mathrm{SVD}} M SVD and M ^ R S D \widehat{\mathbf{M}}^{\mathrm{RSD}} M RSD for the bipartite model with K = 5 K = 5 K = 5 .
Figure 11. ∣ τ ^ − τ 0 ∣ |\widehat{\tau}-\tau_0| ∣ τ − τ 0 ∣ and d c ( Θ ^ , Θ ∗ ) d_c(\widehat{\Theta},\Theta^*) d c ( Θ , Θ ∗ ) for the bipartite model with K = 5 K=5 K = 5 .
The results for K = 5 K = 5 K = 5 broadly agree with those for K = 3 K = 3 K = 3 in the main text. Across the observation models, the signal-estimation and configuration recovery errors generally decrease as the matrix size increases. The refinement is particularly effective in reducing the entrywise maximum error for the nonlinear observation models. For the bipartite setting, the drift parameter error also generally decreases with the matrix size. These results exhibit the same overall finite-sample patterns as those reported in the main text.
Additional Results for the European Parliament Voting Data
Table 4 reports the two legislative procedures closest to the centroid of each political group in the estimated joint configuration. These results complement Figure 7 (right) in the main text by providing the complete list of group–procedure proximities used in the visualization. As discussed in the main text, these associations are descriptive and should be interpreted in terms of proximity in the recovered voting geometry rather than as direct measures of issue importance or uniform political support.
Proofs for Spectral Denoising
The proof of Theorem 1 is organized by data structure and link function. We first consider bipartite data and then give the corresponding arguments for unipartite data, where symmetry and the unobserved diagonal require additional treatment. For the identity link, we establish both the normalized Frobenius and max-norm bounds, whereas for nonlinear links we establish the normalized Frobenius bound used in the subsequent refinement analysis.
Throughout Appendix B, let C 0 > 0 C_0>0 C 0 > 0 denote a fixed constant such that Assumption 1.1 holds, in particular, ∥ M ∗ ∥ max ≤ C 0 \lVert M^*\rVert_{\max}\le C_0 ∥ M ∗ ∥ m a x ≤ C 0 . To distinguish this population bound from the projection constant used in Algorithm 1, we denote the latter by C p r o j C_{\mathrm{proj}} C proj throughout the proof and take C p r o j ≥ C 0 C_{\mathrm{proj}}\ge C_0 C proj ≥ C 0 .
Bipartite Spectral Denoising
We treat the identity and nonlinear link functions separately.
Identity Link
In the identity-link case, f ( x ) = x f(x) = x f ( x ) = x , hence Y = M ∗ + E \mathbf{Y} = \mathbf{M}^{*} + \mathcal{E} Y = M ∗ + E , where E = ( y i j − m i j ∗ ) N × J \mathcal{E} = (y_{ij} - m^{*}_{ij})_{N \times J} E = ( y ij − m ij ∗ ) N × J is the noise matrix. Algorithm 1 computes X ^ = SVD K + 2 ( Y ) \widehat{\mathbf{X}} = \operatorname{SVD}_{K+2}(\mathbf{Y}) X = SVD K + 2 ( Y ) and outputs M ^ S V D = ( m ^ i j S V D ) N × J \widehat{\mathbf{M}}^{\mathrm{SVD}} = (\widehat{m}^{\mathrm{SVD}}_{ij})_{N \times J} M SVD = ( m ij SVD ) N × J with
m ^ i j S V D = P [ − C p r o j , C p r o j ] ( x ^ i j ) . \widehat{m}^{\mathrm{SVD}}_{ij} = \mathcal{P}_{[-C_{\mathrm{proj}},C_{\mathrm{proj}}]}(\widehat{x}_{ij}). m ij SVD = P [ − C proj , C proj ] ( x ij ) .
We first establish the normalized Frobenius bound. By the Eckart–Young–Mirsky theorem, X ^ = argmin rank ( X ) ≤ K + 2 ∥ Y − X ∥ F 2 \widehat{\mathbf{X}} = \operatorname*{argmin}_{\operatorname{rank}(\mathbf{X}) \le K+2} \lVert\mathbf{Y} - \mathbf{X} \rVert_{F}^{2} X = argmin rank ( X ) ≤ K + 2 ∥ Y − X ∥ F 2 . Since rank ( M ∗ ) ≤ K + 2 \operatorname{rank}(\mathbf{M}^{*}) \le K+2 rank ( M ∗ ) ≤ K + 2 , ∥ Y − X ^ ∥ F 2 ≤ ∥ Y − M ∗ ∥ F 2 \lVert\mathbf{Y} - \widehat{\mathbf{X}} \rVert_{F}^{2} \le\lVert\mathbf{Y} - \mathbf{M}^{*} \rVert_{F}^{2} ∥ Y − X ∥ F 2 ≤ ∥ Y − M ∗ ∥ F 2 . Let Δ = X ^ − M ∗ \Delta= \widehat{X} - M^* Δ = X − M ∗ . Since Y − X ^ = E − Δ Y - \widehat{X} = \mathcal{E} - \Delta Y − X = E − Δ ,
∥ Δ ∥ F 2 ≤ 2 ⟨ E , Δ ⟩ ≤ 2 ∥ E ∥ 2 ∥ Δ ∥ ∗ ≤ 2 2 ( K + 2 ) ∥ E ∥ 2 ∥ Δ ∥ F , \lVert\Delta\rVert_F^2 \le2\langle\mathcal{E}, \Delta\rangle\le2\lVert\mathcal{E}\rVert_2\lVert\Delta\rVert_* \le2\sqrt{2(K+2)}\lVert\mathcal{E}\rVert_2\lVert\Delta\rVert_F, ∥ Δ ∥ F 2 ≤ 2 ⟨ E , Δ ⟩ ≤ 2 ∥ E ∥ 2 ∥ Δ ∥ ∗ ≤ 2 2 ( K + 2 ) ∥ E ∥ 2 ∥ Δ ∥ F ,
where rank ( Δ ) ≤ 2 ( K + 2 ) \operatorname{rank}(\Delta) \le2(K+2) rank ( Δ ) ≤ 2 ( K + 2 ) . Consequently,
∥ X ^ − M ∗ ∥ F ≤ 2 2 ( K + 2 ) ∥ E ∥ 2 . (11) \lVert\widehat{X} - M^*\rVert_F \le2\sqrt{2(K+2)}\lVert\mathcal{E}\rVert_2. \tag*{(11)} ∥ X − M ∗ ∥ F ≤ 2 2 ( K + 2 ) ∥ E ∥ 2 . (11)
Since m i j ∗ ∈ [ − C 0 , C 0 ] ⊆ [ − C p r o j , C p r o j ] m^*_{ij} \in[-C_0,C_0] \subseteq[-C_{\mathrm{proj}},C_{\mathrm{proj}}] m ij ∗ ∈ [ − C 0 , C 0 ] ⊆ [ − C proj , C proj ] , entrywise projection cannot increase the error relative to M ∗ M^* M ∗ . Thus,
∥ M ^ S V D − M ∗ ∥ F ≤ ∥ X ^ − M ∗ ∥ F , \lVert\widehat{M}^{\mathrm{SVD}} - M^*\rVert_F \le\lVert\widehat{X} - M^*\rVert_F, ∥ M SVD − M ∗ ∥ F ≤ ∥ X − M ∗ ∥ F ,
and hence
1 N J ∥ M ^ S V D − M ∗ ∥ F 2 ≤ 8 ( K + 2 ) N J ∥ E ∥ 2 2 . (12) \frac{1}{NJ}\lVert\widehat{M}^{\mathrm{SVD}} - M^*\rVert_F^2 \le\frac{8(K+2)}{NJ}\lVert\mathcal{E}\rVert_2^2. \tag*{(12)} N J 1 ∥ M SVD − M ∗ ∥ F 2 ≤ N J 8 ( K + 2 ) ∥ E ∥ 2 2 . (12)
Conditional on the true latent configuration, the entries of E \mathcal{E} E are independent, mean zero, and uniformly sub-exponential under Assumption 1.2. Therefore, by the rectangular analogue of Corollary 3.5 of Bandeira and van Handel [4 ] , obtained from the rectangular argument of their Theorem 3.1,
E ∥ E ∥ 2 ≲ N + J + log J ≲ N , \mathbb{E}\lVert\mathcal{E}\rVert_2 \lesssim\sqrt{N}+\sqrt{J}+\log J \lesssim\sqrt{N}, E ∥ E ∥ 2 ≲ N + J + log J ≲ N ,
where J ≤ N J \le N J ≤ N . Hence ∥ E ∥ 2 = O p ( N ) \lVert\mathcal{E}\rVert_2 = O_p(\sqrt{N}) ∥ E ∥ 2 = O p ( N ) , and
1 N J ∥ M ^ S V D − M ∗ ∥ F 2 = O p ( J − 1 ) . \frac{1}{NJ}\lVert\widehat{M}^{\mathrm{SVD}} - M^*\rVert_F^2 = O_p(J^{-1}). N J 1 ∥ M SVD − M ∗ ∥ F 2 = O p ( J − 1 ) .
We next establish the max-norm bound under Assumptions 1.4 and 1.5. Put r = K + 2 r = K + 2 r = K + 2 and L = log ( N J ) L = \log(NJ) L = log ( N J ) , and write
M ∗ = U D V T , X ^ = SVDr r ( Y ) , \mathbf{M}^{*} = \mathbf{U}\mathbf{D}\mathbf{V}^{\mathsf{T}}, \qquad\widehat{\mathbf{X}} = \operatorname{SVDr}_{r}(\mathbf{Y}), M ∗ = UD V T , X = SVDr r ( Y ) ,
where U ∈ R N × r \mathbf{U} \in\mathbb{R}^{N \times r} U ∈ R N × r and V ∈ R J × r \mathbf{V} \in\mathbb{R}^{J \times r} V ∈ R J × r have orthonormal columns. The rank bound together with Assumption 1.5 implies rank ( M ∗ ) = r \operatorname{rank}(\mathbf{M}^{*}) = r rank ( M ∗ ) = r , and
σ r ( M ∗ ) ≳ N J , ∥ M ∗ ∥ ≲ N J , ∥ U ∥ 2 → ∞ ≲ N − 1 / 2 , ∥ V ∥ 2 → ∞ ≲ J − 1 / 2 , ∥ M ∗ ∥ 2 → ∞ ≲ J , ∥ M ∗ T ∥ 2 → ∞ ≲ N . (13) \begin{aligned}
\sigma_r(\mathbf{M}^{*}) &\gtrsim\sqrt{NJ}, & \|\mathbf{M}^{*}\| &\lesssim\sqrt{NJ}, \\
\|\mathbf{U}\|_{2\to\infty} &\lesssim N^{-1/2}, & \|\mathbf{V}\|_{2\to\infty} &\lesssim J^{-1/2}, \\
\|\mathbf{M}^{*}\|_{2\to\infty} &\lesssim\sqrt{J}, & \|{\mathbf{M}^{*}}^{\mathsf{T}}\|_{2\to\infty} &\lesssim\sqrt{N}. \tag*{(13)}
\end{aligned} σ r ( M ∗ ) ∥ U ∥ 2 → ∞ ∥ M ∗ ∥ 2 → ∞ ≳ N J , ≲ N − 1/2 , ≲ J , ∥ M ∗ ∥ ∥ V ∥ 2 → ∞ ∥ M ∗ T ∥ 2 → ∞ ≲ N J , ≲ J − 1/2 , ≲ N . (13)
Again, entrywise projection cannot increase the error, so it suffices to control ∥ X ^ − M ∗ ∥ max \|\widehat{\mathbf{X}}-\mathbf{M}^{*}\|_{\max} ∥ X − M ∗ ∥ m a x .
Let E = Y − M ∗ \mathbf{E} = \mathbf{Y} - \mathbf{M}^{*} E = Y − M ∗ , let V Y \mathbf{V}_{Y} V Y contain the leading r r r right singular vectors of Y \mathbf{Y} Y , and define
P = V V T , P ^ = V Y V Y T , H = Y T Y − M ∗ T M ∗ . \mathbf{P} = \mathbf{V}\mathbf{V}^{\mathsf{T}}, \qquad\widehat{\mathbf{P}} = \mathbf{V}_{Y}\mathbf{V}_{Y}^{\mathsf{T}}, \qquad\mathbf{H} = \mathbf{Y}^{\mathsf{T}}\mathbf{Y} - {\mathbf{M}^{*}}^{\mathsf{T}}\mathbf{M}^{*}. P = V V T , P = V Y V Y T , H = Y T Y − M ∗ T M ∗ .
Writing Y T Y V Y = V Y Λ Y \mathbf{Y}^{\mathsf{T}}\mathbf{Y}\mathbf{V}_{Y} = \mathbf{V}_{Y}\mathbf{\Lambda}_{Y} Y T Y V Y = V Y Λ Y and λ Y = λ min ( Λ Y ) = σ r ( Y ) 2 \lambda_{Y} = \lambda_{\min}(\mathbf{\Lambda}_{Y}) = \sigma_r(\mathbf{Y})^2 λ Y = λ m i n ( Λ Y ) = σ r ( Y ) 2 , we have, whenever λ Y > 0 \lambda_{Y} > 0 λ Y > 0 ,
( I J − P ) V Y = ( I J − P ) H V Y Λ Y − 1 . (14) (\mathbf{I}_{J} - \mathbf{P})\mathbf{V}_{Y} = (\mathbf{I}_{J} - \mathbf{P})\mathbf{H}\mathbf{V}_{Y}\mathbf{\Lambda}_{Y}^{-1}. \tag*{(14)} ( I J − P ) V Y = ( I J − P ) H V Y Λ Y − 1 . (14)
Consequently,
∥ ( I J − P ) V Y ∥ ≤ ∥ H ∥ λ Y . \|(\mathbf{I}_{J} - \mathbf{P})\mathbf{V}_{Y}\| \le\frac{\|\mathbf{H}\|}{\lambda_{Y}}. ∥ ( I J − P ) V Y ∥ ≤ λ Y ∥ H ∥ .
Moreover, for each j ∈ [ J ] j \in[J] j ∈ [ J ] ,
∥ e j T ( I J − P ) V Y ∥ ≤ λ Y − 1 { ∥ e j T H ∥ + ∥ e j T V ∥ ∥ H ∥ } . \|\mathbf{e}_{j}^{\mathsf{T}}(\mathbf{I}_{J} - \mathbf{P})\mathbf{V}_{Y}\| \le\lambda_{Y}^{-1}\left\{\|\mathbf{e}_{j}^{\mathsf{T}}\mathbf{H}\| + \|\mathbf{e}_{j}^{\mathsf{T}}\mathbf{V}\|\|\mathbf{H}\|\right\}. ∥ e j T ( I J − P ) V Y ∥ ≤ λ Y − 1 { ∥ e j T H ∥ + ∥ e j T V ∥∥ H ∥ } .
Since right multiplication by V Y T \mathbf{V}_{Y}^{\mathsf{T}} V Y T preserves the Euclidean norm of vectors in R r \mathbb{R}^{r} R r ,
∥ ( I J − P ) P ^ ∥ 2 → ∞ ≤ ∥ H ∥ 2 → ∞ + ∥ V ∥ 2 → ∞ ∥ H ∥ λ Y . (15) \left\|(\mathbf{I}_{J}-\mathbf{P})\widehat{\mathbf{P}}\right\|_{2\to\infty}\leq\frac{\left\|\mathbf{H}\right\|_{2\to\infty}+\left\|\mathbf{V}\right\|_{2\to\infty}\left\|\mathbf{H}\right\|}{\lambda_{Y}}. \tag*{(15)} ( I J − P ) P 2 → ∞ ≤ λ Y ∥ H ∥ 2 → ∞ + ∥ V ∥ 2 → ∞ ∥ H ∥ . (15)
Let C Y = V T V Y \mathbf{C}_{Y}=\mathbf{V}^{\mathsf{T}}\mathbf{V}_{Y} C Y = V T V Y . Orthogonality gives
∥ V T ( I J − P ^ ) ∥ 2 = 1 − σ r 2 ( C Y ) = ∥ ( I J − P ) V Y ∥ 2 , \left\|\mathbf{V}^{\mathsf{T}}(\mathbf{I}_{J}-\widehat{\mathbf{P}})\right\|^{2}=1-\sigma_{r}^{2}(\mathbf{C}_{Y})=\left\|(\mathbf{I}_{J}-\mathbf{P})\mathbf{V}_{Y}\right\|^{2}, V T ( I J − P ) 2 = 1 − σ r 2 ( C Y ) = ∥ ( I J − P ) V Y ∥ 2 ,
and hence
∥ P ( I J − P ^ ) ∥ 2 → ∞ ≤ ∥ V ∥ 2 → ∞ ∥ H ∥ λ Y . \left\|\mathbf{P}(\mathbf{I}_{J}-\widehat{\mathbf{P}})\right\|_{2\to\infty}\leq\left\|\mathbf{V}\right\|_{2\to\infty}\frac{\left\|\mathbf{H}\right\|}{\lambda_{Y}}. P ( I J − P ) 2 → ∞ ≤ ∥ V ∥ 2 → ∞ λ Y ∥ H ∥ .
Using P ^ − P = ( I J − P ) P ^ − P ( I J − P ^ ) \widehat{\mathbf{P}}-\mathbf{P}=(\mathbf{I}_{J}-\mathbf{P})\widehat{\mathbf{P}}-\mathbf{P}(\mathbf{I}_{J}-\widehat{\mathbf{P}}) P − P = ( I J − P ) P − P ( I J − P ) , we obtain
∥ P ^ − P ∥ 2 → ∞ ≤ ∥ H ∥ 2 → ∞ + 2 ∥ V ∥ 2 → ∞ ∥ H ∥ λ Y . (16) \left\|\widehat{\mathbf{P}}-\mathbf{P}\right\|_{2\to\infty}\leq\frac{\left\|\mathbf{H}\right\|_{2\to\infty}+2\left\|\mathbf{V}\right\|_{2\to\infty}\left\|\mathbf{H}\right\|}{\lambda_{Y}}. \tag*{(16)} P − P 2 → ∞ ≤ λ Y ∥ H ∥ 2 → ∞ + 2 ∥ V ∥ 2 → ∞ ∥ H ∥ . (16)
Since X ^ = Y P ^ \widehat{\mathbf{X}}=\mathbf{Y}\widehat{\mathbf{P}} X = Y P and M ∗ P = M ∗ \mathbf{M}^{*}\mathbf{P}=\mathbf{M}^{*} M ∗ P = M ∗ ,
X ^ − M ∗ = E P + Y ( P ^ − P ) . (17) \widehat{\mathbf{X}}-\mathbf{M}^{*}=\mathbf{E}\mathbf{P}+\mathbf{Y}(\widehat{\mathbf{P}}-\mathbf{P}). \tag*{(17)} X − M ∗ = EP + Y ( P − P ) . (17)
Therefore,
∥ X ^ − M ∗ ∥ max ≤ ∥ E V ∥ 2 → ∞ ∥ V ∥ 2 → ∞ + ∥ Y ∥ 2 → ∞ ∥ P ^ − P ∥ 2 → ∞ . (18) \left\|\widehat{\mathbf{X}}-\mathbf{M}^{*}\right\|_{\max}\leq\left\|\mathbf{E}\mathbf{V}\right\|_{2\to\infty}\left\|\mathbf{V}\right\|_{2\to\infty}+\left\|\mathbf{Y}\right\|_{2\to\infty}\left\|\widehat{\mathbf{P}}-\mathbf{P}\right\|_{2\to\infty}. \tag*{(18)} X − M ∗ m a x ≤ ∥ EV ∥ 2 → ∞ ∥ V ∥ 2 → ∞ + ∥ Y ∥ 2 → ∞ P − P 2 → ∞ . (18)
The preceding sub-exponential assumption gives
∥ E ∥ = O p ( N ) , (19) \left\|\mathbf{E}\right\|=O_{p}(\sqrt{N}), \tag*{(19)} ∥ E ∥ = O p ( N ) , (19)
and a truncation argument followed by Bernstein’s inequality and a union bound gives
∥ E ∥ 2 → ∞ = O p ( J ) , ∥ Y ∥ 2 → ∞ = O p ( J ) . (20) \lVert\mathbf{E}\rVert_{2\to\infty}=O_p(\sqrt{J}),\qquad\lVert\mathbf{Y}\rVert_{2\to\infty}=O_p(\sqrt{J}). \tag*{(20)} ∥ E ∥ 2 → ∞ = O p ( J ) , ∥ Y ∥ 2 → ∞ = O p ( J ) . (20)
Using the incoherence of U and V together with the weighted sub-exponential Bernstein inequality [40 Theorem 2.8.2] , we further obtain
∥ E V ∥ 2 → ∞ = O p ( L ) , ∥ E ⊤ U ∥ 2 → ∞ = O p ( L ) . (21) \lVert\mathbf{E}\mathbf{V}\rVert_{2\to\infty}=O_p(\sqrt{L}),\qquad\lVert\mathbf{E}^{\top}\mathbf{U}\rVert_{2\to\infty}=O_p(\sqrt{L}). \tag*{(21)} ∥ EV ∥ 2 → ∞ = O p ( L ) , ∥ E ⊤ U ∥ 2 → ∞ = O p ( L ) . (21)
By (13) and (19) , ∥ E ∥ / σ r ( M ∗ ) = O p ( J − 1 / 2 ) = o p ( 1 ) \lVert\mathbf{E}\rVert/\sigma_r(\mathbf{M}^{*})=O_p(J^{-1/2})=o_p(1) ∥ E ∥ / σ r ( M ∗ ) = O p ( J − 1/2 ) = o p ( 1 ) . Thus, with probability tending to one, σ r ( Y ) ≳ N J \sigma_r(\mathbf{Y})\gtrsim\sqrt{NJ} σ r ( Y ) ≳ N J and λ Y − 1 ≲ ( N J ) − 1 \lambda_Y^{-1}\lesssim(NJ)^{-1} λ Y − 1 ≲ ( N J ) − 1 . Since
H = M ∗ ⊤ E + E ⊤ M ∗ + E ⊤ E , \mathbf{H}=\mathbf{M}^{*\top}\mathbf{E}+\mathbf{E}^{\top}\mathbf{M}^{*}+\mathbf{E}^{\top}\mathbf{E}, H = M ∗ ⊤ E + E ⊤ M ∗ + E ⊤ E ,
we have
∥ H ∥ = O p ( N J ) , (22) \lVert\mathbf{H}\rVert=O_p(N\sqrt{J}), \tag*{(22)} ∥ H ∥ = O p ( N J ) , (22)
and, using M ∗ = U D V ⊤ \mathbf{M}^{*}=\mathbf{U}\mathbf{D}\mathbf{V}^{\top} M ∗ = UD V ⊤ ,
∥ H ∥ 2 → ∞ = O p ( N + N J L ) . (23) \lVert\mathbf{H}\rVert_{2\to\infty}=O_p\left(N+\sqrt{NJL}\right). \tag*{(23)} ∥ H ∥ 2 → ∞ = O p ( N + N J L ) . (23)
Substituting these bounds into (16) yields
∥ P ^ − P ∥ 2 → ∞ = O p ( 1 J + L N J ) . (24) \lVert\widehat{\mathbf{P}}-\mathbf{P}\rVert_{2\to\infty}=O_p\left(\frac{1}{J}+\sqrt{\frac{L}{NJ}}\right). \tag*{(24)} ∥ P − P ∥ 2 → ∞ = O p ( J 1 + N J L ) . (24)
Combining (18) , (20) , (21) , and (24) , we obtain
∥ X ^ − M ∗ ∥ max = O p { L J + J ( 1 J + L N J ) } = O p ( L J ) , \lVert\widehat{\mathbf{X}}-\mathbf{M}^{*}\rVert_{\max}
=O_p\left\{\sqrt{\frac{L}{J}}+\sqrt{J}\left(\frac{1}{J}+\sqrt{\frac{L}{NJ}}\right)\right\}
=O_p\left(\sqrt{\frac{L}{J}}\right), ∥ X − M ∗ ∥ m a x = O p { J L + J ( J 1 + N J L ) } = O p ( J L ) ,
where the last equality uses N ≥ J N\geq J N ≥ J . Finally, entrywise projection gives
∥ M ^ S V D − M ∗ ∥ max = O p ( log ( N J ) J ) . \lVert\widehat{\mathbf{M}}^{\mathrm{SVD}}-\mathbf{M}^{*}\rVert_{\max}
=O_p\left(\sqrt{\frac{\log(NJ)}{J}}\right). ∥ M SVD − M ∗ ∥ m a x = O p ( J log ( N J ) ) .
This proves the identity-link bounds in the bipartite setting.
Nonlinear Link
Let
Y ∗ = f ( M ∗ ) = ( y i j ∗ ) N × J , y i j ∗ = f ( m i j ∗ ) . \mathbf{Y}^{*}=f(\mathbf{M}^{*})=(y_{ij}^{*})_{N\times J},\qquad y_{ij}^{*}=f(m_{ij}^{*}). Y ∗ = f ( M ∗ ) = ( y ij ∗ ) N × J , y ij ∗ = f ( m ij ∗ ) .
Define the truncated observation matrix Y ^ = ( y ^ i j ) N × J \widehat{\mathbf{Y}}=(\widehat{y}_{ij})_{N\times J} Y = ( y ij ) N × J with y ^ i j = P [ a N J , b N J ] ( y i j ) \widehat{y}_{ij}=\mathcal{P}_{[a_{NJ},b_{NJ}]}(y_{ij}) y ij = P [ a N J , b N J ] ( y ij ) . We first decompose the estimation error as
1 N J E ∥ X ^ − Y ∗ ∥ F 2 ≤ 2 N J E ∥ X ^ − E ( Y ^ ) ∥ F 2 + 2 N J ∥ E ( Y ^ ) − Y ∗ ∥ F 2 . (25) \frac{1}{NJ}\mathbb{E}\lVert\widehat{\mathbf{X}}-\mathbf{Y}^{*}\rVert_{F}^{2}
\leq\frac{2}{NJ}\mathbb{E}\lVert\widehat{\mathbf{X}}-\mathbb{E}(\widehat{\mathbf{Y}})\rVert_{F}^{2}
+\frac{2}{NJ}\lVert\mathbb{E}(\widehat{\mathbf{Y}})-\mathbf{Y}^{*}\rVert_{F}^{2}. \tag*{(25)} N J 1 E ∥ X − Y ∗ ∥ F 2 ≤ N J 2 E ∥ X − E ( Y ) ∥ F 2 + N J 2 ∥ E ( Y ) − Y ∗ ∥ F 2 . (25)
By the definition of r N J r_{NJ} r N J , ∣ E y ^ i j − y i j ∗ ∣ ≤ r N J \lvert\mathbb{E}\widehat{y}_{ij}-y_{ij}^{*}\rvert\leq r_{NJ} ∣ E y ij − y ij ∗ ∣ ≤ r N J , and hence
2 N J ∥ E ( Y ^ ) − Y ∗ ∥ F 2 ≤ 2 r N J 2 . (26) \frac{2}{NJ}\lVert\mathbb{E}(\widehat{\mathbf{Y}})-\mathbf{Y}^{*}\rVert_{F}^{2}\leq2r_{NJ}^{2}. \tag*{(26)} N J 2 ∥ E ( Y ) − Y ∗ ∥ F 2 ≤ 2 r N J 2 . (26)
It remains to bound the first term in (25) . Let
r ~ 0 : = max { k : σ k ( Y ~ ) ≥ 1.01 N } , \widetilde{r}_{0}:=\max\left\{k:\sigma_{k}(\widetilde{\mathbf{Y}})\geq1.01\sqrt{N}\right\}, r 0 := max { k : σ k ( Y ) ≥ 1.01 N } ,
with r ~ 0 = 0 \widetilde r_0=0 r 0 = 0 if the above set is empty, and define the pure-threshold estimator
X ~ U S V T : = SVD r ~ 0 ( Y ~ ) . \widetilde X^{\mathrm{USVT}}:=\operatorname{SVD}_{\widetilde r_0}(\widetilde Y). X USVT := SVD r 0 ( Y ) .
Since y ~ i j ∈ [ 0 , 1 ] \widetilde y_{ij}\in[0,1] y ij ∈ [ 0 , 1 ] , Var ( y ~ i j ) ≤ 1 / 4 \operatorname{Var}(\widetilde y_{ij})\le1/4 Var ( y ij ) ≤ 1/4 . Conditional on the true latent configuration, the entries of Y ~ \widetilde Y Y are independent in the bipartite setting. Since N ≥ J N\ge J N ≥ J , we apply the variance-adjusted rectangular USVT argument to Y ~ T ∈ R J × N \widetilde Y^{\mathsf T}\in\mathbb{R}^{J\times N} Y T ∈ R J × N . Taking the fixed slack parameter so that the threshold equals 1.01 N 1.01\sqrt{N} 1.01 N , the argument underlying Theorem 1.1 of Chatterjee [9 ] yields
1 N J E ∥ X ~ U S V T − E ( Y ~ ) ∥ F 2 ≤ c min { ∥ E ( Y ~ ) ∥ ∗ J N , ∥ E ( Y ~ ) ∥ ∗ 2 N J , 1 } + c e − c N . (27) \frac{1}{NJ}\mathbb{E}\left\|\widetilde X^{\mathrm{USVT}}-\mathbb{E}(\widetilde Y)\right\|_{F}^{2}\le c\min\left\{\frac{\left\|\mathbb{E}(\widetilde Y)\right\|_*}{J\sqrt{N}},\frac{\left\|\mathbb{E}(\widetilde Y)\right\|_*^2}{NJ},1\right\}+ce^{-cN}. \tag*{(27)} N J 1 E X USVT − E ( Y ) F 2 ≤ c min ⎩ ⎨ ⎧ J N E ( Y ) ∗ , N J E ( Y ) ∗ 2 , 1 ⎭ ⎬ ⎫ + c e − c N . (27)
Our estimator instead uses r ~ = max { K + 2 , r ~ 0 } \widetilde r=\max\{K+2,\widetilde r_0\} r = max { K + 2 , r 0 } and X ~ = SVD r ~ ( Y ~ ) \widetilde X=\operatorname{SVD}_{\widetilde r}(\widetilde Y) X = SVD r ( Y ) . If r ~ 0 ≥ K + 2 \widetilde r_0\ge K+2 r 0 ≥ K + 2 , then X ~ = X ~ U S V T \widetilde X=\widetilde X^{\mathrm{USVT}} X = X USVT . Otherwise, all additionally retained singular values are smaller than 1.01 N 1.01\sqrt{N} 1.01 N , and hence
∥ X ~ − X ~ U S V T ∥ F 2 ≤ ( K + 2 ) ( 1.01 ) 2 N ≤ c N . \left\|\widetilde X-\widetilde X^{\mathrm{USVT}}\right\|_{F}^{2}\le(K+2)(1.01)^2N\le cN. X − X USVT F 2 ≤ ( K + 2 ) ( 1.01 ) 2 N ≤ c N .
Using ∥ A + B ∥ F 2 ≤ 2 ∥ A ∥ F 2 + 2 ∥ B ∥ F 2 \|A+B\|_{F}^{2}\le2\|A\|_{F}^{2}+2\|B\|_{F}^{2} ∥ A + B ∥ F 2 ≤ 2∥ A ∥ F 2 + 2∥ B ∥ F 2 and retaining only the first term in the minimum in (27) , we obtain
1 N J E ∥ X ~ − E ( Y ~ ) ∥ F 2 ≤ c ∥ E ( Y ~ ) ∥ ∗ J N + c e − c N + c J . (28) \frac{1}{NJ}\mathbb{E}\left\|\widetilde X-\mathbb{E}(\widetilde Y)\right\|_{F}^{2}\le c\frac{\left\|\mathbb{E}(\widetilde Y)\right\|_*}{J\sqrt{N}}+ce^{-cN}+\frac{c}{J}. \tag*{(28)} N J 1 E X − E ( Y ) F 2 ≤ c J N E ( Y ) ∗ + c e − c N + J c . (28)
Given the rescaling relations
E ( y ~ i j ) = ( b N J − a N J ) E ( y ~ i j ) + a N J , x ~ i j = ( b N J − a N J ) x ~ i j + a N J , \mathbb{E}(\widetilde y_{ij})=(b_{NJ}-a_{NJ})\mathbb{E}(\widetilde y_{ij})+a_{NJ},\qquad\widetilde x_{ij}=(b_{NJ}-a_{NJ})\widetilde x_{ij}+a_{NJ}, E ( y ij ) = ( b N J − a N J ) E ( y ij ) + a N J , x ij = ( b N J − a N J ) x ij + a N J ,
we have
∥ X ^ − E ( Y ) ∥ F 2 = ( b N J − a N J ) 2 ∥ X ~ − E ( Y ~ ) ∥ F 2 . \left\|\widehat X-\mathbb{E}(Y)\right\|_{F}^{2}=(b_{NJ}-a_{NJ})^{2}\left\|\widetilde X-\mathbb{E}(\widetilde Y)\right\|_{F}^{2}. X − E ( Y ) F 2 = ( b N J − a N J ) 2 X − E ( Y ) F 2 .
Moreover,
E Y ~ = 1 b N J − a N J { E Y ~ − a N J 1 N 1 J T } . \mathbb{E}\widetilde{Y}=\frac{1}{b_{NJ}-a_{NJ}}\left\{\mathbb{E}\widetilde{Y}-a_{NJ}\mathbf{1}_{N}\mathbf{1}_{J}^{\mathsf{T}}\right\}. E Y = b N J − a N J 1 { E Y − a N J 1 N 1 J T } .
Substituting these relations into (28) gives
1 N J E ( ∥ X ^ − E ( Y ^ ) ∥ F 2 ) ≤ c ( b N J − a N J ) 2 ∥ E Y ~ ∥ ∗ J N + c ( b N J − a N J ) 2 e − c N + c ( b N J − a N J ) 2 J = c b N J − a N J J N ∥ E Y ~ − a N J 1 N 1 J T ∥ ∗ + c ( b N J − a N J ) 2 e − c N + c ( b N J − a N J ) 2 J . (29) \begin{aligned}
\frac{1}{NJ}\mathbb{E}\left(\left\|\widehat{X}-\mathbb{E}(\widehat{Y})\right\|_{F}^{2}\right)
&\leq c(b_{NJ}-a_{NJ})^{2}\frac{\left\|\mathbb{E}\widetilde{Y}\right\|_{*}}{J\sqrt{N}}+c(b_{NJ}-a_{NJ})^{2}e^{-cN}+c\frac{(b_{NJ}-a_{NJ})^{2}}{J}\\
&=c\frac{b_{NJ}-a_{NJ}}{J\sqrt{N}}\left\|\mathbb{E}\widetilde{Y}-a_{NJ}\mathbf{1}_{N}\mathbf{1}_{J}^{\mathsf{T}}\right\|_{*}\\
&\quad+c(b_{NJ}-a_{NJ})^{2}e^{-cN}+c\frac{(b_{NJ}-a_{NJ})^{2}}{J}. \tag*{(29)}
\end{aligned} N J 1 E ( X − E ( Y ) F 2 ) ≤ c ( b N J − a N J ) 2 J N E Y ∗ + c ( b N J − a N J ) 2 e − c N + c J ( b N J − a N J ) 2 = c J N b N J − a N J E Y − a N J 1 N 1 J T ∗ + c ( b N J − a N J ) 2 e − c N + c J ( b N J − a N J ) 2 . (29)
To bound ∥ E Y ~ − a N J 1 N 1 J T ∥ ∗ \left\|\mathbb{E}\widetilde{Y}-a_{NJ}\mathbf{1}_{N}\mathbf{1}_{J}^{\mathsf{T}}\right\|_{*} E Y − a N J 1 N 1 J T ∗ , we construct configuration-dependent δ \delta δ -nets by selecting centers from the true embedding vectors. Specifically, let
G 1 ⊂ { θ i ∗ : i ∈ [ N ] } , G 2 ⊂ { θ N + j ∗ : j ∈ [ J ] } \mathcal{G}_{1}\subset\{\theta_{i}^{*}:i\in[N]\},\qquad\mathcal{G}_{2}\subset\{\theta_{N+j}^{*}:j\in[J]\} G 1 ⊂ { θ i ∗ : i ∈ [ N ]} , G 2 ⊂ { θ N + j ∗ : j ∈ [ J ]}
be maximal δ \delta δ -separated subsets. Then G 1 \mathcal{G}_{1} G 1 and G 2 \mathcal{G}_{2} G 2 are δ \delta δ -nets of the two configurations Θ P 1 ∗ \Theta_{\mathcal{P}_{1}}^{*} Θ P 1 ∗ and Θ P 2 ∗ \Theta_{\mathcal{P}_{2}}^{*} Θ P 2 ∗ , respectively, and every net center coincides with a true embedding vector.
For any θ i ∗ \theta_{i}^{*} θ i ∗ and θ N + j ∗ \theta_{N+j}^{*} θ N + j ∗ , let p ( θ i ∗ ) p(\theta_{i}^{*}) p ( θ i ∗ ) and p ( θ N + j ∗ ) p(\theta_{N+j}^{*}) p ( θ N + j ∗ ) be the corresponding nearest points in G 1 \mathcal{G}_{1} G 1 and G 2 \mathcal{G}_{2} G 2 . Under Assumption 1.1, the cardinalities satisfy ∣ G 1 ∣ , ∣ G 2 ∣ ≤ N δ : = c ( C 0 / δ ) K |\mathcal{G}_{1}|,|\mathcal{G}_{2}|\leq N_{\delta}:=c(C_{0}/\delta)^{K} ∣ G 1 ∣ , ∣ G 2 ∣ ≤ N δ := c ( C 0 / δ ) K .
We construct the approximation matrix Y ^ δ = ( y ^ i j δ ) N × J \widehat{Y}_{\delta}=(\widehat{y}_{ij}^{\delta})_{N\times J} Y δ = ( y ij δ ) N × J with entries
y ^ i j δ = f ( τ 0 − ∥ p ( θ i ∗ ) − p ( θ N + j ∗ ) ∥ 2 ) . \widehat{y}_{ij}^{\delta}=f\left(\tau_{0}-\left\|p(\theta_{i}^{*})-p(\theta_{N+j}^{*})\right\|^{2}\right). y ij δ = f ( τ 0 − p ( θ i ∗ ) − p ( θ N + j ∗ ) 2 ) .
Since Y ^ δ \widehat{Y}_{\delta} Y δ is generated by at most N δ N_{\delta} N δ distinct row vectors, its rank is bounded by
rank ( Y ^ δ ) ≤ min { ∣ G 1 ∣ , ∣ G 2 ∣ } ≤ N δ . \operatorname{rank}(\widehat{Y}_{\delta})\leq\min\{|\mathcal{G}_{1}|,|\mathcal{G}_{2}|\}\leq N_{\delta}. rank ( Y δ ) ≤ min { ∣ G 1 ∣ , ∣ G 2 ∣ } ≤ N δ .
We decompose the target nuclear norm into two parts:
∥ E Y ^ − a N , J 1 N 1 J T ∥ ∗ ≤ ∥ E Y ^ − Y ^ δ ∥ ∗ ⏟ (I) + ∥ Y ^ δ − a N , J 1 N 1 J T ∥ ∗ ⏟ (II) . (30) \left\|\mathbb{E}\widehat{Y}-a_{N,J}1_N1_J^{\mathsf{T}}\right\|_* \le\underbrace{\left\|\mathbb{E}\widehat{Y}-\widehat{Y}_{\delta}\right\|_*}_{\text{(I)}}+\underbrace{\left\|\widehat{Y}_{\delta}-a_{N,J}1_N1_J^{\mathsf{T}}\right\|_*}_{\text{(II)}}. \tag*{(30)} E Y − a N , J 1 N 1 J T ∗ ≤ (I) E Y − Y δ ∗ + (II) Y δ − a N , J 1 N 1 J T ∗ . (30)
Part (I):
Define m i j δ = τ 0 − ∥ p ( θ i ∗ ) − p ( θ N + j ∗ ) ∥ 2 m_{ij}^{\delta}=\tau_0-\left\|p(\theta_i^*)-p(\theta_{N+j}^*)\right\|^2 m ij δ = τ 0 − p ( θ i ∗ ) − p ( θ N + j ∗ ) 2 . Since the centers of G 1 G_1 G 1 and G 2 G_2 G 2 are selected from the true embedding vectors, for every ( i , j ) (i,j) ( i , j ) there exist i ′ ∈ [ N ] i'\in[N] i ′ ∈ [ N ] and j ′ ∈ [ J ] j'\in[J] j ′ ∈ [ J ] such that
p ( θ i ∗ ) = θ i ′ ∗ , p ( θ N + j ∗ ) = θ N + j ′ ∗ , p(\theta_i^*)=\theta_{i'}^*,\qquad p(\theta_{N+j}^*)=\theta_{N+j'}^*, p ( θ i ∗ ) = θ i ′ ∗ , p ( θ N + j ∗ ) = θ N + j ′ ∗ ,
and hence
m i j δ = τ 0 − ∥ θ i ′ ∗ − θ N + j ′ ∗ ∥ 2 = m i ′ j ′ ∗ . m_{ij}^{\delta}=\tau_0-\left\|\theta_{i'}^*-\theta_{N+j'}^*\right\|^2=m_{i'j'}^*. m ij δ = τ 0 − θ i ′ ∗ − θ N + j ′ ∗ 2 = m i ′ j ′ ∗ .
Therefore, ∣ m i j δ ∣ ≤ C 0 \left|m_{ij}^{\delta}\right|\le C_0 m ij δ ≤ C 0 . Hence, both m i j ∗ m_{ij}^* m ij ∗ and m i j δ m_{ij}^{\delta} m ij δ lie in [ − C 0 , C 0 ] [-C_0,C_0] [ − C 0 , C 0 ] . By Assumption 1.3, let L f : = sup ∣ m ∣ ≤ C 0 ∣ f ′ ( m ) ∣ < ∞ L_f:=\sup_{|m|\le C_0}|f'(m)|<\infty L f := sup ∣ m ∣ ≤ C 0 ∣ f ′ ( m ) ∣ < ∞ . Thus, f f f is L f L_f L f -Lipschitz on [ − C 0 , C 0 ] [-C_0,C_0] [ − C 0 , C 0 ] . Then
∣ E y ^ i j − y ^ i j δ ∣ ≤ ∣ E y ^ i j − f ( m i j ∗ ) ∣ + ∣ f ( m i j ∗ ) − y ^ i j δ ∣ ≤ r N J + L f ∣ ∥ θ i ∗ − θ N + j ∗ ∥ 2 − ∥ p ( θ i ∗ ) − p ( θ N + j ∗ ) ∥ 2 ∣ ≤ r N J + 8 C 0 L f δ . (31) \begin{aligned}
\left|\mathbb{E}\widehat{y}_{ij}-\widehat{y}_{ij}^{\delta}\right|\le\left|\mathbb{E}\widehat{y}_{ij}-f(m_{ij}^*)\right|+\left|f(m_{ij}^*)-\widehat{y}_{ij}^{\delta}\right| \\
&\le r_{NJ}+L_f\left|\left\|\theta_i^*-\theta_{N+j}^*\right\|^2-\left\|p(\theta_i^*)-p(\theta_{N+j}^*)\right\|^2\right| \\
&\le r_{NJ}+8C_0L_f\delta. \tag*{(31)}
\end{aligned} E y ij − y ij δ ≤ E y ij − f ( m ij ∗ ) + f ( m ij ∗ ) − y ij δ ≤ r N J + L f θ i ∗ − θ N + j ∗ 2 − p ( θ i ∗ ) − p ( θ N + j ∗ ) 2 ≤ r N J + 8 C 0 L f δ . (31)
Therefore,
(I) ≤ J ∥ E Y ^ − Y ^ δ ∥ F ≤ J N J ( r N J + 8 C 0 L f δ ) ≤ ( r N J + c δ ) J N . (32) \begin{aligned}
\text{(I)}&\le\sqrt{J}\left\|\mathbb{E}\widehat{Y}-\widehat{Y}_{\delta}\right\|_F\\
&\le\sqrt{J}\sqrt{NJ}(r_{NJ}+8C_0L_f\delta)\\
&\le(r_{NJ}+c\delta)J\sqrt{N}. \tag*{(32)}
\end{aligned} (I) ≤ J E Y − Y δ F ≤ J N J ( r N J + 8 C 0 L f δ ) ≤ ( r N J + cδ ) J N . (32)
Part (II):
For fixed i i i , the i i i th row of Y ^ δ \widehat{Y}_{\delta} Y δ depends on i i i only through p ( θ i ∗ ) p(\theta_i^*) p ( θ i ∗ ) . Therefore,
rank ( Y ^ δ − a N J 1 N 1 J T ) ≤ N δ . (33) \operatorname{rank}\left(\widehat{Y}_{\delta}-a_{NJ}1_N1_J^{\mathsf T}\right)\le N_{\delta}. \tag*{(33)} rank ( Y δ − a N J 1 N 1 J T ) ≤ N δ . (33)
Since the net centers are true embedding vectors, for every ( i , j ) (i,j) ( i , j ) there exist indices i ′ ∈ [ N ] i'\in[N] i ′ ∈ [ N ] and j ′ ∈ [ J ] j'\in[J] j ′ ∈ [ J ] such that m i j δ = m i ′ j ′ ∗ m_{ij}^{\delta}=m_{i'j'}^* m ij δ = m i ′ j ′ ∗ . Because E Y ^ i ′ j ′ ∈ [ a N J , b N J ] \mathbb{E}\widehat{Y}_{i'j'}\in[a_{NJ},b_{NJ}] E Y i ′ j ′ ∈ [ a N J , b N J ] and ∣ E Y ^ i ′ j ′ − f ( m i ′ j ′ ∗ ) ∣ ≤ r N J \left|\mathbb{E}\widehat{Y}_{i'j'}-f(m_{i'j'}^*)\right|\le r_{NJ} E Y i ′ j ′ − f ( m i ′ j ′ ∗ ) ≤ r N J , we obtain
∣ Y ^ i j δ − a N J ∣ ≤ b N J − a N J + r N J . \left|\widehat{Y}_{ij}^{\delta}-a_{NJ}\right|\le b_{NJ}-a_{NJ}+r_{NJ}. Y ij δ − a N J ≤ b N J − a N J + r N J .
Consequently,
( I I ) ≤ N δ ∥ Y ^ δ − a N J 1 N 1 J T ∥ F ≤ c δ − K / 2 N J ( b N J − a N J + r N J ) . (34) \begin{aligned}
(\mathrm{II})&\le\sqrt{N_{\delta}}\left\|\widehat{Y}_{\delta}-a_{NJ}1_N1_J^{\mathsf T}\right\|_F\\
&\le c\delta^{-K/2}\sqrt{NJ}\left(b_{NJ}-a_{NJ}+r_{NJ}\right). \tag*{(34)}
\end{aligned} ( II ) ≤ N δ Y δ − a N J 1 N 1 J T F ≤ c δ − K /2 N J ( b N J − a N J + r N J ) . (34)
Substituting (32) and (34) into (30) , and then into (29) , we obtain
1 N J E ( ∥ X ^ − E ( Y ^ ) ∥ F 2 ) ≤ c R N J ( r N J + δ ) + c R N J ( R N J + r N J ) δ − K / 2 J + c R N J 2 e − c N + c R N J 2 J . \frac{1}{NJ}\mathbb{E}\left(\left\|\widehat{X}-\mathbb{E}(\widehat{Y})\right\|_F^2\right)
\le cR_{NJ}(r_{NJ}+\delta)+cR_{NJ}(R_{NJ}+r_{NJ})\frac{\delta^{-K/2}}{\sqrt{J}}+cR_{NJ}^2e^{-cN}+c\frac{R_{NJ}^2}{J}. N J 1 E ( X − E ( Y ) F 2 ) ≤ c R N J ( r N J + δ ) + c R N J ( R N J + r N J ) J δ − K /2 + c R N J 2 e − c N + c J R N J 2 .
Combining this with (26) , we obtain
1 N J E ∥ X ^ − Y ∗ ∥ F 2 ≤ c R N J ( r N J + δ ) + c R N J ( R N J + r N J ) δ − K / 2 J + c r N J 2 + c R N J 2 e − c N + c R N J 2 J . \frac{1}{NJ}\mathbb{E}\left\|\widehat{X}-Y^*\right\|_F^2
\le cR_{NJ}(r_{NJ}+\delta)+cR_{NJ}(R_{NJ}+r_{NJ})\frac{\delta^{-K/2}}{\sqrt{J}}+cr_{NJ}^2+cR_{NJ}^2e^{-cN}+c\frac{R_{NJ}^2}{J}. N J 1 E X − Y ∗ F 2 ≤ c R N J ( r N J + δ ) + c R N J ( R N J + r N J ) J δ − K /2 + c r N J 2 + c R N J 2 e − c N + c J R N J 2 .
Choose
δ = { ( R N J + r N J ) 2 J } 1 K + 2 . \delta=\left\{\frac{(R_{NJ}+r_{NJ})^2}{J}\right\}^{\frac{1}{K+2}}. δ = { J ( R N J + r N J ) 2 } K + 2 1 .
Under the assumption R N J + r N J = o ( J ) R_{NJ}+r_{NJ}=o(\sqrt{J}) R N J + r N J = o ( J ) , we have δ = o ( 1 ) \delta=o(1) δ = o ( 1 ) . Substituting this choice and absorbing the remainder terms gives
1 N J E ∥ X ^ − Y ∗ ∥ F 2 ≤ c R N J { ( R N J + r N J ) 2 J } 1 K + 2 + c r N J ( R N J + r N J ) . (35) \frac{1}{NJ}\mathbb{E}\lVert\widehat{X}-Y^*\rVert_F^2 \le cR_{NJ}\left\{\frac{(R_{NJ}+r_{NJ})^2}{J}\right\}^{\frac{1}{K+2}}+cr_{NJ}(R_{NJ}+r_{NJ}). \tag*{(35)} N J 1 E ∥ X − Y ∗ ∥ F 2 ≤ c R N J { J ( R N J + r N J ) 2 } K + 2 1 + c r N J ( R N J + r N J ) . (35)
Since m i j ∗ ∈ [ − C 0 , C 0 ] ⊆ [ − C p r o j , C p r o j ] m^*_{ij}\in[-C_0,C_0]\subseteq[-C_{\mathrm{proj}},C_{\mathrm{proj}}] m ij ∗ ∈ [ − C 0 , C 0 ] ⊆ [ − C proj , C proj ] , we have
y i j ∗ = f ( m i j ∗ ) ∈ [ f ( − C p r o j ) , f ( C p r o j ) ] . y^*_{ij}=f(m^*_{ij})\in[f(-C_{\mathrm{proj}}),f(C_{\mathrm{proj}})]. y ij ∗ = f ( m ij ∗ ) ∈ [ f ( − C proj ) , f ( C proj )] .
Hence,
∣ m ^ i j S V D − m i j ∗ ∣ ≤ ∣ f − 1 ( P [ f ( − C p r o j ) , f ( C p r o j ) ] ( x ^ i j ) ) − f − 1 ( y i j ∗ ) ∣ ≤ ℓ † ∣ x ^ i j − y i j ∗ ∣ . \left|\widehat{m}^{\mathrm{SVD}}_{ij}-m^*_{ij}\right|
\le\left|f^{-1}\left(\mathcal{P}_{[f(-C_{\mathrm{proj}}),f(C_{\mathrm{proj}})]}(\widehat{x}_{ij})\right)-f^{-1}(y^*_{ij})\right|
\le\ell^\dagger\left|\widehat{x}_{ij}-y^*_{ij}\right|. m ij SVD − m ij ∗ ≤ f − 1 ( P [ f ( − C proj ) , f ( C proj )] ( x ij ) ) − f − 1 ( y ij ∗ ) ≤ ℓ † x ij − y ij ∗ .
where
ℓ † : = { inf ∣ m ∣ ≤ C p r o j f ′ ( m ) } − 1 < ∞ . \ell^\dagger:=\left\{\inf_{|m|\le C_{\mathrm{proj}}}f'(m)\right\}^{-1}<\infty. ℓ † := { ∣ m ∣ ≤ C proj inf f ′ ( m ) } − 1 < ∞.
Therefore,
∥ M ^ S V D − M ∗ ∥ F 2 ≤ ( ℓ † ) 2 ∥ X ^ − Y ∗ ∥ F 2 . \lVert\widehat{M}^{\mathrm{SVD}}-M^*\rVert_F^2\le(\ell^\dagger)^2\lVert\widehat{X}-Y^*\rVert_F^2. ∥ M SVD − M ∗ ∥ F 2 ≤ ( ℓ † ) 2 ∥ X − Y ∗ ∥ F 2 .
Combining this inequality with (35) and applying Markov’s inequality yields
1 N J ∥ M ^ S V D − M ∗ ∥ F 2 = O p ( R N J { ( R N J + r N J ) 2 J } 1 K + 2 + r N J ( R N J + r N J ) ) . \frac{1}{NJ}\lVert\widehat{M}^{\mathrm{SVD}}-M^*\rVert_F^2
=O_p\left(R_{NJ}\left\{\frac{(R_{NJ}+r_{NJ})^2}{J}\right\}^{\frac{1}{K+2}}+r_{NJ}(R_{NJ}+r_{NJ})\right). N J 1 ∥ M SVD − M ∗ ∥ F 2 = O p ( R N J { J ( R N J + r N J ) 2 } K + 2 1 + r N J ( R N J + r N J ) ) .
Unipartite Spectral Denoising
For unipartite data, we set J = N J=N J = N . The main arguments from the bipartite setting continue to apply, but the observation matrix is symmetric, only the upper-triangular observations are mutually independent, and the diagonal entries are not observed. We record the required modifications separately.
Identity Link
For unipartite data, let ε i j = y i j − m i j ∗ \varepsilon_{ij}=y_{ij}-m^{*}_{ij} ε ij = y ij − m ij ∗ for i ≠ j i\ne j i = j , and define W = ( w i j ) N × N \mathbf{W}=(w_{ij})_{N\times N} W = ( w ij ) N × N by
w i j = { ε i j , i < j , ε j i , i > j , 0 , i = j . w_{ij}=
\begin{cases}
\varepsilon_{ij}, & i<j,\\
\varepsilon_{ji}, & i>j,\\
0, & i=j.
\end{cases} w ij = ⎩ ⎨ ⎧ ε ij , ε j i , 0 , i < j , i > j , i = j .
Recall that y i i = 0 y_{ii}=0 y ii = 0 is not treated as an observation, while the algebraically completed signal satisfies m i i ∗ = τ 0 m^{*}_{ii}=\tau_0 m ii ∗ = τ 0 . Therefore,
Y = M ∗ + W − τ 0 I N . \mathbf{Y}=\mathbf{M}^{*}+\mathbf{W}-\tau_0\mathbf{I}_N. Y = M ∗ + W − τ 0 I N .
By Assumption 1.1, ∣ τ 0 ∣ ≤ C 0 |\tau_0|\le C_0 ∣ τ 0 ∣ ≤ C 0 . Conditional on the true latent configuration, the upper-triangular entries { w i j : i < j } \{w_{ij}:i<j\} { w ij : i < j } are independent, mean zero, and uniformly sub-exponential. Hence ∥ W ∥ 2 = O p ( N ) \|\mathbf{W}\|_2=O_p(\sqrt{N}) ∥ W ∥ 2 = O p ( N ) and therefore ∥ W − τ 0 I N ∥ 2 = O p ( N ) \|\mathbf{W}-\tau_0\mathbf{I}_N\|_2=O_p(\sqrt{N}) ∥ W − τ 0 I N ∥ 2 = O p ( N ) . Repeating the preceding Eckart–Young argument and using C p r o j ≥ C 0 C_{\mathrm{proj}}\ge C_0 C proj ≥ C 0 gives
1 N 2 ∥ M ^ S V D − M ∗ ∥ F 2 = O p ( N − 1 ) , \frac{1}{N^2}\|\widehat{\mathbf{M}}^{\mathrm{SVD}}-\mathbf{M}^{*}\|_F^2=O_p(N^{-1}), N 2 1 ∥ M SVD − M ∗ ∥ F 2 = O p ( N − 1 ) ,
which is O p ( J − 1 ) O_p(J^{-1}) O p ( J − 1 ) since J = N J=N J = N .
We next establish the max-norm bound. Put r = K + 2 r=K+2 r = K + 2 and write
M ∗ = U D V T , X ^ = SVD r ( Y ) . \mathbf{M}^{*}=\mathbf{U}\mathbf{D}\mathbf{V}^{\mathsf{T}},\qquad\widehat{\mathbf{X}}=\operatorname{SVD}_r(\mathbf{Y}). M ∗ = UD V T , X = SVD r ( Y ) .
The deterministic relations (14)–(18) remain valid because they do not require independence of the entries of the perturbation matrix. Set
E = Y − M ∗ = W − τ 0 I N , L = log ( N 2 ) . \mathbf{E}=\mathbf{Y}-\mathbf{M}^{*}=\mathbf{W}-\tau_{0}\mathbf{I}_{N}, \qquad L=\log(N^{2}). E = Y − M ∗ = W − τ 0 I N , L = log ( N 2 ) .
Let W + \mathbf{W}_{+} W + denote the strictly upper-triangular part of W \mathbf{W} W . Its entries, including the deterministic zeros, are independent and have uniformly bounded second and fourth moments. By Theorem 2 of Latala [27 ] ,
E ∥ W + ∥ ≲ N . \mathbb{E}\|\mathbf{W}_{+}\|\lesssim\sqrt{N}. E ∥ W + ∥ ≲ N .
Since W = W + + W + T \mathbf{W}=\mathbf{W}_{+}+\mathbf{W}_{+}^{\mathsf{T}} W = W + + W + T ,
∥ E ∥ = O p ( N ) , ∥ E ∥ 2 → ∞ = O p ( N ) , ∥ Y ∥ 2 → ∞ = O p ( N ) . \|\mathbf{E}\|=O_{p}(\sqrt{N}), \qquad\|\mathbf{E}\|_{2\to\infty}=O_{p}(\sqrt{N}), \qquad\|\mathbf{Y}\|_{2\to\infty}=O_{p}(\sqrt{N}). ∥ E ∥ = O p ( N ) , ∥ E ∥ 2 → ∞ = O p ( N ) , ∥ Y ∥ 2 → ∞ = O p ( N ) .
For each fixed i ∈ [ N ] i\in[N] i ∈ [ N ] and k ∈ [ r ] k\in[r] k ∈ [ r ] ,
( W V ) i k = ∑ j ≠ i W i j V j k . (\mathbf{W}\mathbf{V})_{ik}=\sum_{j\ne i}W_{ij}V_{jk}. ( WV ) ik = j = i ∑ W ij V j k .
The variables in this sum correspond to distinct unordered pairs and are therefore independent. Using the incoherence of V \mathbf{V} V and the weighted Bernstein inequality, and proceeding analogously for W T U \mathbf{W}^{\mathsf{T}}\mathbf{U} W T U , gives
∥ W V ∥ 2 → ∞ + ∥ W T U ∥ 2 → ∞ = O p ( L ) . \|\mathbf{W}\mathbf{V}\|_{2\to\infty}+\|\mathbf{W}^{\mathsf{T}}\mathbf{U}\|_{2\to\infty}=O_{p}(\sqrt{L}). ∥ WV ∥ 2 → ∞ + ∥ W T U ∥ 2 → ∞ = O p ( L ) .
The deterministic diagonal correction satisfies
∥ τ 0 V ∥ 2 → ∞ + ∥ τ 0 U ∥ 2 → ∞ = O ( N − 1 / 2 ) , \|\tau_{0}\mathbf{V}\|_{2\to\infty}+\|\tau_{0}\mathbf{U}\|_{2\to\infty}=O(N^{-1/2}), ∥ τ 0 V ∥ 2 → ∞ + ∥ τ 0 U ∥ 2 → ∞ = O ( N − 1/2 ) ,
so
∥ E V ∥ 2 → ∞ + ∥ E ⊤ U ∥ 2 → ∞ = O p ( L ) . (36) \lVert\mathbf{E}\mathbf{V}\rVert_{2\to\infty}+\lVert\mathbf{E}^{\top}\mathbf{U}\rVert_{2\to\infty}=O_p(\sqrt{L}). \tag*{(36)} ∥ EV ∥ 2 → ∞ + ∥ E ⊤ U ∥ 2 → ∞ = O p ( L ) . (36)
Since σ r ( M ∗ ) ≳ N \sigma_r(\mathbf{M}^*)\gtrsim N σ r ( M ∗ ) ≳ N and ∥ E ∥ = O p ( N ) \lVert\mathbf{E}\rVert=O_p(\sqrt{N}) ∥ E ∥ = O p ( N ) , we have, with probability tending to one, λ Y ≳ N 2 \lambda_Y\gtrsim N^2 λ Y ≳ N 2 .
Moreover,
∥ H ∥ ≤ 2 ∥ M ∗ ∥ ∥ E ∥ + ∥ E ∥ 2 = O p ( N 3 / 2 ) , \lVert\mathbf{H}\rVert\leq2\lVert\mathbf{M}^*\rVert\lVert\mathbf{E}\rVert+\lVert\mathbf{E}\rVert^2=O_p(N^{3/2}), ∥ H ∥ ≤ 2 ∥ M ∗ ∥ ∥ E ∥ + ∥ E ∥ 2 = O p ( N 3/2 ) ,
∥ H ∥ 2 → ∞ ≤ ∥ M ∗ ⊤ ∥ 2 → ∞ ∥ E ∥ + ∥ E ⊤ U ∥ 2 → ∞ ∥ D ∥ + ∥ E ∥ 2 = O p ( N + N L ) . \begin{aligned}
\lVert\mathbf{H}\rVert_{2\to\infty}
&\leq\lVert\mathbf{M}^{*\top}\rVert_{2\to\infty}\lVert\mathbf{E}\rVert+\lVert\mathbf{E}^{\top}\mathbf{U}\rVert_{2\to\infty}\lVert\mathbf{D}\rVert+\lVert\mathbf{E}\rVert^2\\
&=O_p(N+N\sqrt{L}).
\end{aligned} ∥ H ∥ 2 → ∞ ≤ ∥ M ∗ ⊤ ∥ 2 → ∞ ∥ E ∥ + ∥ E ⊤ U ∥ 2 → ∞ ∥ D ∥ + ∥ E ∥ 2 = O p ( N + N L ) .
Thus, by (16) ,
∥ P ^ − P ∥ 2 → ∞ = O p ( 1 + L N ) . \lVert\widehat{\mathbf{P}}-\mathbf{P}\rVert_{2\to\infty}=O_p\left(\frac{1+\sqrt{L}}{N}\right). ∥ P − P ∥ 2 → ∞ = O p ( N 1 + L ) .
Combining this with (18) and (36) yields
∥ X ^ − M ∗ ∥ max = O p { L N + N 1 + L N } = O p ( L N ) . \begin{aligned}
\lVert\widehat{\mathbf{X}}-\mathbf{M}^*\rVert_{\max}
&=O_p\left\{\sqrt{\frac{L}{N}}+\sqrt{N}\frac{1+\sqrt{L}}{N}\right\}\\
&=O_p\left(\sqrt{\frac{L}{N}}\right).
\end{aligned} ∥ X − M ∗ ∥ m a x = O p { N L + N N 1 + L } = O p ( N L ) .
Since entrywise projection cannot increase the error relative to M ∗ \mathbf{M}^* M ∗ ,
∥ M ^ S V D − M ∗ ∥ max = O p ( log N N ) = O p ( log ( N J ) J ) , \lVert\widehat{\mathbf{M}}^{\mathrm{SVD}}-\mathbf{M}^*\rVert_{\max}
=O_p\left(\sqrt{\frac{\log N}{N}}\right)
=O_p\left(\sqrt{\frac{\log(NJ)}{J}}\right), ∥ M SVD − M ∗ ∥ m a x = O p ( N log N ) = O p ( J log ( N J ) ) ,
where J = N J=N J = N . This proves the identity-link bounds in the unipartite setting.
Nonlinear Link
We now consider the unipartite case, where J = N J=N J = N , y j i = y i j y_{ji}=y_{ij} y j i = y ij , and y i i = 0 y_{ii}=0 y ii = 0 . Let
Y ∗ = f ( M ∗ ) = ( y i j ∗ ) N × N , y i j ∗ = f ( m i j ∗ ) , \mathbf{Y}^*=f(\mathbf{M}^*)=(y_{ij}^*)_{N\times N},\qquad y_{ij}^*=f(m_{ij}^*), Y ∗ = f ( M ∗ ) = ( y ij ∗ ) N × N , y ij ∗ = f ( m ij ∗ ) ,
and let R N N : = b N N − a N N R_{NN} := b_{NN} - a_{NN} R N N := b N N − a N N . Recall that
r N N = max i ≠ j ∣ E y ^ i j − y i j ∗ ∣ . r_{NN} = \max_{i \ne j} \left|\mathbb{E}\widehat{y}_{ij} - y^*_{ij}\right|. r N N = i = j max E y ij − y ij ∗ .
For the diagonal entries, y i i = 0 y_{ii} = 0 y ii = 0 , while the algebraically completed signal satisfies m i i ∗ = τ 0 m^*_{ii} = \tau_0 m ii ∗ = τ 0 . Hence, under the condition f ( τ 0 ) ∈ [ a N N , b N N ] f(\tau_0) \in[a_{NN}, b_{NN}] f ( τ 0 ) ∈ [ a N N , b N N ] ,
∣ E y ^ i i − y i i ∗ ∣ = ∣ P [ a N N , b N N ] ( 0 ) − f ( τ 0 ) ∣ ≤ R N N . \left|\mathbb{E}\widehat{y}_{ii} - y^*_{ii}\right| = \left|\mathcal{P}_{[a_{NN},b_{NN}]}(0) - f(\tau_0)\right| \le R_{NN}. ∣ E y ii − y ii ∗ ∣ = P [ a N N , b N N ] ( 0 ) − f ( τ 0 ) ≤ R N N .
We then decompose
1 N 2 E ∥ X ^ − Y ∗ ∥ F 2 ≤ 2 N 2 E ∥ X ^ − E ( Y ^ ) ∥ F 2 + 2 N 2 ∥ E ( Y ^ ) − Y ∗ ∥ F 2 . (37) \frac{1}{N^2}\mathbb{E}\left\|\widehat{X} - Y^*\right\|_F^2
\le\frac{2}{N^2}\mathbb{E}\left\|\widehat{X} - \mathbb{E}(\widehat{Y})\right\|_F^2
+ \frac{2}{N^2}\left\|\mathbb{E}(\widehat{Y}) - Y^*\right\|_F^2. \tag*{(37)} N 2 1 E X − Y ∗ F 2 ≤ N 2 2 E X − E ( Y ) F 2 + N 2 2 E ( Y ) − Y ∗ F 2 . (37)
By the definition of r N N r_{NN} r N N , the off-diagonal entries satisfy ∣ E y ^ i j − y i j ∗ ∣ ≤ r N N \left|\mathbb{E}\widehat{y}_{ij} - y^*_{ij}\right| \le r_{NN} E y ij − y ij ∗ ≤ r N N , i ≠ j i \ne j i = j .
Combining this with the diagonal bound above yields
2 N 2 ∥ E ( Y ^ ) − Y ∗ ∥ F 2 ≤ 2 r N N 2 + 2 R N N 2 N . (38) \frac{2}{N^2}\left\|\mathbb{E}(\widehat{Y}) - Y^*\right\|_F^2
\le2r_{NN}^2 + \frac{2R_{NN}^2}{N}. \tag*{(38)} N 2 2 E ( Y ) − Y ∗ F 2 ≤ 2 r N N 2 + N 2 R N N 2 . (38)
Applying the symmetric analogue of the modified USVT argument above gives
1 N 2 E ∥ X ^ − E ( Y ^ ) ∥ F 2 ≤ R N N N 3 / 2 ∥ E ( Y ^ ) − a N N 1 N 1 N T ∥ ∗ + c R N N 2 e − c N + c R N N 2 N . (39) \frac{1}{N^2}\mathbb{E}\left\|\widehat{X} - \mathbb{E}(\widehat{Y})\right\|_F^2
\le\frac{R_{NN}}{N^{3/2}}\left\|\mathbb{E}(\widehat{Y}) - a_{NN}\mathbf{1}_N\mathbf{1}_N^\mathsf{T}\right\|_* + cR_{NN}^2e^{-cN} + c\frac{R_{NN}^2}{N}. \tag*{(39)} N 2 1 E X − E ( Y ) F 2 ≤ N 3/2 R N N E ( Y ) − a N N 1 N 1 N T ∗ + c R N N 2 e − c N + c N R N N 2 . (39)
Define the diagonal completion Y ‾ = ( y ‾ i j ) N × N \overline{Y} = (\overline{y}_{ij})_{N \times N} Y = ( y ij ) N × N by
y ‾ i j = { E y ^ i j , i ≠ j , f ( τ 0 ) , i = j . \overline{y}_{ij} =
\begin{cases}
\mathbb{E}\widehat{y}_{ij}, & i \ne j,\\
f(\tau_0), & i = j.
\end{cases} y ij = { E y ij , f ( τ 0 ) , i = j , i = j .
Since E Y ^ i i = P [ a N N , b N N ] ( 0 ) \mathbb{E}\widehat{Y}_{ii}=\mathbb{P}_{[a_{NN},b_{NN}]}(0) E Y ii = P [ a N N , b N N ] ( 0 ) , we have
∥ E ( Y ^ ) − Y ‾ ∥ ∗ ≤ N R N N . (40) \left\|\mathbb{E}(\widehat{Y})-\overline{Y}\right\|_* \leq N R_{NN}. \tag*{(40)} E ( Y ) − Y ∗ ≤ N R N N . (40)
Consequently,
∥ E ( Y ^ ) − a N N 1 N 1 N T ∥ ∗ ≤ N R N N + ∥ Y ‾ − a N N 1 N 1 N T ∥ ∗ . (41) \left\|\mathbb{E}(\widehat{Y})-a_{NN}1_N1_N^{\mathsf T}\right\|_* \leq N R_{NN}+\left\|\overline{Y}-a_{NN}1_N1_N^{\mathsf T}\right\|_*. \tag*{(41)} E ( Y ) − a N N 1 N 1 N T ∗ ≤ N R N N + Y − a N N 1 N 1 N T ∗ . (41)
Let G \mathcal{G} G be a maximal δ \delta δ -separated subset of { θ i ∗ : i ∈ [ N ] } \{\theta_i^*:i\in[N]\} { θ i ∗ : i ∈ [ N ]} . Then ∣ G ∣ ≤ N δ : = c ( C 0 / δ ) K |\mathcal{G}|\leq N_\delta:=c(C_0/\delta)^K ∣ G ∣ ≤ N δ := c ( C 0 / δ ) K . For each i ∈ [ N ] i\in[N] i ∈ [ N ] , let p ( θ i ∗ ) ∈ G p(\theta_i^*)\in\mathcal{G} p ( θ i ∗ ) ∈ G satisfy ∥ θ i ∗ − p ( θ i ∗ ) ∥ ≤ δ \|\theta_i^*-p(\theta_i^*)\|\leq\delta ∥ θ i ∗ − p ( θ i ∗ ) ∥ ≤ δ , and define
m i j δ : = τ 0 − ∥ p ( θ i ∗ ) − p ( θ j ∗ ) ∥ 2 , y ^ i j δ : = f ( m i j δ ) . m_{ij}^{\delta}:=\tau_0-\|p(\theta_i^*)-p(\theta_j^*)\|^2,\qquad\widehat{y}_{ij}^{\delta}:=f(m_{ij}^{\delta}). m ij δ := τ 0 − ∥ p ( θ i ∗ ) − p ( θ j ∗ ) ∥ 2 , y ij δ := f ( m ij δ ) .
Let Y ^ δ = ( y ^ i j δ ) N × N \widehat{Y}_{\delta}=(\widehat{y}_{ij}^{\delta})_{N\times N} Y δ = ( y ij δ ) N × N .
Let L f = sup ∣ m ∣ ≤ C 0 ∣ f ′ ( m ) ∣ < ∞ L_f=\sup_{|m|\leq C_0}|f'(m)|<\infty L f = sup ∣ m ∣ ≤ C 0 ∣ f ′ ( m ) ∣ < ∞ . As in the bipartite case, ∣ m i j ∗ − m i j δ ∣ ≤ 8 C 0 δ |m_{ij}^*-m_{ij}^{\delta}|\leq8C_0\delta ∣ m ij ∗ − m ij δ ∣ ≤ 8 C 0 δ .
We decompose
∥ Y ‾ − a N N 1 N 1 N T ∥ ∗ ≤ ∥ Y ‾ − Y ^ δ ∥ ∗ ⏟ (I) + ∥ Y ^ δ − a N N 1 N 1 N T ∥ ∗ ⏟ (II) . (42) \left\|\overline{Y}-a_{NN}1_N1_N^{\mathsf T}\right\|_*
\leq
\underbrace{\left\|\overline{Y}-\widehat{Y}_{\delta}\right\|_*}_{\text{(I)}}
+
\underbrace{\left\|\widehat{Y}_{\delta}-a_{NN}1_N1_N^{\mathsf T}\right\|_*}_{\text{(II)}}. \tag*{(42)} Y − a N N 1 N 1 N T ∗ ≤ (I) Y − Y δ ∗ + (II) Y δ − a N N 1 N 1 N T ∗ . (42)
For Part (I), if i ≠ j i\ne j i = j ,
∣ y ‾ i j − y ^ i j δ ∣ ≤ r N N + 8 C 0 L f δ . |\overline{y}_{ij}-\widehat{y}_{ij}^{\delta}|\leq r_{NN}+8C_0L_f\delta. ∣ y ij − y ij δ ∣ ≤ r N N + 8 C 0 L f δ .
For i = j i=j i = j , y ‾ i i = y ^ i i δ = f ( τ 0 ) \overline{y}_{ii}=\widehat{y}_{ii}^{\delta}=f(\tau_0) y ii = y ii δ = f ( τ 0 ) . Therefore,
( I ) ≤ N 3 / 2 ( r N N + c δ ) . (43) \mathrm{(I)}\leq N^{3/2}(r_{NN}+c\delta). \tag*{(43)} ( I ) ≤ N 3/2 ( r N N + cδ ) . (43)
For Part (II),
rank ( Y ^ δ − a N N 1 N 1 N T ) ≤ N δ . \operatorname{rank}\left(\widehat{Y}_{\delta}-a_{NN}\mathbf{1}_{N}\mathbf{1}_{N}^{\mathsf{T}}\right)\leq N_{\delta}. rank ( Y δ − a N N 1 N 1 N T ) ≤ N δ .
Moreover,
∣ y ^ i j δ − a N N ∣ ≤ R N N + r N N , \left|\widehat{y}_{ij}^{\delta}-a_{NN}\right|\leq R_{NN}+r_{NN}, y ij δ − a N N ≤ R N N + r N N ,
and hence
( I I ) ≤ c δ − K / 2 N ( R N N + r N N ) . (44) (\mathrm{II})\leq c\delta^{-K/2}N(R_{NN}+r_{NN}). \tag*{(44)} ( II ) ≤ c δ − K /2 N ( R N N + r N N ) . (44)
Combining the preceding bounds gives
∥ E ( Y ^ ) − a N N 1 N 1 N T ∥ ∗ ≤ N 3 / 2 ( r N N + c δ ) + c N ( R N N + r N N ) δ − K / 2 + N R N N . (45) \left\|\mathbb{E}(\widehat{Y})-a_{NN}\mathbf{1}_{N}\mathbf{1}_{N}^{\mathsf{T}}\right\|_{*}\leq N^{3/2}(r_{NN}+c\delta)+cN(R_{NN}+r_{NN})\delta^{-K/2}+NR_{NN}. \tag*{(45)} E ( Y ) − a N N 1 N 1 N T ∗ ≤ N 3/2 ( r N N + cδ ) + c N ( R N N + r N N ) δ − K /2 + N R N N . (45)
Substituting this into (39) and combining with (38) gives
1 N 2 E ∥ X ^ − Y ∗ ∥ F 2 ≤ c R N N ( r N N + δ ) + c R N N ( R N N + r N N ) δ − K / 2 N + c r N N 2 + c R N N 2 N + c R N N 2 e − c N . (46) \frac{1}{N^{2}}\mathbb{E}\left\|\widehat{X}-Y^{*}\right\|_{F}^{2}\leq cR_{NN}(r_{NN}+\delta)+cR_{NN}(R_{NN}+r_{NN})\frac{\delta^{-K/2}}{\sqrt{N}}+cr_{NN}^{2}+c\frac{R_{NN}^{2}}{\sqrt{N}}+cR_{NN}^{2}e^{-cN}. \tag*{(46)} N 2 1 E X − Y ∗ F 2 ≤ c R N N ( r N N + δ ) + c R N N ( R N N + r N N ) N δ − K /2 + c r N N 2 + c N R N N 2 + c R N N 2 e − c N . (46)
Choose
δ = { ( R N N + r N N ) 2 N } 1 K + 2 . \delta=\left\{\frac{(R_{NN}+r_{NN})^{2}}{N}\right\}^{\frac{1}{K+2}}. δ = { N ( R N N + r N N ) 2 } K + 2 1 .
Under R N N + r N N = o ( N ) R_{NN}+r_{NN}=o(\sqrt{N}) R N N + r N N = o ( N ) , we obtain
1 N 2 E ∥ X ^ − Y ∗ ∥ F 2 ≤ c R N N { ( R N N + r N N ) 2 N } 1 K + 2 + c r N N ( R N N + r N N ) . (47) \frac{1}{N^{2}}\mathbb{E}\left\|\widehat{X}-Y^{*}\right\|_{F}^{2}\leq cR_{NN}\left\{\frac{(R_{NN}+r_{NN})^{2}}{N}\right\}^{\frac{1}{K+2}}+cr_{NN}(R_{NN}+r_{NN}). \tag*{(47)} N 2 1 E X − Y ∗ F 2 ≤ c R N N { N ( R N N + r N N ) 2 } K + 2 1 + c r N N ( R N N + r N N ) . (47)
Finally, since m i j ∗ ∈ [ − C 0 , C 0 ] ⊆ [ − C p r o j , C p r o j ] m_{ij}^{*}\in[-C_{0},C_{0}]\subseteq[-C_{\mathrm{proj}},C_{\mathrm{proj}}] m ij ∗ ∈ [ − C 0 , C 0 ] ⊆ [ − C proj , C proj ] , the same inverse-link argument as in the bipartite case gives
1 N 2 ∥ M ^ S V D − M ∗ ∥ F 2 = O p ( R N N { ( R N N + r N N ) 2 N } 1 K + 2 + r N N ( R N N + r N N ) ) . \frac{1}{N^2}\left\|\widehat{\mathbf{M}}^{\mathrm{SVD}}-\mathbf{M}^*\right\|_F^2=O_p\left(R_{NN}\left\{\frac{(R_{NN}+r_{NN})^2}{N}\right\}^{\frac{1}{K+2}}+r_{NN}(R_{NN}+r_{NN})\right). N 2 1 M SVD − M ∗ F 2 = O p ( R N N { N ( R N N + r N N ) 2 } K + 2 1 + r N N ( R N N + r N N ) ) .
Since J = N J=N J = N , this is the claimed bound. Together with the preceding bipartite arguments, this completes the proof of Theorem 1.
Proof of Corollary 1
Proof. By Assumption 1.1, ∣ m i j ∗ ∣ ≤ C 0 |m^*_{ij}|\le C_0 ∣ m ij ∗ ∣ ≤ C 0 uniformly over the observed entries. Since f f f is continuous, there exists a fixed constant C μ > 0 C_\mu>0 C μ > 0 such that max i , j ∣ f ( m i j ∗ ) ∣ ≤ C μ \max_{i,j}|f(m^*_{ij})|\le C_\mu max i , j ∣ f ( m ij ∗ ) ∣ ≤ C μ for bipartite data, with the maximum taken over i ≠ j i\ne j i = j in the unipartite case.
By Assumption 1.2, there exist fixed constants c 0 , C t a i l > 0 c_0,C_{\mathrm{tail}}>0 c 0 , C tail > 0 such that
Pr ( ∣ y i j − f ( m i j ∗ ) ∣ > t ) ≤ C t a i l e − c 0 t , t ≥ 0 , \Pr\left(|y_{ij}-f(m^*_{ij})|>t\right)\le C_{\mathrm{tail}}e^{-c_0t},\qquad t\ge0, Pr ( ∣ y ij − f ( m ij ∗ ) ∣ > t ) ≤ C tail e − c 0 t , t ≥ 0 ,
uniformly over all observed entries for all sufficiently large N N N and J J J . Choose a fixed constant
κ > 1 c 0 ( K + 2 ) . \kappa>\frac{1}{c_0(K+2)}. κ > c 0 ( K + 2 ) 1 .
Recall that
a N J = max { inf Y N J , − κ log J } , b N J = min { sup Y N J , κ log J } . a_{NJ}=\max\{\inf\mathcal{Y}_{NJ},-\kappa\log J\},\qquad b_{NJ}=\min\{\sup\mathcal{Y}_{NJ},\kappa\log J\}. a N J = max { inf Y N J , − κ log J } , b N J = min { sup Y N J , κ log J } .
Since inf Y N J ≤ E ( y i j ) ≤ sup Y N J \inf\mathcal{Y}_{NJ}\le\mathbb{E}(y_{ij})\le\sup\mathcal{Y}_{NJ} inf Y N J ≤ E ( y ij ) ≤ sup Y N J and ∣ E ( y i j ) ∣ = ∣ f ( m i j ∗ ) ∣ ≤ C μ |\mathbb{E}(y_{ij})|=|f(m^*_{ij})|\le C_\mu ∣ E ( y ij ) ∣ = ∣ f ( m ij ∗ ) ∣ ≤ C μ , the condition inf Y N J < sup Y N J \inf\mathcal{Y}_{NJ}<\sup\mathcal{Y}_{NJ} inf Y N J < sup Y N J implies, for all sufficiently large J J J ,
− κ log J ≤ a N J < b N J ≤ κ log J . -\kappa\log J\le a_{NJ}<b_{NJ}\le\kappa\log J. − κ log J ≤ a N J < b N J ≤ κ log J .
Hence,
R N J = b N J − a N J = O ( log J ) . (48) R_{NJ}=b_{NJ}-a_{NJ}=O(\log J). \tag*{(48)} R N J = b N J − a N J = O ( log J ) . (48)
We next bound the truncation bias. For every observed entry,
∣ E { P [ a N J , b N J ] ( y i j ) − f ( m i j ∗ ) } ∣ = ∣ E { P [ a N J , b N J ] ( y i j ) − y i j } ∣ ≤ E { a N J − y i j } + + E { y i j − b N J } + . \begin{aligned}
\left|\mathbb{E}\left\{\mathbb{P}_{[a_{NJ},b_{NJ}]}(y_{ij})-f(m_{ij}^{*})\right\}\right|
&=\left|\mathbb{E}\left\{\mathbb{P}_{[a_{NJ},b_{NJ}]}(y_{ij})-y_{ij}\right\}\right|\\
&\leq\mathbb{E}\{a_{NJ}-y_{ij}\}_{+}+\mathbb{E}\{y_{ij}-b_{NJ}\}_{+}.
\end{aligned} E { P [ a N J , b N J ] ( y ij ) − f ( m ij ∗ ) } = E { P [ a N J , b N J ] ( y ij ) − y ij } ≤ E { a N J − y ij } + + E { y ij − b N J } + .
Consider first the lower-tail term. If a N J = inf Y N J a_{NJ}=\inf\mathcal{Y}_{NJ} a N J = inf Y N J , then y i j ≥ a N J y_{ij}\geq a_{NJ} y ij ≥ a N J almost surely for every observed entry and therefore E { a N J − y i j } + = 0 \mathbb{E}\{a_{NJ}-y_{ij}\}_{+}=0 E { a N J − y ij } + = 0 . Otherwise, a N J = − κ log J a_{NJ}=-\kappa\log J a N J = − κ log J . For all sufficiently large J J J ,
f ( m i j ∗ ) − a N J = f ( m i j ∗ ) + κ log J ≥ κ log J − C μ . f(m_{ij}^{*})-a_{NJ}=f(m_{ij}^{*})+\kappa\log J\geq\kappa\log J-C_{\mu}. f ( m ij ∗ ) − a N J = f ( m ij ∗ ) + κ log J ≥ κ log J − C μ .
By tail integration,
E { a N J − y i j } + = ∫ 0 ∞ Pr ( y i j < a N J − s ) d s ≤ ∫ 0 ∞ C t a i l exp [ − c 0 { f ( m i j ∗ ) − a N J + s } ] d s ≲ J − c 0 κ . \begin{aligned}
\mathbb{E}\{a_{NJ}-y_{ij}\}_{+}
&=\int_{0}^{\infty}\Pr(y_{ij}<a_{NJ}-s)\,ds\\
&\leq\int_{0}^{\infty}C_{\mathrm{tail}}\exp\left[-c_{0}\{f(m_{ij}^{*})-a_{NJ}+s\}\right]\,ds\\
&\lesssim J^{-c_{0}\kappa}.
\end{aligned} E { a N J − y ij } + = ∫ 0 ∞ Pr ( y ij < a N J − s ) d s ≤ ∫ 0 ∞ C tail exp [ − c 0 { f ( m ij ∗ ) − a N J + s } ] d s ≲ J − c 0 κ .
Similarly, if b N J = sup Y N J b_{NJ}=\sup\mathcal{Y}_{NJ} b N J = sup Y N J , then E { y i j − b N J } + = 0 \mathbb{E}\{y_{ij}-b_{NJ}\}_{+}=0 E { y ij − b N J } + = 0 . Otherwise, b N J = κ log J b_{NJ}=\kappa\log J b N J = κ log J , and for all sufficiently large J J J , b N J − f ( m i j ∗ ) ≥ κ log J − C μ b_{NJ}-f(m^*_{ij}) \ge\kappa\log J-C_\mu b N J − f ( m ij ∗ ) ≥ κ log J − C μ . Thus,
E { y i j − b N J } + = ∫ 0 ∞ Pr ( y i j > b N J + s ) d s ≤ ∫ 0 ∞ C t a i l exp [ − c 0 { b N J + s − f ( m i j ∗ ) } ] d s ≲ J − c 0 κ . \mathbb{E}\{y_{ij}-b_{NJ}\}_+ = \int_0^\infty\Pr(y_{ij}>b_{NJ}+s)\,ds
\le\int_0^\infty C_{\mathrm{tail}}\exp\left[-c_0\{b_{NJ}+s-f(m^*_{ij})\}\right]\,ds
\lesssim J^{-c_0\kappa}. E { y ij − b N J } + = ∫ 0 ∞ Pr ( y ij > b N J + s ) d s ≤ ∫ 0 ∞ C tail exp [ − c 0 { b N J + s − f ( m ij ∗ )} ] d s ≲ J − c 0 κ .
Consequently,
r N J = O ( J − c 0 κ ) . (49) r_{NJ}=O\left(J^{-c_0\kappa}\right). \tag*{(49)} r N J = O ( J − c 0 κ ) . (49)
Since R N J + r N J = O ( log J ) = o ( J ) R_{NJ}+r_{NJ}=O(\log J)=o(\sqrt{J}) R N J + r N J = O ( log J ) = o ( J ) , the nonlinear-link part of Theorem 1 applies. Moreover,
R N J { ( R N J + r N J ) 2 J } 1 K + 2 = O [ ( log J ) { ( log J ) 2 J } 1 K + 2 ] = O ( J − 1 K + 2 ( log J ) K + 4 K + 2 ) . R_{NJ}\left\{\frac{(R_{NJ}+r_{NJ})^2}{J}\right\}^{\frac{1}{K+2}}
=O\left[(\log J)\left\{\frac{(\log J)^2}{J}\right\}^{\frac{1}{K+2}}\right]
=O\left(J^{-\frac{1}{K+2}}(\log J)^{\frac{K+4}{K+2}}\right). R N J { J ( R N J + r N J ) 2 } K + 2 1 = O [ ( log J ) { J ( log J ) 2 } K + 2 1 ] = O ( J − K + 2 1 ( log J ) K + 2 K + 4 ) .
Also, since c 0 κ > 1 K + 2 c_0\kappa>\frac{1}{K+2} c 0 κ > K + 2 1 , we have
r N J ( R N J + r N J ) = O ( J − c 0 κ log J ) = O ( J − 1 K + 2 ( log J ) K + 4 K + 2 ) . r_{NJ}(R_{NJ}+r_{NJ})=O\left(J^{-c_0\kappa}\log J\right)
=O\left(J^{-\frac{1}{K+2}}(\log J)^{\frac{K+4}{K+2}}\right). r N J ( R N J + r N J ) = O ( J − c 0 κ log J ) = O ( J − K + 2 1 ( log J ) K + 2 K + 4 ) .
Therefore,
1 N J ∥ M ^ S V D − M ∗ ∥ F 2 = O p ( J − 1 K + 2 ( log J ) K + 4 K + 2 ) . \frac{1}{NJ}\left\|\widehat{\mathbf{M}}^{\mathrm{SVD}}-\mathbf{M}^*\right\|_F^2
=O_p\left(J^{-\frac{1}{K+2}}(\log J)^{\frac{K+4}{K+2}}\right). N J 1 M SVD − M ∗ F 2 = O p ( J − K + 2 1 ( log J ) K + 2 K + 4 ) .
We next consider the uniformly bounded-support case. Suppose sup N , J sup y ∈ Y N J ∣ y ∣ < ∞ \sup_{N,J}\sup_{y\in\mathcal{Y}_{NJ}}|y|<\infty sup N , J sup y ∈ Y N J ∣ y ∣ < ∞ . Then there exists a fixed constant B > 0 B>0 B > 0 such that Y N J ⊆ [ − B , B ] \mathcal{Y}_{NJ}\subseteq[-B,B] Y N J ⊆ [ − B , B ] for all N , J N,J N , J . Since κ log J > B \kappa\log J>B κ log J > B for all sufficiently large J J J ,
a N J = inf Y N J , b N J = sup Y N J . a_{NJ}=\inf\mathcal{Y}_{NJ},\qquad b_{NJ}=\sup\mathcal{Y}_{NJ}. a N J = inf Y N J , b N J = sup Y N J .
Hence P [ a N J , b N J ] ( y i j ) = y i j \mathcal{P}_{[a_{NJ},b_{NJ}]}(y_{ij})=y_{ij} P [ a N J , b N J ] ( y ij ) = y ij almost surely for every observed entry, and therefore r N J = 0 r_{NJ}=0 r N J = 0 . Moreover, R N J = b N J − a N J ≤ 2 B = O ( 1 ) R_{NJ}=b_{NJ}-a_{NJ}\le2B=O(1) R N J = b N J − a N J ≤ 2 B = O ( 1 ) . Applying Theorem 1 again gives
1 N J ∥ M ^ S V D − M ∗ ∥ F 2 = O p ( J − 1 K + 2 ) . \frac{1}{NJ}\left\|\widehat{\mathbf{M}}^{\mathrm{SVD}}-\mathbf{M}^{*}\right\|_{F}^{2}=O_{p}\left(J^{-\frac{1}{K+2}}\right). N J 1 M SVD − M ∗ F 2 = O p ( J − K + 2 1 ) .
For unipartite data, the same argument applies to the off-diagonal observations with J = N J=N J = N . The additional condition f ( τ 0 ) ∈ [ a N N , b N N ] f(\tau_{0})\in[a_{NN},b_{NN}] f ( τ 0 ) ∈ [ a N N , b N N ] for all sufficiently large N N N is precisely the condition required in the unipartite part of Theorem 1. This completes the proof.
Proofs for One-Step Refinement
Throughout Appendix C, f f f is assumed to be nonlinear, as in Theorem 2. Let C 0 > 0 C_{0}>0 C 0 > 0 denote the fixed population bound in Assumption 1.1, so that ∥ M ∗ ∥ max ≤ C 0 \|\mathbf{M}^{*}\|_{\max}\le C_{0} ∥ M ∗ ∥ m a x ≤ C 0 , and let C p r o j C_{\mathrm{proj}} C proj denote the projection radius used in Algorithm 1, chosen so that C p r o j ≥ C 0 C_{\mathrm{proj}}\ge C_{0} C proj ≥ C 0 .
Preliminary Factor Perturbation
Throughout the proof of Theorem 2, we apply the refinement step of Algorithm 1 with M i n i t \mathbf{M}^{\mathrm{init}} M init in place of M ^ S V D \widehat{\mathbf{M}}^{\mathrm{SVD}} M SVD , while retaining the notation used therein. In particular, V ^ \widehat{\mathbf{V}} V denotes the matrix of the leading K + 2 K+2 K + 2 right singular vectors of M i n i t \mathbf{M}^{\mathrm{init}} M init , and A ^ \widehat{\mathbf{A}} A , B ^ \widehat{\mathbf{B}} B , and A ^ \widehat{\mathbf{A}} A are constructed accordingly as in Algorithm 1.
Recall that M ∗ = U ∗ D ∗ V ∗ ⊤ \mathbf{M}^{*}=\mathbf{U}^{*}\mathbf{D}^{*}\mathbf{V}^{*\top} M ∗ = U ∗ D ∗ V ∗ ⊤ , with the rescaled SVD factors J V ∗ \sqrt{J}\mathbf{V}^{*} J V ∗ and J − 1 / 2 U ∗ D ∗ J^{-1/2}\mathbf{U}^{*}\mathbf{D}^{*} J − 1/2 U ∗ D ∗ . To account for the rotational non-identifiability of the factorization, let
P ^ ∈ argmin P ∈ O K + 2 ∥ V ^ − V ∗ P ∥ F . \widehat{P}\in\underset{P\in\mathcal{O}_{K+2}}{\operatorname{argmin}}\left\|\widehat{V}-V^{*}P\right\|_{F}. P ∈ P ∈ O K + 2 argmin V − V ∗ P F .
For notational simplicity, throughout Appendix C we use A ∗ \mathbf{A}^{*} A ∗ and B ∗ \mathbf{B}^{*} B ∗ to denote the corresponding aligned true factors,
A ∗ : = J V ∗ P ^ , B ∗ : = J − 1 / 2 U ∗ D ∗ P ^ . \mathbf A^* := \sqrt{J}\,\mathbf V^*\widehat{\mathbf P}, \qquad\mathbf B^* := J^{-1/2}\mathbf U^*\mathbf D^*\widehat{\mathbf P}. A ∗ := J V ∗ P , B ∗ := J − 1/2 U ∗ D ∗ P .
Since P ^ \widehat{\mathbf P} P is orthogonal, M ∗ = B ∗ A ∗ ⊤ \mathbf M^* = \mathbf B^*\mathbf A^{*\top} M ∗ = B ∗ A ∗ ⊤ .
Thus, all factor estimation errors below are measured relative to this optimally aligned representation. By Assumptions 1.1 and 1.4, together with the orthogonality of P ^ \widehat{\mathbf P} P , there exist fixed constants C ‾ 1 , C ‾ 2 > 0 \overline{C}_1,\overline{C}_2 > 0 C 1 , C 2 > 0 , independent of ( N , J ) (N,J) ( N , J ) , such that
∥ B ∗ ∥ 2 → ∞ ≤ C ‾ 1 , ∥ A ∗ ∥ 2 → ∞ ≤ C ‾ 2 . \lVert\mathbf B^* \rVert_{2\to\infty} \le\overline{C}_1, \qquad\lVert\mathbf A^* \rVert_{2\to\infty} \le\overline{C}_2. ∥ B ∗ ∥ 2 → ∞ ≤ C 1 , ∥ A ∗ ∥ 2 → ∞ ≤ C 2 .
Fix an arbitrary constant η B > 0 \eta_B > 0 η B > 0 and set C 1 : = C ‾ 1 + η B C_1 := \overline{C}_1 + \eta_B C 1 := C 1 + η B . Then ∥ B ∗ ∥ 2 → ∞ ≤ C ‾ 1 = C 1 − η B \lVert\mathbf B^* \rVert_{2\to\infty} \le\overline{C}_1 = C_1 - \eta_B ∥ B ∗ ∥ 2 → ∞ ≤ C 1 = C 1 − η B .
We take C p r o j C_{\mathrm{proj}} C proj sufficiently large so that C p r o j ≥ C 0 ∨ C ‾ 2 C_{\mathrm{proj}} \ge C_0 \vee\overline{C}_2 C proj ≥ C 0 ∨ C 2 , and set C 2 : = C p r o j C_2 := C_{\mathrm{proj}} C 2 := C proj . Then
∥ A ∗ ∥ 2 → ∞ ≤ C 2 , ∥ A ^ ∥ 2 → ∞ ≤ C 2 . \lVert\mathbf A^* \rVert_{2\to\infty} \le C_2, \qquad\lVert\widehat{\mathbf A} \rVert_{2\to\infty} \le C_2. ∥ A ∗ ∥ 2 → ∞ ≤ C 2 , ∥ A ∥ 2 → ∞ ≤ C 2 .
We first establish the following lemma.
Lemma 1. Suppose that ∥ M i n i t − M ∗ ∥ F ≤ 1 2 σ K + 2 ( M ∗ ) \lVert\mathbf M^{\mathrm{init}}-\mathbf M^*\rVert_F \le\frac{1}{2}\sigma_{K+2}(\mathbf M^*) ∥ M init − M ∗ ∥ F ≤ 2 1 σ K + 2 ( M ∗ ) . Then
∥ A ^ − A ∗ ∥ F ≤ 8 J σ K + 2 − 1 ( M ∗ ) ∥ M i n i t − M ∗ ∥ F . \lVert\widehat{\mathbf A}-\mathbf A^*\rVert_F \le8\sqrt{J}\,\sigma_{K+2}^{-1}(\mathbf M^*)\lVert\mathbf M^{\mathrm{init}}-\mathbf M^*\rVert_F. ∥ A − A ∗ ∥ F ≤ 8 J σ K + 2 − 1 ( M ∗ ) ∥ M init − M ∗ ∥ F .
Proof. Let Δ M : = M i n i t − M ∗ \Delta_{\mathbf M} := \mathbf M^{\mathrm{init}}-\mathbf M^* Δ M := M init − M ∗ . By Weyl’s inequality,
σ K + 2 ( M i n i t ) ≥ σ K + 2 ( M ∗ ) − ∥ Δ M ∥ 2 ≥ σ K + 2 ( M ∗ ) − ∥ Δ M ∥ F ≥ 1 2 σ K + 2 ( M ∗ ) . (50) \begin{aligned}
\sigma_{K+2}(\mathbf M^{\mathrm{init}}) &\ge\sigma_{K+2}(\mathbf M^*)-\lVert\Delta_{\mathbf M}\rVert_2 \\
&\ge\sigma_{K+2}(\mathbf M^*)-\lVert\Delta_{\mathbf M}\rVert_F \\
&\ge\frac{1}{2}\sigma_{K+2}(\mathbf M^*). \tag*{(50)}
\end{aligned} σ K + 2 ( M init ) ≥ σ K + 2 ( M ∗ ) − ∥ Δ M ∥ 2 ≥ σ K + 2 ( M ∗ ) − ∥ Δ M ∥ F ≥ 2 1 σ K + 2 ( M ∗ ) . (50)
Since M ∗ M^* M ∗ has rank K + 2 K+2 K + 2 , Wedin’s sin Θ \Theta Θ theorem gives
inf P ∈ O K + 2 ∥ V ^ − V ∗ P ∥ F ≤ 2 ∥ M i n i t − M ∗ ∥ F σ K + 2 ( M i n i t ) . \inf_{P\in\mathcal{O}_{K+2}}\|\widehat{V}-V^*P\|_F \leq\frac{2\|M^{\mathrm{init}}-M^*\|_F}{\sigma_{K+2}(M^{\mathrm{init}})}. P ∈ O K + 2 inf ∥ V − V ∗ P ∥ F ≤ σ K + 2 ( M init ) 2∥ M init − M ∗ ∥ F .
By the definition of P ^ \widehat{P} P ,
∥ V ^ − V ∗ P ^ ∥ F ≤ 4 ∥ M i n i t − M ∗ ∥ F σ K + 2 ( M ∗ ) . \|\widehat{V}-V^*\widehat{P}\|_F \leq\frac{4\|M^{\mathrm{init}}-M^*\|_F}{\sigma_{K+2}(M^*)}. ∥ V − V ∗ P ∥ F ≤ σ K + 2 ( M ∗ ) 4∥ M init − M ∗ ∥ F .
Since A ∗ = J V ∗ P ^ A^*=\sqrt{J}V^*\widehat{P} A ∗ = J V ∗ P , it follows that
∥ J V ^ − A ∗ ∥ F ≤ 4 J σ K + 2 − 1 ( M ∗ ) ∥ M i n i t − M ∗ ∥ F . \|\sqrt{J}\widehat{V}-A^*\|_F \leq4\sqrt{J}\sigma_{K+2}^{-1}(M^*)\|M^{\mathrm{init}}-M^*\|_F. ∥ J V − A ∗ ∥ F ≤ 4 J σ K + 2 − 1 ( M ∗ ) ∥ M init − M ∗ ∥ F .
Let
C A : = { A ∈ R J × ( K + 2 ) : ∥ A ∥ 2 → ∞ ≤ C p r o j } . \mathcal{C}_A:=\left\{A\in\mathbb{R}^{J\times(K+2)}:\|A\|_{2\to\infty}\leq C_{\mathrm{proj}}\right\}. C A := { A ∈ R J × ( K + 2 ) : ∥ A ∥ 2 → ∞ ≤ C proj } .
By the choice of C p r o j C_{\mathrm{proj}} C proj above, A ∗ ∈ C A A^*\in\mathcal{C}_A A ∗ ∈ C A . Since A ^ = P C p r o j , 2 → ∞ ( J V ^ ) \widehat{A}=\mathcal{P}_{C_{\mathrm{proj},2\to\infty}}(\sqrt{J}\widehat{V}) A = P C proj , 2 → ∞ ( J V ) , the definition of the projection yields
∥ A ^ − J V ^ ∥ F ≤ ∥ A ∗ − J V ^ ∥ F . \|\widehat{A}-\sqrt{J}\widehat{V}\|_F\leq\|A^*-\sqrt{J}\widehat{V}\|_F. ∥ A − J V ∥ F ≤ ∥ A ∗ − J V ∥ F .
Therefore,
∥ A ^ − A ∗ ∥ F ≤ ∥ A ^ − J V ^ ∥ F + ∥ J V ^ − A ∗ ∥ F ≤ 2 ∥ J V ^ − A ∗ ∥ F ≤ 8 J σ K + 2 − 1 ( M ∗ ) ∥ M i n i t − M ∗ ∥ F . \begin{aligned}
\|\widehat{A}-A^*\|_F
&\leq\|\widehat{A}-\sqrt{J}\widehat{V}\|_F+\|\sqrt{J}\widehat{V}-A^*\|_F\\
&\leq2\|\sqrt{J}\widehat{V}-A^*\|_F\\
&\leq8\sqrt{J}\sigma_{K+2}^{-1}(M^*)\|M^{\mathrm{init}}-M^*\|_F.
\end{aligned} ∥ A − A ∗ ∥ F ≤ ∥ A − J V ∥ F + ∥ J V − A ∗ ∥ F ≤ 2∥ J V − A ∗ ∥ F ≤ 8 J σ K + 2 − 1 ( M ∗ ) ∥ M init − M ∗ ∥ F .
The remainder of the proof is organized by data structure. Section C.2 develops the bipartite refinement analysis, including the deterministic estimating-equation bounds, the required stochastic controls, and the resulting two-to-infinity factor bounds. Section C.3 then treats the unipartite case by accounting for diagonal exclusion, symmetry, and the final symmetrization and rank-( K + 2 ) (K+2) ( K + 2 ) truncation. Section C.4 proves Corollary 2.
Bipartite One-Step Refinement
We first establish deterministic properties of the row- and column-wise estimating equations, then derive the required stochastic bounds and two-to-infinity perturbation controls, and finally complete the proof of Theorem 2 for bipartite data.
Deterministic Bounds for Solutions to the Estimating Equations
For ℓ ∈ { 1 , 2 } \ell\in\{1,2\} ℓ ∈ { 1 , 2 } , define
κ ℓ ( x ) : = sup ∣ m ∣ ≤ x ∣ f ( ℓ ) ( m ) ∣ , δ 1 ( x ) : = inf ∣ m ∣ ≤ x f ′ ( m ) . \kappa_{\ell}(x) := \sup_{|m|\leq x} |f^{(\ell)}(m)|,\qquad\delta_1(x) := \inf_{|m|\leq x} f'(m). κ ℓ ( x ) := ∣ m ∣ ≤ x sup ∣ f ( ℓ ) ( m ) ∣ , δ 1 ( x ) := ∣ m ∣ ≤ x inf f ′ ( m ) .
Define
κ 1 ∗ : = κ 1 ( C 0 ) , κ 2 ∗ : = κ 2 ( 2 C 1 C 2 ) , δ 1 ∗ : = δ 1 ( C 0 ) . \kappa_1^* := \kappa_1(C_0),\qquad\kappa_2^* := \kappa_2(2C_1C_2),\qquad\delta_1^* := \delta_1(C_0). κ 1 ∗ := κ 1 ( C 0 ) , κ 2 ∗ := κ 2 ( 2 C 1 C 2 ) , δ 1 ∗ := δ 1 ( C 0 ) .
By Assumption 1.3, κ 1 ∗ , κ 2 ∗ < ∞ \kappa_1^*,\kappa_2^* < \infty κ 1 ∗ , κ 2 ∗ < ∞ and δ 1 ∗ > 0 \delta_1^* > 0 δ 1 ∗ > 0 . For each i ∈ [ N ] i \in[N] i ∈ [ N ] , define
S 1 , i ( β i ; A ) : = ∑ j = 1 J { y i j − f ( α j ⊤ β i ) } α j . S_{1,i}(\beta_i;\mathcal{A}) := \sum_{j=1}^{J}\{y_{ij}-f(\alpha_j^{\top}\beta_i)\}\alpha_j. S 1 , i ( β i ; A ) := j = 1 ∑ J { y ij − f ( α j ⊤ β i )} α j .
The next lemma gives a deterministic bound for a local solution to S 1 , i ( β i ; A ) = 0 K + 2 S_{1,i}(\beta_i;\mathcal{A}) = \mathbf{0}_{K+2} S 1 , i ( β i ; A ) = 0 K + 2 .
Lemma 2. Let B ∗ ∈ R N × ( K + 2 ) \mathbf{B}^* \in\mathbb{R}^{N\times(K+2)} B ∗ ∈ R N × ( K + 2 ) and A ∗ ∈ R J × ( K + 2 ) \mathbf{A}^* \in\mathbb{R}^{J\times(K+2)} A ∗ ∈ R J × ( K + 2 ) be such that M ∗ = B ∗ A ∗ ⊤ \mathbf{M}^* = \mathbf{B}^*\mathbf{A}^{*\top} M ∗ = B ∗ A ∗ ⊤ , and let E = ( E i j ) N × J \mathcal{E} = (\mathcal{E}_{ij})_{N\times J} E = ( E ij ) N × J with E i j = y i j − f ( m i j ∗ ) \mathcal{E}_{ij} = y_{ij} - f(m^*_{ij}) E ij = y ij − f ( m ij ∗ ) . Suppose Assumption 1.3 holds, ∥ M ∗ ∥ max ≤ C 0 \|\mathbf{M}^*\|_{\max} \leq C_0 ∥ M ∗ ∥ m a x ≤ C 0 , and rank ( A ) = K + 2 \operatorname{rank}(\mathcal{A}) = K+2 rank ( A ) = K + 2 . If ∥ B ∗ ∥ 2 → ∞ ≤ C 1 \|\mathbf{B}^*\|_{2\to\infty} \leq C_1 ∥ B ∗ ∥ 2 → ∞ ≤ C 1 , ∥ A ∗ ∥ 2 → ∞ , ∥ A ∥ 2 → ∞ ≤ C 2 \|\mathbf{A}^*\|_{2\to\infty},\|\mathcal{A}\|_{2\to\infty} \leq C_2 ∥ A ∗ ∥ 2 → ∞ , ∥ A ∥ 2 → ∞ ≤ C 2 , and there exists ξ > 0 \xi> 0 ξ > 0 such that
2 σ K + 2 − 1 ( I 1 , i ( A ) ) { ∥ E i ⋅ A ∥ + ∥ B 1 , i ( A ) ∥ + β 1 , i ( A ) κ 2 ( C 2 ( C 1 + ξ ) ) } ≤ ξ , 2\sigma^{-1}_{K+2}(\mathcal{I}_{1,i}(\mathcal{A}))\left\{\|\mathcal{E}_{i\cdot}\mathcal{A}\|+\|\mathbf{B}_{1,i}(\mathcal{A})\|+\beta_{1,i}(\mathcal{A})\kappa_2(C_2(C_1+\xi))\right\}\leq\xi, 2 σ K + 2 − 1 ( I 1 , i ( A )) { ∥ E i ⋅ A ∥ + ∥ B 1 , i ( A ) ∥ + β 1 , i ( A ) κ 2 ( C 2 ( C 1 + ξ )) } ≤ ξ , 2 γ 1 , i ( A ) κ 2 ( C 2 ( C 1 + ξ ) ) ξ ≤ σ K + 2 ( I 1 , i ( A ) ) . (51) 2\gamma_{1,i}(\mathcal{A})\kappa_2(C_2(C_1+\xi))\xi\leq\sigma_{K+2}(\mathcal{I}_{1,i}(\mathcal{A})). \tag*{(51)} 2 γ 1 , i ( A ) κ 2 ( C 2 ( C 1 + ξ )) ξ ≤ σ K + 2 ( I 1 , i ( A )) . (51)
where we define E i = ( E i j ) j ∈ [ J ] ∈ R 1 × J \mathcal{E}_i=(\mathcal{E}_{ij})_{j\in[J]}\in\mathbb{R}^{1\times J} E i = ( E ij ) j ∈ [ J ] ∈ R 1 × J ,
B 1 , i ( A ) : = ∑ j = 1 J f ′ ( m i j ∗ ) α j ( α j − α j ∗ ) T β i ∗ ∈ R K + 2 , (52) \mathcal{B}_{1,i}(\mathcal{A}):=\sum_{j=1}^{J}f'(m_{ij}^{*})\alpha_j(\alpha_j-\alpha_j^{*})^{\mathsf{T}}\beta_i^{*}\in\mathbb{R}^{K+2}, \tag*{(52)} B 1 , i ( A ) := j = 1 ∑ J f ′ ( m ij ∗ ) α j ( α j − α j ∗ ) T β i ∗ ∈ R K + 2 , (52)
L 1 , i ( A ) : = ∑ j = 1 J f ′ ( m i j ∗ ) α j α j T ∈ R ( K + 2 ) × ( K + 2 ) , (53) \mathcal{L}_{1,i}(\mathcal{A}):=\sum_{j=1}^{J}f'(m_{ij}^{*})\alpha_j\alpha_j^{\mathsf{T}}\in\mathbb{R}^{(K+2)\times(K+2)}, \tag*{(53)} L 1 , i ( A ) := j = 1 ∑ J f ′ ( m ij ∗ ) α j α j T ∈ R ( K + 2 ) × ( K + 2 ) , (53)
and
β 1 , i ( A ) : = sup ∥ u ∥ = 1 ∑ j = 1 J ( ( α j − α j ∗ ) T β i ∗ ) 2 ∣ α j T u ∣ , γ 1 , i ( A ) : = sup ∥ u ∥ = 1 ∑ j = 1 J ∣ α j T u ∣ 3 . (54) \beta_{1,i}(\mathcal{A}):=\sup_{\lVert u\rVert=1}\sum_{j=1}^{J}\left((\alpha_j-\alpha_j^{*})^{\mathsf{T}}\beta_i^{*}\right)^2\lvert\alpha_j^{\mathsf{T}}u\rvert,\qquad\gamma_{1,i}(\mathcal{A}):=\sup_{\lVert u\rVert=1}\sum_{j=1}^{J}\lvert\alpha_j^{\mathsf{T}}u\rvert^3. \tag*{(54)} β 1 , i ( A ) := ∥ u ∥ = 1 sup j = 1 ∑ J ( ( α j − α j ∗ ) T β i ∗ ) 2 ∣ α j T u ∣ , γ 1 , i ( A ) := ∥ u ∥ = 1 sup j = 1 ∑ J ∣ α j T u ∣ 3 . (54)
Then there exists a unique β ~ i \widetilde{\beta}_i β i such that ∥ β ~ i − β i ∗ ∥ ≤ ξ \lVert\widetilde{\beta}_i-\beta_i^{*}\rVert\leq\xi ∥ β i − β i ∗ ∥ ≤ ξ and S 1 , i ( β ~ i ; A ) = 0 K + 2 S_{1,i}(\widetilde{\beta}_i;\mathcal{A})=\mathbf{0}_{K+2} S 1 , i ( β i ; A ) = 0 K + 2 .
Proof. The proof is divided into two parts: existence and uniqueness.
Existence. For β i ∈ R K + 2 \beta_i\in\mathbb{R}^{K+2} β i ∈ R K + 2 , let m i j : = α j T β i m_{ij}:=\alpha_j^{\mathsf{T}}\beta_i m ij := α j T β i . Consider any β i \beta_i β i satisfying ∥ β i − β i ∗ ∥ ≤ ξ \lVert\beta_i-\beta_i^{*}\rVert\leq\xi ∥ β i − β i ∗ ∥ ≤ ξ . By ∥ B ∗ ∥ 2 → ∞ ≤ C 1 \lVert\mathcal{B}^{*}\rVert_{2\to\infty}\leq C_1 ∥ B ∗ ∥ 2 → ∞ ≤ C 1 and ∥ A ∗ ∥ 2 → ∞ , ∥ A ∥ 2 → ∞ ≤ C 2 \lVert\mathcal{A}^{*}\rVert_{2\to\infty},\lVert\mathcal{A}\rVert_{2\to\infty}\leq C_2 ∥ A ∗ ∥ 2 → ∞ , ∥ A ∥ 2 → ∞ ≤ C 2 , we have
∣ m i j ∗ ∣ = ∣ α j ∗ T β i ∗ ∣ ≤ C 1 C 2 ≤ C 2 ( C 1 + ξ ) , \lvert m_{ij}^{*}\rvert=\lvert\alpha_j^{*\mathsf{T}}\beta_i^{*}\rvert\leq C_1C_2\leq C_2(C_1+\xi), ∣ m ij ∗ ∣ = ∣ α j ∗ T β i ∗ ∣ ≤ C 1 C 2 ≤ C 2 ( C 1 + ξ ) ,
and
∣ m i j ∣ = ∣ α j T β i ∣ ≤ ∥ α j ∥ ∥ β i ∥ ≤ C 2 ( C 1 + ξ ) . \lvert m_{ij}\rvert=\lvert\alpha_j^{\mathsf{T}}\beta_i\rvert\leq\lVert\alpha_j\rVert\lVert\beta_i\rVert\leq C_2(C_1+\xi). ∣ m ij ∣ = ∣ α j T β i ∣ ≤ ∥ α j ∥ ∥ β i ∥ ≤ C 2 ( C 1 + ξ ) .
Hence, every point between m i j ∗ m_{ij}^{*} m ij ∗ and m i j m_{ij} m ij lies in [ − C 2 ( C 1 + ξ ) , C 2 ( C 1 + ξ ) ] [-C_2(C_1+\xi),C_2(C_1+\xi)] [ − C 2 ( C 1 + ξ ) , C 2 ( C 1 + ξ )] . By Assumption 1.3, f f f is twice continuously differentiable on this interval.
We write
S 1 , i ( β i ; A ) = ∑ j = 1 J { y i j − f ( m i j ∗ ) } α j − ∑ j = 1 J { f ( m i j ) − f ( m i j ∗ ) } α j . S_{1,i}(\beta_i;\mathcal{A})=\sum_{j=1}^{J}\{y_{ij}-f(m_{ij}^{*})\}\alpha_j-\sum_{j=1}^{J}\{f(m_{ij})-f(m_{ij}^{*})\}\alpha_j. S 1 , i ( β i ; A ) = j = 1 ∑ J { y ij − f ( m ij ∗ )} α j − j = 1 ∑ J { f ( m ij ) − f ( m ij ∗ )} α j .
Applying a second-order Taylor expansion to f ( m i j ) f(m_{ij}) f ( m ij ) around m i j ∗ m_{ij}^{*} m ij ∗ gives
f ( m i j ) − f ( m i j ∗ ) = f ′ ( m i j ∗ ) ( m i j − m i j ∗ ) + 2 − 1 f ′ ′ ( m ~ i j ) ( m i j − m i j ∗ ) 2 , f(m_{ij})-f(m_{ij}^{*})=f'(m_{ij}^{*})(m_{ij}-m_{ij}^{*})+2^{-1}f''(\widetilde{m}_{ij})(m_{ij}-m_{ij}^{*})^2, f ( m ij ) − f ( m ij ∗ ) = f ′ ( m ij ∗ ) ( m ij − m ij ∗ ) + 2 − 1 f ′′ ( m ij ) ( m ij − m ij ∗ ) 2 ,
for some m ~ i j \widetilde{m}_{ij} m ij between m i j ∗ m_{ij}^{*} m ij ∗ and m i j m_{ij} m ij . Therefore,
S 1 , i ( β i ; A ) = A ⊤ E i ⊤ − ∑ j = 1 J f ′ ( m i j ∗ ) ( m i j − m i j ∗ ) α j − 2 − 1 ∑ j = 1 J f ′ ′ ( m ~ i j ) ( m i j − m i j ∗ ) 2 α j . (55) S_{1,i}(\beta_i;\mathcal{A})=\mathcal{A}^{\top}\mathcal{E}_i^{\top}-\sum_{j=1}^{J}f'(m_{ij}^{*})(m_{ij}-m_{ij}^{*})\alpha_j
-2^{-1}\sum_{j=1}^{J}f''(\widetilde{m}_{ij})(m_{ij}-m_{ij}^{*})^2\alpha_j. \tag*{(55)} S 1 , i ( β i ; A ) = A ⊤ E i ⊤ − j = 1 ∑ J f ′ ( m ij ∗ ) ( m ij − m ij ∗ ) α j − 2 − 1 j = 1 ∑ J f ′′ ( m ij ) ( m ij − m ij ∗ ) 2 α j . (55)
Since
m i j − m i j ∗ = α j ⊤ ( β i − β i ∗ ) + ( α j − α j ∗ ) ⊤ β i ∗ , m_{ij}-m_{ij}^{*}=\alpha_j^{\top}(\beta_i-\beta_i^{*})+(\alpha_j-\alpha_j^{*})^{\top}\beta_i^{*}, m ij − m ij ∗ = α j ⊤ ( β i − β i ∗ ) + ( α j − α j ∗ ) ⊤ β i ∗ ,
we obtain
S 1 , i ( β i ; A ) = A ⊤ E i ⊤ − ∑ j = 1 J f ′ ( m i j ∗ ) α j α j ⊤ ( β i − β i ∗ ) − ∑ j = 1 J f ′ ( m i j ∗ ) α j ( α j − α j ∗ ) ⊤ β i ∗ − 2 − 1 ∑ j = 1 J f ′ ′ ( m ~ i j ) ( m i j − m i j ∗ ) 2 α j . \begin{aligned}
S_{1,i}(\beta_i;\mathcal{A})={}&\mathcal{A}^{\top}\mathcal{E}_i^{\top}-\sum_{j=1}^{J}f'(m_{ij}^{*})\alpha_j\alpha_j^{\top}(\beta_i-\beta_i^{*})\\
&-\sum_{j=1}^{J}f'(m_{ij}^{*})\alpha_j(\alpha_j-\alpha_j^{*})^{\top}\beta_i^{*}\\
&-2^{-1}\sum_{j=1}^{J}f''(\widetilde{m}_{ij})(m_{ij}-m_{ij}^{*})^2\alpha_j.
\end{aligned} S 1 , i ( β i ; A ) = A ⊤ E i ⊤ − j = 1 ∑ J f ′ ( m ij ∗ ) α j α j ⊤ ( β i − β i ∗ ) − j = 1 ∑ J f ′ ( m ij ∗ ) α j ( α j − α j ∗ ) ⊤ β i ∗ − 2 − 1 j = 1 ∑ J f ′′ ( m ij ) ( m ij − m ij ∗ ) 2 α j .
Multiplying both sides from the left by ( β i − β i ∗ ) ⊤ (\beta_i-\beta_i^{*})^{\top} ( β i − β i ∗ ) ⊤ gives
( β i − β i ∗ ) ⊤ S 1 , i ( β i ; A ) = ( β i − β i ∗ ) ⊤ A ⊤ E i ⊤ − ( β i − β i ∗ ) ⊤ I 1 , i ( A ) ( β i − β i ∗ ) − ( β i − β i ∗ ) ⊤ B 1 , i ( A ) − 2 − 1 ( β i − β i ∗ ) ⊤ ∑ j = 1 J f ′ ′ ( m ~ i j ) ( m i j − m i j ∗ ) 2 α j . \begin{aligned}
(\beta_i-\beta_i^{*})^{\top}S_{1,i}(\beta_i;\mathcal{A})={}&(\beta_i-\beta_i^{*})^{\top}\mathcal{A}^{\top}\mathcal{E}_i^{\top}\\
&-(\beta_i-\beta_i^{*})^{\top}\mathcal{I}_{1,i}(\mathcal{A})(\beta_i-\beta_i^{*})\\
&-(\beta_i-\beta_i^{*})^{\top}\mathcal{B}_{1,i}(\mathcal{A})\\
&-2^{-1}(\beta_i-\beta_i^{*})^{\top}\sum_{j=1}^{J}f''(\widetilde{m}_{ij})(m_{ij}-m_{ij}^{*})^2\alpha_j.
\end{aligned} ( β i − β i ∗ ) ⊤ S 1 , i ( β i ; A ) = ( β i − β i ∗ ) ⊤ A ⊤ E i ⊤ − ( β i − β i ∗ ) ⊤ I 1 , i ( A ) ( β i − β i ∗ ) − ( β i − β i ∗ ) ⊤ B 1 , i ( A ) − 2 − 1 ( β i − β i ∗ ) ⊤ j = 1 ∑ J f ′′ ( m ij ) ( m ij − m ij ∗ ) 2 α j .
Now consider β i \beta_i β i satisfying ∥ β i − β i ∗ ∥ = ξ \lVert\beta_i-\beta_i^{*}\rVert=\xi ∥ β i − β i ∗ ∥ = ξ . The first term satisfies
∣ ( β i − β i ∗ ) ⊤ A ⊤ E i ⊤ ∣ ≤ ξ ∥ E i ⋅ A ∥ . (56) \left|(\beta_i-\beta_i^{*})^{\top}\mathcal{A}^{\top}\mathcal{E}_i^{\top}\right|\leq\xi\lVert\mathcal{E}_{i\cdot}\mathcal{A}\rVert. \tag*{(56)} ( β i − β i ∗ ) ⊤ A ⊤ E i ⊤ ≤ ξ ∥ E i ⋅ A ∥ . (56)
Since ∣ m i j ∗ ∣ ≤ C 0 |m_{ij}^{*}| \le C_0 ∣ m ij ∗ ∣ ≤ C 0 , Assumption 1.3 implies f ′ ( m i j ∗ ) ≥ δ 1 ∗ > 0 f'(m_{ij}^{*}) \ge\delta_1^{*} > 0 f ′ ( m ij ∗ ) ≥ δ 1 ∗ > 0 . Hence,
I 1 , i ( A ) = ∑ j = 1 J f ′ ( m i j ∗ ) α j α j T ≥ δ 1 ∗ A T A . \mathcal{I}_{1,i}(\mathcal{A}) = \sum_{j=1}^{J} f'(m_{ij}^{*})\alpha_j\alpha_j^{\mathsf{T}} \ge\delta_1^{*}\mathcal{A}^{\mathsf{T}}\mathcal{A}. I 1 , i ( A ) = j = 1 ∑ J f ′ ( m ij ∗ ) α j α j T ≥ δ 1 ∗ A T A .
Since rank ( A ) = K + 2 \operatorname{rank}(\mathcal{A}) = K + 2 rank ( A ) = K + 2 ,
σ K + 2 ( I 1 , i ( A ) ) ≥ δ 1 ∗ σ K + 2 2 ( A ) > 0. \sigma_{K+2}(\mathcal{I}_{1,i}(\mathcal{A})) \ge\delta_1^{*}\sigma_{K+2}^{2}(\mathcal{A}) > 0. σ K + 2 ( I 1 , i ( A )) ≥ δ 1 ∗ σ K + 2 2 ( A ) > 0.
Therefore,
− ( β i − β i ∗ ) T I 1 , i ( A ) ( β i − β i ∗ ) ≤ − ξ 2 σ K + 2 ( I 1 , i ( A ) ) . (57) -(\beta_i-\beta_i^{*})^{\mathsf{T}}\mathcal{I}_{1,i}(\mathcal{A})(\beta_i-\beta_i^{*}) \le-\xi^{2}\sigma_{K+2}(\mathcal{I}_{1,i}(\mathcal{A})). \tag*{(57)} − ( β i − β i ∗ ) T I 1 , i ( A ) ( β i − β i ∗ ) ≤ − ξ 2 σ K + 2 ( I 1 , i ( A )) . (57)
Moreover,
∣ ( β i − β i ∗ ) T B 1 , i ( A ) ∣ ≤ ξ ∥ B 1 , i ( A ) ∥ . (58) \left|(\beta_i-\beta_i^{*})^{\mathsf{T}}\mathbf{B}_{1,i}(\mathcal{A})\right| \le\xi\|\mathbf{B}_{1,i}(\mathcal{A})\|. \tag*{(58)} ( β i − β i ∗ ) T B 1 , i ( A ) ≤ ξ ∥ B 1 , i ( A ) ∥. (58)
For the last term, by the preceding bounds, ∣ m ~ i j ∣ ≤ C 2 ( C 1 + ξ ) |\widetilde{m}_{ij}| \le C_2(C_1+\xi) ∣ m ij ∣ ≤ C 2 ( C 1 + ξ ) . Writing β i − β i ∗ = ξ u \beta_i-\beta_i^{*}=\xi u β i − β i ∗ = ξ u with ∥ u ∥ = 1 \|u\|=1 ∥ u ∥ = 1 , we obtain
∣ 2 − 1 ( β i − β i ∗ ) T ∑ j = 1 J f ′ ′ ( m ~ i j ) ( m i j − m i j ∗ ) 2 α j ∣ ≤ 2 − 1 κ 2 ( C 2 ( C 1 + ξ ) ) ξ sup ∥ u ∥ = 1 ∑ j = 1 J { ( α j − α j ∗ ) T β i ∗ + ξ α j T u } 2 ∣ α j T u ∣ ≤ κ 2 ( C 2 ( C 1 + ξ ) ) { ξ β 1 , i ( A ) + ξ 3 γ 1 , i ( A ) } , (59) \begin{aligned}
\left|2^{-1}(\beta_i-\beta_i^{*})^{\mathsf{T}}\sum_{j=1}^{J}f''(\widetilde{m}_{ij})(m_{ij}-m_{ij}^{*})^{2}\alpha_j\right|
&\le2^{-1}\kappa_2(C_2(C_1+\xi))\xi\sup_{\|u\|=1}\sum_{j=1}^{J}\left\{(\alpha_j-\alpha_j^{*})^{\mathsf{T}}\beta_i^{*}+\xi\alpha_j^{\mathsf{T}}u\right\}^{2}|\alpha_j^{\mathsf{T}}u| \\
&\le\kappa_2(C_2(C_1+\xi))\left\{\xi\beta_{1,i}(\mathcal{A})+\xi^{3}\gamma_{1,i}(\mathcal{A})\right\}, \tag*{(59)}
\end{aligned} 2 − 1 ( β i − β i ∗ ) T j = 1 ∑ J f ′′ ( m ij ) ( m ij − m ij ∗ ) 2 α j ≤ 2 − 1 κ 2 ( C 2 ( C 1 + ξ )) ξ ∥ u ∥ = 1 sup j = 1 ∑ J { ( α j − α j ∗ ) T β i ∗ + ξ α j T u } 2 ∣ α j T u ∣ ≤ κ 2 ( C 2 ( C 1 + ξ )) { ξ β 1 , i ( A ) + ξ 3 γ 1 , i ( A ) } , (59)
where the second inequality follows from ( x + y ) 2 ≤ 2 x 2 + 2 y 2 (x+y)^2 \le2x^2+2y^2 ( x + y ) 2 ≤ 2 x 2 + 2 y 2 .
Combining (55)–(58) gives
( β i − β i ∗ ) T S 1 , i ( β i ; A ) ≤ − σ K + 2 ( I 1 , i ( A ) ) ξ 2 + γ 1 , i ( A ) κ 2 ( C 2 ( C 1 + ξ ) ) ξ 3 + { ∥ E i ⋅ A ∥ + ∥ B 1 , i ( A ) ∥ + β 1 , i ( A ) κ 2 ( C 2 ( C 1 + ξ ) ) } ξ . (60) \begin{aligned}
(\beta_i-\beta_i^{*})^{\mathsf{T}}S_{1,i}(\beta_i;\mathcal{A})
&\le-\sigma_{K+2}(\mathcal{I}_{1,i}(\mathcal{A}))\xi^{2}
+\gamma_{1,i}(\mathcal{A})\kappa_2(C_2(C_1+\xi))\xi^{3} \\
&\quad+\left\{\|\mathbf{E}_{i\cdot}\mathcal{A}\|+\|\mathbf{B}_{1,i}(\mathcal{A})\|+\beta_{1,i}(\mathcal{A})\kappa_2(C_2(C_1+\xi))\right\}\xi. \tag*{(60)}
\end{aligned} ( β i − β i ∗ ) T S 1 , i ( β i ; A ) ≤ − σ K + 2 ( I 1 , i ( A )) ξ 2 + γ 1 , i ( A ) κ 2 ( C 2 ( C 1 + ξ )) ξ 3 + { ∥ E i ⋅ A ∥ + ∥ B 1 , i ( A ) ∥ + β 1 , i ( A ) κ 2 ( C 2 ( C 1 + ξ )) } ξ . (60)
By the first inequality in (50),
{ ∥ E i , ⋅ A ∥ + ∥ B 1 , i ( A ) ∥ + β 1 , i ( A ) κ 2 ( C 2 ( C 1 + ξ ) ) } ξ ≤ 1 2 σ K + 2 ( I 1 , i ( A ) ) ξ 2 . \left\{\lVert\mathcal{E}_{i,\cdot}\mathcal{A}\rVert+\lVert\mathbf{B}_{1,i}(\mathcal{A})\rVert+\beta_{1,i}(\mathcal{A})\kappa_{2}\bigl(\mathcal{C}_{2}(\mathcal{C}_{1}+\xi)\bigr)\right\}\xi\leq\frac{1}{2}\sigma_{K+2}(\mathcal{I}_{1,i}(\mathcal{A}))\xi^{2}. { ∥ E i , ⋅ A ∥ + ∥ B 1 , i ( A )∥ + β 1 , i ( A ) κ 2 ( C 2 ( C 1 + ξ ) ) } ξ ≤ 2 1 σ K + 2 ( I 1 , i ( A )) ξ 2 .
By the second inequality in (50),
γ 1 , i ( A ) κ 2 ( C 2 ( C 1 + ξ ) ) ξ 3 ≤ 1 2 σ K + 2 ( I 1 , i ( A ) ) ξ 2 . \gamma_{1,i}(\mathcal{A})\kappa_{2}\bigl(\mathcal{C}_{2}(\mathcal{C}_{1}+\xi)\bigr)\xi^{3}\leq\frac{1}{2}\sigma_{K+2}(\mathcal{I}_{1,i}(\mathcal{A}))\xi^{2}. γ 1 , i ( A ) κ 2 ( C 2 ( C 1 + ξ ) ) ξ 3 ≤ 2 1 σ K + 2 ( I 1 , i ( A )) ξ 2 .
Substituting these two bounds into (59) yields
sup ∥ β i − β i ∗ ∥ = ξ ( β i − β i ∗ ) T S 1 , i ( β i ; A ) ≤ 0. \sup_{\lVert\beta_i-\beta_i^*\rVert=\xi}(\beta_i-\beta_i^*)^{\mathsf{T}}S_{1,i}(\beta_i;\mathcal{A})\leq0. ∥ β i − β i ∗ ∥ = ξ sup ( β i − β i ∗ ) T S 1 , i ( β i ; A ) ≤ 0.
By Result 6.3.4 of Ortega and Rheinboldt (2000), there exists at least one β ~ i \widetilde{\beta}_i β i satisfying
∥ β ~ i − β i ∗ ∥ ≤ ξ , S 1 , i ( β ~ i ; A ) = 0 K + 2 . \lVert\widetilde{\beta}_i-\beta_i^*\rVert\leq\xi,\qquad S_{1,i}(\widetilde{\beta}_i;\mathcal{A})=\mathbf{0}_{K+2}. ∥ β i − β i ∗ ∥ ≤ ξ , S 1 , i ( β i ; A ) = 0 K + 2 .
Uniqueness. Take any β ( 1 ) , β ( 2 ) ∈ R K + 2 \beta^{(1)},\beta^{(2)}\in\mathbb{R}^{K+2} β ( 1 ) , β ( 2 ) ∈ R K + 2 and let Δ : = β ( 1 ) − β ( 2 ) \Delta:=\beta^{(1)}-\beta^{(2)} Δ := β ( 1 ) − β ( 2 ) . Then
Δ T { S 1 , i ( β ( 1 ) ; A ) − S 1 , i ( β ( 2 ) ; A ) } = − ∑ j = 1 J { f ( α j T β ( 1 ) ) − f ( α j T β ( 2 ) ) } α j T Δ . \Delta^{\mathsf{T}}\left\{S_{1,i}(\beta^{(1)};\mathcal{A})-S_{1,i}(\beta^{(2)};\mathcal{A})\right\}
=
-\sum_{j=1}^{J}\left\{f(\alpha_j^{\mathsf{T}}\beta^{(1)})-f(\alpha_j^{\mathsf{T}}\beta^{(2)})\right\}\alpha_j^{\mathsf{T}}\Delta. Δ T { S 1 , i ( β ( 1 ) ; A ) − S 1 , i ( β ( 2 ) ; A ) } = − j = 1 ∑ J { f ( α j T β ( 1 ) ) − f ( α j T β ( 2 ) ) } α j T Δ.
Since Assumption 1.3 implies that f f f is strictly increasing, for every j j j ,
{ f ( α j T β ( 1 ) ) − f ( α j T β ( 2 ) ) } α j T Δ ≥ 0 , \left\{f(\alpha_j^{\mathsf{T}}\beta^{(1)})-f(\alpha_j^{\mathsf{T}}\beta^{(2)})\right\}\alpha_j^{\mathsf{T}}\Delta\geq0, { f ( α j T β ( 1 ) ) − f ( α j T β ( 2 ) ) } α j T Δ ≥ 0 ,
with strict inequality whenever α j T Δ ≠ 0 \alpha_j^{\mathsf{T}}\Delta\ne0 α j T Δ = 0 . Because rank ( A ) = K + 2 \operatorname{rank}(\mathcal{A})=K+2 rank ( A ) = K + 2 , for every Δ ≠ 0 \Delta\ne0 Δ = 0 there exists at least one j j j such that α j ⊤ Δ ≠ 0 \alpha_j^{\top}\Delta\ne0 α j ⊤ Δ = 0 . Hence,
Δ ⊤ { S 1 , i ( β ( 1 ) ; A ) − S 1 , i ( β ( 2 ) ; A ) } < 0 for every Δ ≠ 0. \Delta^{\top}\left\{S_{1,i}(\beta^{(1)};\mathcal{A})-S_{1,i}(\beta^{(2)};\mathcal{A})\right\}<0 \qquad\text{for every }\Delta\ne0. Δ ⊤ { S 1 , i ( β ( 1 ) ; A ) − S 1 , i ( β ( 2 ) ; A ) } < 0 for every Δ = 0.
Therefore, two distinct zeros of S 1 , i ( ⋅ ; A ) S_{1,i}(\cdot;\mathcal{A}) S 1 , i ( ⋅ ; A ) cannot exist, and the solution obtained above is unique.
Lemma 3. Let B ∗ ∈ R N × ( K + 2 ) \mathbf{B}^{*}\in\mathbb{R}^{N\times(K+2)} B ∗ ∈ R N × ( K + 2 ) and A ∗ ∈ R J × ( K + 2 ) \mathcal{A}^{*}\in\mathbb{R}^{J\times(K+2)} A ∗ ∈ R J × ( K + 2 ) be such that M ∗ = B ∗ A ∗ ⊤ \mathbf{M}^{*}=\mathbf{B}^{*}\mathcal{A}^{*\top} M ∗ = B ∗ A ∗ ⊤ , and let E = ( E i j ) N × J \mathcal{E}=(\mathcal{E}_{ij})_{N\times J} E = ( E ij ) N × J with E i j = y i j − f ( m i j ∗ ) \mathcal{E}_{ij}=y_{ij}-f(m_{ij}^{*}) E ij = y ij − f ( m ij ∗ ) . Suppose Assumption 1.3 holds, ∥ M ∗ ∥ max ≤ C 0 \|\mathbf{M}^{*}\|_{\max}\le C_{0} ∥ M ∗ ∥ m a x ≤ C 0 , rank ( A ) = K + 2 \operatorname{rank}(\mathcal{A})=K+2 rank ( A ) = K + 2 , ∥ B ∗ ∥ 2 → ∞ ≤ C 1 \|\mathbf{B}^{*}\|_{2\to\infty}\le C_{1} ∥ B ∗ ∥ 2 → ∞ ≤ C 1 , ∥ A ∗ ∥ 2 → ∞ , ∥ A ∥ 2 → ∞ ≤ C 2 \|\mathcal{A}^{*}\|_{2\to\infty},\|\mathcal{A}\|_{2\to\infty}\le C_{2} ∥ A ∗ ∥ 2 → ∞ , ∥ A ∥ 2 → ∞ ≤ C 2 . If
2 { ∥ E i ⋅ A ∥ + ∥ B 1 , i ( A ) ∥ + β 1 , i ( A ) κ 2 ∗ } ≤ σ K + 2 ( I 1 , i ( A ) ) C 1 , 4 γ 1 , i ( A ) κ 2 ∗ { ∥ E i ⋅ A ∥ + ∥ B 1 , i ( A ) ∥ + β 1 , i ( A ) κ 2 ∗ } ≤ σ K + 2 2 ( I 1 , i ( A ) ) , (61) \begin{aligned}
2\left\{\|\mathcal{E}_{i\cdot}\mathcal{A}\|+\|\mathbf{B}_{1,i}(\mathcal{A})\|+\beta_{1,i}(\mathcal{A})\kappa_{2}^{*}\right\}&\le\sigma_{K+2}(\mathcal{I}_{1,i}(\mathcal{A}))C_{1},\\
4\gamma_{1,i}(\mathcal{A})\kappa_{2}^{*}\left\{\|\mathcal{E}_{i\cdot}\mathcal{A}\|+\|\mathbf{B}_{1,i}(\mathcal{A})\|+\beta_{1,i}(\mathcal{A})\kappa_{2}^{*}\right\}&\le\sigma_{K+2}^{2}(\mathcal{I}_{1,i}(\mathcal{A})), \tag*{(61)}
\end{aligned} 2 { ∥ E i ⋅ A ∥ + ∥ B 1 , i ( A ) ∥ + β 1 , i ( A ) κ 2 ∗ } 4 γ 1 , i ( A ) κ 2 ∗ { ∥ E i ⋅ A ∥ + ∥ B 1 , i ( A ) ∥ + β 1 , i ( A ) κ 2 ∗ } ≤ σ K + 2 ( I 1 , i ( A )) C 1 , ≤ σ K + 2 2 ( I 1 , i ( A )) , (61) then there exists a unique β ~ i \widetilde{\beta}_{i} β i such that S 1 , i ( β ~ i ; A ) = 0 K + 2 S_{1,i}(\widetilde{\beta}_{i};\mathcal{A})=\mathbf{0}_{K+2} S 1 , i ( β i ; A ) = 0 K + 2 , and
∥ β ~ i − β i ∗ ∥ ≤ 2 σ K + 2 − 1 ( I 1 , i ( A ) ) { ∥ E i ⋅ A ∥ + ∥ B 1 , i ( A ) ∥ + β 1 , i ( A ) κ 2 ∗ } . (62) \|\widetilde{\beta}_{i}-\beta_{i}^{*}\|\le2\sigma_{K+2}^{-1}(\mathcal{I}_{1,i}(\mathcal{A}))\left\{\|\mathcal{E}_{i\cdot}\mathcal{A}\|+\|\mathbf{B}_{1,i}(\mathcal{A})\|+\beta_{1,i}(\mathcal{A})\kappa_{2}^{*}\right\}. \tag*{(62)} ∥ β i − β i ∗ ∥ ≤ 2 σ K + 2 − 1 ( I 1 , i ( A )) { ∥ E i ⋅ A ∥ + ∥ B 1 , i ( A ) ∥ + β 1 , i ( A ) κ 2 ∗ } . (62) Moreover, the solution β ~ i \widetilde{\beta}_{i} β i also satisfies ∥ β ~ i − β i ∗ ∥ ≤ C 1 \|\widetilde{\beta}_{i}-\beta_{i}^{*}\|\le C_{1} ∥ β i − β i ∗ ∥ ≤ C 1 .
Proof. Let
ξ = 2 σ K + 2 − 1 ( I 1 , i ( A ) ) { ∥ E i ⋅ A ∥ + ∥ B 1 , i ( A ) ∥ + β 1 , i ( A ) κ 2 ∗ } . \xi=2\sigma_{K+2}^{-1}(\mathcal{I}_{1,i}(\mathcal{A}))\left\{\|\mathcal{E}_{i\cdot}\mathcal{A}\|+\|\mathbf{B}_{1,i}(\mathcal{A})\|+\beta_{1,i}(\mathcal{A})\kappa_{2}^{*}\right\}. ξ = 2 σ K + 2 − 1 ( I 1 , i ( A )) { ∥ E i ⋅ A ∥ + ∥ B 1 , i ( A ) ∥ + β 1 , i ( A ) κ 2 ∗ } .
By the first inequality in (60), ξ ≤ C 1 \xi\le C_{1} ξ ≤ C 1 . Therefore, C 2 ( C 1 + ξ ) ≤ 2 C 1 C 2 C_{2}(C_{1}+\xi)\le2C_{1}C_{2} C 2 ( C 1 + ξ ) ≤ 2 C 1 C 2 . By the monotonicity of κ 2 ( ⋅ ) \kappa_{2}(\cdot) κ 2 ( ⋅ ) ,
κ 2 ( C 2 ( C 1 + ξ ) ) ≤ κ 2 ( 2 C 1 C 2 ) = κ 2 ∗ . (63) \kappa_{2}\left(C_{2}(C_{1}+\xi)\right)\le\kappa_{2}(2C_{1}C_{2})=\kappa_{2}^{*}. \tag*{(63)} κ 2 ( C 2 ( C 1 + ξ ) ) ≤ κ 2 ( 2 C 1 C 2 ) = κ 2 ∗ . (63)
Hence,
2 σ K + 2 − 1 ( I 1 , i ( A ) ) { ∥ E i ⋅ A ∥ + ∥ B 1 , i ( A ) ∥ + β 1 , i ( A ) κ 2 ( C 2 ( C 1 + ξ ) ) } ≤ ξ . (64) 2\sigma^{-1}_{K+2}(\mathcal{I}_{1,i}(\mathcal{A}))\left\{\left\|\mathcal{E}_i\cdot\mathcal{A}\right\|+\left\|\mathcal{B}_{1,i}(\mathcal{A})\right\|+\beta_{1,i}(\mathcal{A})\kappa_2\left(C_2(C_1+\xi)\right)\right\}\leq\xi. \tag*{(64)} 2 σ K + 2 − 1 ( I 1 , i ( A )) { ∥ E i ⋅ A ∥ + ∥ B 1 , i ( A ) ∥ + β 1 , i ( A ) κ 2 ( C 2 ( C 1 + ξ ) ) } ≤ ξ . (64)
Moreover, by (63) , the definition of ξ \xi ξ , and the second inequality in (61) ,
2 γ 1 , i ( A ) κ 2 ( C 2 ( C 1 + ξ ) ) ξ ≤ 2 γ 1 , i ( A ) κ 2 ∗ ξ = 4 γ 1 , i ( A ) κ 2 ∗ σ K + 2 − 1 ( I 1 , i ( A ) ) { ∥ E i ⋅ A ∥ + ∥ B 1 , i ( A ) ∥ + β 1 , i ( A ) κ 2 ∗ } ≤ σ K + 2 ( I 1 , i ( A ) ) . (65) \begin{aligned}
2\gamma_{1,i}(\mathcal{A})\kappa_2\left(C_2(C_1+\xi)\right)\xi
&\leq2\gamma_{1,i}(\mathcal{A})\kappa_2^*\xi\\
&=4\gamma_{1,i}(\mathcal{A})\kappa_2^*\sigma^{-1}_{K+2}\left(\mathcal{I}_{1,i}(\mathcal{A})\right)\left\{\left\|\mathcal{E}_i\cdot\mathcal{A}\right\|+\left\|\mathcal{B}_{1,i}(\mathcal{A})\right\|+\beta_{1,i}(\mathcal{A})\kappa_2^*\right\} \\
&\leq\sigma_{K+2}\left(\mathcal{I}_{1,i}(\mathcal{A})\right). \tag*{(65)}
\end{aligned} 2 γ 1 , i ( A ) κ 2 ( C 2 ( C 1 + ξ ) ) ξ ≤ 2 γ 1 , i ( A ) κ 2 ∗ ξ = 4 γ 1 , i ( A ) κ 2 ∗ σ K + 2 − 1 ( I 1 , i ( A ) ) { ∥ E i ⋅ A ∥ + ∥ B 1 , i ( A ) ∥ + β 1 , i ( A ) κ 2 ∗ } ≤ σ K + 2 ( I 1 , i ( A ) ) . (65)
Therefore, (64) and (65) imply that all the conditions in (51) of Lemma 2 hold with this choice of ξ \xi ξ . Applying Lemma 2, there exists a unique β ~ i \widetilde{\beta}_i β i such that
∥ β ~ i − β i ∗ ∥ ≤ ξ and S 1 , i ( β ~ i ; A ) = 0 K + 2 . \left\|\widetilde{\beta}_i-\beta_i^*\right\|\leq\xi\quad\text{and}\quad S_{1,i}(\widetilde{\beta}_i;\mathcal{A})=\mathbf{0}_{K+2}. β i − β i ∗ ≤ ξ and S 1 , i ( β i ; A ) = 0 K + 2 .
Substituting the definition of ξ \xi ξ gives (62) . Finally, since ξ ≤ C 1 \xi\leq C_1 ξ ≤ C 1 , ∥ β ~ i − β i ∗ ∥ ≤ C 1 \left\|\widetilde{\beta}_i-\beta_i^*\right\|\leq C_1 β i − β i ∗ ≤ C 1 .
Non-Asymptotic Stochastic Bounds
Lemma 4. Let E = ( E i j ) N × J \mathcal{E}=(\mathcal{E}_{ij})_{N\times J} E = ( E ij ) N × J and let E i j \mathcal{E}_{ij} E ij be independent mean-zero sub-exponential random variables with parameters ( ν i j , b i j ) (\nu_{ij},b_{ij}) ( ν ij , b ij ) . Define
ν = max i ∈ [ N ] , j ∈ [ J ] ν i j , b = max i ∈ [ N ] , j ∈ [ J ] b i j . \nu=\max_{i\in[N],\,j\in[J]}\nu_{ij},\qquad b=\max_{i\in[N],\,j\in[J]}b_{ij}. ν = i ∈ [ N ] , j ∈ [ J ] max ν ij , b = i ∈ [ N ] , j ∈ [ J ] max b ij . Then, for any δ ∈ ( 0 , 1 ) \delta\in(0,1) δ ∈ ( 0 , 1 ) , with probability at least 1 − δ 1-\delta 1 − δ ,
∥ E ∥ max ≤ max { ν 2 log 2 N J δ , 2 b log 2 N J δ } . \left\|\mathcal{E}\right\|_{\max}\leq\max\left\{\nu\sqrt{2\log\frac{2NJ}{\delta}},\,2b\log\frac{2NJ}{\delta}\right\}. ∥ E ∥ m a x ≤ max { ν 2 log δ 2 N J , 2 b log δ 2 N J } .
Proof. For each ( i , j ) (i,j) ( i , j ) and any t > 0 t>0 t > 0 , the sub-exponential tail bound gives
Pr ( ∣ E i j ∣ ≥ t ) ≤ 2 ( e − t 2 / ( 2 ν i j 2 ) ∨ e − t / ( 2 b i j ) ) ≤ 2 ( e − t 2 / ( 2 ν 2 ) ∨ e − t / ( 2 b ) ) . \Pr(|\mathcal{E}_{ij}|\ge t)\le2\left(e^{-t^2/(2\nu_{ij}^2)}\vee e^{-t/(2b_{ij})}\right)\le2\left(e^{-t^2/(2\nu^2)}\vee e^{-t/(2b)}\right). Pr ( ∣ E ij ∣ ≥ t ) ≤ 2 ( e − t 2 / ( 2 ν ij 2 ) ∨ e − t / ( 2 b ij ) ) ≤ 2 ( e − t 2 / ( 2 ν 2 ) ∨ e − t / ( 2 b ) ) .
Therefore, by the union bound,
Pr ( ∥ E ∥ max ≥ t ) = Pr ( max i ∈ [ N ] , j ∈ [ J ] ∣ E i j ∣ ≥ t ) ≤ ∑ i = 1 N ∑ j = 1 J Pr ( ∣ E i j ∣ ≥ t ) \begin{aligned}
\Pr(\|\mathcal{E}\|_{\max}\ge t)=\Pr\left(\max_{i\in[N],\,j\in[J]}|\mathcal{E}_{ij}|\ge t\right) \\
&\le\sum_{i=1}^{N}\sum_{j=1}^{J}\Pr(|\mathcal{E}_{ij}|\ge t)
\end{aligned} Pr ( ∥ E ∥ m a x ≥ t ) = Pr ( i ∈ [ N ] , j ∈ [ J ] max ∣ E ij ∣ ≥ t ) ≤ i = 1 ∑ N j = 1 ∑ J Pr ( ∣ E ij ∣ ≥ t )
≤ 2 N J ( e − t 2 / ( 2 ν 2 ) ∨ e − t / ( 2 b ) ) . \le2NJ\left(e^{-t^2/(2\nu^2)}\vee e^{-t/(2b)}\right). ≤ 2 N J ( e − t 2 / ( 2 ν 2 ) ∨ e − t / ( 2 b ) ) .
Now choose
t = max { ν 2 log 2 N J δ , 2 b log 2 N J δ } . t=\max\left\{\nu\sqrt{2\log\frac{2NJ}{\delta}},\,2b\log\frac{2NJ}{\delta}\right\}. t = max { ν 2 log δ 2 N J , 2 b log δ 2 N J } .
Then
e − t 2 / ( 2 ν 2 ) ≤ δ 2 N J , e − t / ( 2 b ) ≤ δ 2 N J , e^{-t^2/(2\nu^2)}\le\frac{\delta}{2NJ},\qquad e^{-t/(2b)}\le\frac{\delta}{2NJ}, e − t 2 / ( 2 ν 2 ) ≤ 2 N J δ , e − t / ( 2 b ) ≤ 2 N J δ ,
and hence Pr ( ∥ E ∥ max ≥ t ) ≤ δ \Pr(\|\mathcal{E}\|_{\max}\ge t)\le\delta Pr ( ∥ E ∥ m a x ≥ t ) ≤ δ .
Lemma 5 (Upper bound for max i ∈ [ N ] ∥ E i ⋅ A ^ ∥ \max_{i\in[N]}\|\mathcal{E}_{i\cdot}\widehat{A}\| max i ∈ [ N ] ∥ E i ⋅ A ∥ ). Let E = ( E i j ) N × J \mathcal{E}=(\mathcal{E}_{ij})_{N\times J} E = ( E ij ) N × J and let E i j \mathcal{E}_{ij} E ij be independent mean-zero sub-exponential random variables with parameters ( ν i j , b i j ) (\nu_{ij},b_{ij}) ( ν ij , b ij ) . Define
ν = max i ∈ [ N ] , j ∈ [ J ] ν i j , b = max i ∈ [ N ] , j ∈ [ J ] b i j , \nu=\max_{i\in[N],\,j\in[J]}\nu_{ij},\qquad b=\max_{i\in[N],\,j\in[J]}b_{ij}, ν = i ∈ [ N ] , j ∈ [ J ] max ν ij , b = i ∈ [ N ] , j ∈ [ J ] max b ij , and
L N , J = log ( 2 N 2 J ( K + 2 ) 2 ) . L_{N,J}=\log\left(2N^2J(K+2)^2\right). L N , J = log ( 2 N 2 J ( K + 2 ) 2 ) .
Assume that ∥ A ∗ ∥ 2 → ∞ ≤ C 2 \lVert\mathcal{A}^{*}\rVert_{2\to\infty}\le C_{2} ∥ A ∗ ∥ 2 → ∞ ≤ C 2 . Then, with probability at least 1 − 2 / ( N ( K + 2 ) ) 1-2/(N(K+2)) 1 − 2/ ( N ( K + 2 )) ,
max i ∈ [ N ] ∥ E i ⋅ A ^ ∥ ≤ 2 C 2 K + 2 L N , J ( J ν ∨ b ) + 2 L N , J ( ν ∨ b ) J ∥ A ^ − A ∗ ∥ F . (66) \max_{i\in[N]}\lVert\mathcal{E}_{i\cdot}\widehat{\mathcal{A}}\rVert\le2C_{2}\sqrt{K+2}\,L_{N,J}\left(\sqrt{J}\nu\vee b\right)+2L_{N,J}\left(\nu\vee b\right)\sqrt{J}\lVert\widehat{\mathcal{A}}-\mathcal{A}^{*}\rVert_{F}. \tag*{(66)} i ∈ [ N ] max ∥ E i ⋅ A ∥ ≤ 2 C 2 K + 2 L N , J ( J ν ∨ b ) + 2 L N , J ( ν ∨ b ) J ∥ A − A ∗ ∥ F . (66)
Proof. For each i ∈ [ N ] i\in[N] i ∈ [ N ] ,
∥ E i ⋅ A ^ ∥ ≤ ∥ E i ⋅ A ∗ ∥ + ∥ E i ⋅ ( A ^ − A ∗ ) ∥ . \lVert\mathcal{E}_{i\cdot}\widehat{\mathcal{A}}\rVert\le\lVert\mathcal{E}_{i\cdot}\mathcal{A}^{*}\rVert+\lVert\mathcal{E}_{i\cdot}(\widehat{\mathcal{A}}-\mathcal{A}^{*})\rVert. ∥ E i ⋅ A ∥ ≤ ∥ E i ⋅ A ∗ ∥ + ∥ E i ⋅ ( A − A ∗ )∥ .
Moreover,
∥ E i ⋅ ( A ^ − A ∗ ) ∥ = ∥ ∑ j = 1 J E i j ( α ^ j − α j ∗ ) ∥ ≤ ∥ E ∥ max ∑ j = 1 J ∥ α ^ j − α j ∗ ∥ ≤ ∥ E ∥ max J ∥ A ^ − A ∗ ∥ F , \begin{aligned}
\lVert\mathcal{E}_{i\cdot}(\widehat{\mathcal{A}}-\mathcal{A}^{*})\rVert
=\left\lVert\sum_{j=1}^{J}\mathcal{E}_{ij}(\widehat{\alpha}_{j}-\alpha_{j}^{*})\right\rVert\\
&\le\lVert\mathcal{E}\rVert_{\max}\sum_{j=1}^{J}\lVert\widehat{\alpha}_{j}-\alpha_{j}^{*}\rVert\\
&\le\lVert\mathcal{E}\rVert_{\max}\sqrt{J}\lVert\widehat{\mathcal{A}}-\mathcal{A}^{*}\rVert_{F},
\end{aligned} ∥ E i ⋅ ( A − A ∗ )∥ = j = 1 ∑ J E ij ( α j − α j ∗ ) ≤ ∥ E ∥ m a x j = 1 ∑ J ∥ α j − α j ∗ ∥ ≤ ∥ E ∥ m a x J ∥ A − A ∗ ∥ F ,
where the last inequality follows from the Cauchy–Schwarz inequality. Therefore,
max i ∈ [ N ] ∥ E i ⋅ A ^ ∥ ≤ max i ∈ [ N ] ∥ E i ⋅ A ∗ ∥ + ∥ E ∥ max J ∥ A ^ − A ∗ ∥ F . (67) \max_{i\in[N]}\lVert\mathcal{E}_{i\cdot}\widehat{\mathcal{A}}\rVert\le\max_{i\in[N]}\lVert\mathcal{E}_{i\cdot}\mathcal{A}^{*}\rVert+\lVert\mathcal{E}\rVert_{\max}\sqrt{J}\lVert\widehat{\mathcal{A}}-\mathcal{A}^{*}\rVert_{F}. \tag*{(67)} i ∈ [ N ] max ∥ E i ⋅ A ∥ ≤ i ∈ [ N ] max ∥ E i ⋅ A ∗ ∥ + ∥ E ∥ m a x J ∥ A − A ∗ ∥ F . (67)
Bound item (I). Recall that A ∗ = J V ∗ P ^ \mathcal{A}^{*}=\sqrt{J}\mathcal{V}^{*}\widehat{\mathbf{P}} A ∗ = J V ∗ P . Although P ^ \widehat{\mathbf{P}} P is random, its randomness does not affect the norm in item ( I ) (I) ( I ) because P ^ \widehat{\mathbf{P}} P is orthogonal. Indeed,
∥ E i ⋅ A ∗ ∥ = ∥ E i ⋅ J V ∗ P ^ ∥ = ∥ E i ⋅ J V ∗ ∥ . \lVert\mathcal{E}_{i\cdot}\mathcal{A}^{*}\rVert=\lVert\mathcal{E}_{i\cdot}\sqrt{J}\mathcal{V}^{*}\widehat{\mathbf{P}}\rVert=\lVert\mathcal{E}_{i\cdot}\sqrt{J}\mathcal{V}^{*}\rVert. ∥ E i ⋅ A ∗ ∥ = ∥ E i ⋅ J V ∗ P ∥ = ∥ E i ⋅ J V ∗ ∥ .
Define A 0 : = J V ∗ \mathcal{A}^{0}:=\sqrt{J}\mathcal{V}^{*} A 0 := J V ∗ . Conditional on the true latent configuration, A 0 \mathcal{A}^{0} A 0 is deterministic. Moreover, by the orthogonality of P ^ \widehat{\mathbf{P}} P ,
∥ A 0 ∥ 2 → ∞ = ∥ A 0 P ^ ∥ 2 → ∞ = ∥ A ∗ ∥ 2 → ∞ ≤ C 2 . \lVert\mathcal{A}^{0}\rVert_{2\to\infty}=\lVert\mathcal{A}^{0}\widehat{\mathbf{P}}\rVert_{2\to\infty}=\lVert\mathcal{A}^{*}\rVert_{2\to\infty}\le C_{2}. ∥ A 0 ∥ 2 → ∞ = ∥ A 0 P ∥ 2 → ∞ = ∥ A ∗ ∥ 2 → ∞ ≤ C 2 .
Consequently, ( I ) = max i ∈ [ N ] ∥ E i . A 0 ∥ (I) = \max_{i \in[N]} \lVert\mathcal{E}_i.\mathcal{A}^0 \rVert ( I ) = max i ∈ [ N ] ∥ E i . A 0 ∥ .
For each i ∈ [ N ] i \in[N] i ∈ [ N ] and k ∈ [ K + 2 ] k \in[K + 2] k ∈ [ K + 2 ] ,
E i . A ⋅ k 0 = ∑ j = 1 J E i j A j k 0 . \mathcal{E}_i.\mathcal{A}^0_{\cdot k} = \sum_{j=1}^{J} \mathcal{E}_{ij}\mathcal{A}^0_{jk}. E i . A ⋅ k 0 = j = 1 ∑ J E ij A j k 0 .
Since ∥ A 0 ∥ 2 → ∞ ≤ C 2 \lVert\mathcal{A}^0 \rVert_{2 \to\infty} \le C_2 ∥ A 0 ∥ 2 → ∞ ≤ C 2 ,
max j ∈ [ J ] ∣ A j k 0 ∣ ≤ C 2 , ∥ A ⋅ k 0 ∥ 2 ≤ J C 2 2 . \max_{j \in[J]} \lvert\mathcal{A}^0_{jk} \rvert\le C_2, \qquad\lVert\mathcal{A}^0_{\cdot k} \rVert^2 \le JC_2^2. j ∈ [ J ] max ∣ A j k 0 ∣ ≤ C 2 , ∥ A ⋅ k 0 ∥ 2 ≤ J C 2 2 .
By the standard tail bound for weighted sums of independent centered sub-exponential random variables,
Pr ( ∣ E i . A ⋅ k 0 ∣ ≥ t ) ≤ 2 ( exp { − t 2 2 J C 2 2 ν 2 } ∨ exp { − t 2 C 2 b } ) . \Pr\left(\lvert\mathcal{E}_i.\mathcal{A}^0_{\cdot k} \rvert\ge t\right) \le2\left(\exp\left\{-\frac{t^2}{2JC_2^2\nu^2}\right\} \vee\exp\left\{-\frac{t}{2C_2b}\right\}\right). Pr ( ∣ E i . A ⋅ k 0 ∣ ≥ t ) ≤ 2 ( exp { − 2 J C 2 2 ν 2 t 2 } ∨ exp { − 2 C 2 b t } ) .
Hence, for any t > 0 t > 0 t > 0 ,
Pr ( ∥ E i . A 0 ∥ ≥ t ) ≤ ∑ k = 1 K + 2 Pr ( ∣ E i . A ⋅ k 0 ∣ ≥ t K + 2 ) \Pr\left(\lVert\mathcal{E}_i.\mathcal{A}^0 \rVert\ge t\right) \le\sum_{k=1}^{K+2} \Pr\left(\lvert\mathcal{E}_i.\mathcal{A}^0_{\cdot k} \rvert\ge\frac{t}{\sqrt{K+2}}\right) Pr ( ∥ E i . A 0 ∥ ≥ t ) ≤ k = 1 ∑ K + 2 Pr ( ∣ E i . A ⋅ k 0 ∣ ≥ K + 2 t )
≤ 2 ( K + 2 ) ( exp { − t 2 2 J C 2 2 ( K + 2 ) ν 2 } ∨ exp { − t 2 C 2 K + 2 b } ) . \le2(K+2)\left(\exp\left\{-\frac{t^2}{2JC_2^2(K+2)\nu^2}\right\} \vee\exp\left\{-\frac{t}{2C_2\sqrt{K+2}b}\right\}\right). ≤ 2 ( K + 2 ) ( exp { − 2 J C 2 2 ( K + 2 ) ν 2 t 2 } ∨ exp { − 2 C 2 K + 2 b t } ) .
Applying the union bound over i ∈ [ N ] i \in[N] i ∈ [ N ] gives
Pr ( max i ∈ [ N ] ∥ E i . A 0 ∥ ≥ t ) ≤ 2 N ( K + 2 ) ( exp { − t 2 2 J C 2 2 ( K + 2 ) ν 2 } ∨ exp { − t 2 C 2 K + 2 b } ) . (68) \Pr\left(\max_{i \in[N]} \lVert\mathcal{E}_i.\mathcal{A}^0 \rVert\ge t\right)
\le2N(K+2)\left(\exp\left\{-\frac{t^2}{2JC_2^2(K+2)\nu^2}\right\} \vee\exp\left\{-\frac{t}{2C_2\sqrt{K+2}b}\right\}\right). \tag*{(68)} Pr ( i ∈ [ N ] max ∥ E i . A 0 ∥ ≥ t ) ≤ 2 N ( K + 2 ) ( exp { − 2 J C 2 2 ( K + 2 ) ν 2 t 2 } ∨ exp { − 2 C 2 K + 2 b t } ) . (68)
Set t 1 = 2 C 2 K + 2 L N , J ( J ν ∨ b ) t_1 = 2C_2\sqrt{K+2}L_{N,J}(\sqrt{J}\nu\vee b) t 1 = 2 C 2 K + 2 L N , J ( J ν ∨ b ) . Since L N , J ≥ log 2 L_{N,J} \ge\log2 L N , J ≥ log 2 , we have 2 L N , J ≥ 2 L N , J 2L_{N,J} \ge\sqrt{2L_{N,J}} 2 L N , J ≥ 2 L N , J , and therefore
t 1 ≥ C 2 2 J ( K + 2 ) L N , J ν , t 1 ≥ 2 C 2 K + 2 L N , J b . t_1 \ge C_2\sqrt{2J(K+2)L_{N,J}}\nu, \qquad t_1 \ge2C_2\sqrt{K+2}L_{N,J}b. t 1 ≥ C 2 2 J ( K + 2 ) L N , J ν , t 1 ≥ 2 C 2 K + 2 L N , J b .
Substituting t = t 1 t=t_1 t = t 1 into (68) yields
Pr ( ( I ) ≥ t 1 ) ≤ 2 N ( K + 2 ) e − L N , J = 1 N J ( K + 2 ) ≤ 1 N ( K + 2 ) . \begin{aligned}
\Pr\left((I)\ge t_1\right)\le2N(K+2)e^{-L_{N,J}} \\
&=\frac{1}{NJ(K+2)}\le\frac{1}{N(K+2)}.
\end{aligned} Pr ( ( I ) ≥ t 1 ) ≤ 2 N ( K + 2 ) e − L N , J = N J ( K + 2 ) 1 ≤ N ( K + 2 ) 1 .
Hence, with probability at least 1 − { N ( K + 2 ) } − 1 1-\{N(K+2)\}^{-1} 1 − { N ( K + 2 ) } − 1 ,
( I ) ≤ 2 C 2 K + 2 L N , J ( J ν ∨ b ) . (69) (I)\le2C_2\sqrt{K+2}L_{N,J}\left(\sqrt{J}\,\nu\vee b\right). \tag*{(69)} ( I ) ≤ 2 C 2 K + 2 L N , J ( J ν ∨ b ) . (69)
Bound item (II). By Lemma 4, with probability at least 1 − δ 1-\delta 1 − δ ,
∥ E ∥ max ≤ ν 2 log ( 2 N J / δ ) ∨ 2 b log ( 2 N J / δ ) . \|\mathcal{E}\|_{\max}\le\nu\sqrt{2\log(2NJ/\delta)}\vee2b\log(2NJ/\delta). ∥ E ∥ m a x ≤ ν 2 log ( 2 N J / δ ) ∨ 2 b log ( 2 N J / δ ) .
Choose δ = { N ( K + 2 ) } − 1 \delta=\{N(K+2)\}^{-1} δ = { N ( K + 2 ) } − 1 . Then log ( 2 N J / δ ) = log ( 2 N 2 J ( K + 2 ) ) ≤ L N , J \log(2NJ/\delta)=\log(2N^2J(K+2))\le L_{N,J} log ( 2 N J / δ ) = log ( 2 N 2 J ( K + 2 )) ≤ L N , J . Since 2 L N , J ≥ 2 L N , J 2L_{N,J}\ge\sqrt{2L_{N,J}} 2 L N , J ≥ 2 L N , J , we obtain ∥ E ∥ max ≤ 2 L N , J ( ν ∨ b ) \|\mathcal{E}\|_{\max}\le2L_{N,J}(\nu\vee b) ∥ E ∥ m a x ≤ 2 L N , J ( ν ∨ b ) with probability at least 1 − { N ( K + 2 ) } − 1 1-\{N(K+2)\}^{-1} 1 − { N ( K + 2 ) } − 1 . Therefore, with the same probability,
( I I ) ≤ 2 L N , J ( ν ∨ b ) J ∥ A ^ − A ∗ ∥ F . (70) (II)\le2L_{N,J}(\nu\vee b)\sqrt{J}\|\widehat{A}-A^*\|_F. \tag*{(70)} ( I I ) ≤ 2 L N , J ( ν ∨ b ) J ∥ A − A ∗ ∥ F . (70)
Finally, combining (67) , (69) , and (70) , and applying the union bound once more, we conclude that (66) holds with probability at least 1 − 2 / ( N ( K + 2 ) ) 1-2/(N(K+2)) 1 − 2/ ( N ( K + 2 )) .
Lemma 6 (Upper bound for max i ∈ [ N ] ∥ B 1 , i ( A ^ ) ∥ \max_{i\in[N]}\|\mathbf{B}_{1,i}(\widehat{A})\| max i ∈ [ N ] ∥ B 1 , i ( A ) ∥ ). Let A ∗ = J ν ∗ P ^ A^*=\sqrt{J}\nu^*\widehat{P} A ∗ = J ν ∗ P and B ∗ = J − 1 / 2 U ∗ D ∗ P ^ \mathcal{B}^*=J^{-1/2}U^*D^*\widehat{P} B ∗ = J − 1/2 U ∗ D ∗ P . Assume ∥ A ^ ∥ 2 → ∞ ≤ C 2 \|\widehat{A}\|_{2\to\infty}\le C_2 ∥ A ∥ 2 → ∞ ≤ C 2 and ∥ B ∗ ∥ 2 → ∞ ≤ C 1 \|\mathcal{B}^*\|_{2\to\infty}\le C_1 ∥ B ∗ ∥ 2 → ∞ ≤ C 1 . Then,
max i ∈ [ N ] ∥ B 1 , i ( A ^ ) ∥ ≤ C 1 C 2 κ 1 ∗ J ∥ A ^ − A ∗ ∥ F . (71) \max_{i\in[N]}\|\mathbf{B}_{1,i}(\widehat{A})\|\le C_1C_2\kappa_1^*\sqrt{J}\|\widehat{A}-A^*\|_F. \tag*{(71)} i ∈ [ N ] max ∥ B 1 , i ( A ) ∥ ≤ C 1 C 2 κ 1 ∗ J ∥ A − A ∗ ∥ F . (71)
Proof. Recall that
B 1 , i ( A ^ ) = ∑ j = 1 J f ′ ( m i j ∗ ) α ^ j ( α ^ j − α j ∗ ) T β i ∗ . \mathbf{B}_{1,i}(\widehat{\mathcal{A}})=\sum_{j=1}^{J}f'(m_{ij}^{*})\widehat{\alpha}_{j}(\widehat{\alpha}_{j}-\alpha_{j}^{*})^{\mathsf{T}}\beta_{i}^{*}. B 1 , i ( A ) = j = 1 ∑ J f ′ ( m ij ∗ ) α j ( α j − α j ∗ ) T β i ∗ .
Since ∣ m i j ∗ ∣ ≤ C 0 |m_{ij}^{*}|\le C_{0} ∣ m ij ∗ ∣ ≤ C 0 , we have ∣ f ′ ( m i j ∗ ) ∣ ≤ κ 1 ( C 0 ) = κ 1 ∗ |f'(m_{ij}^{*})|\le\kappa_{1}(C_{0})=\kappa_{1}^{*} ∣ f ′ ( m ij ∗ ) ∣ ≤ κ 1 ( C 0 ) = κ 1 ∗ . Therefore, for each i ∈ [ N ] i\in[N] i ∈ [ N ] ,
∥ B 1 , i ( A ^ ) ∥ ≤ ∑ j = 1 J ∣ f ′ ( m i j ∗ ) ∣ ∥ α ^ j ( α ^ j − α j ∗ ) T β i ∗ ∣ = ∑ j = 1 J ∣ f ′ ( m i j ∗ ) ∣ ∥ α ^ j ∥ ∣ ( α j − α j ∗ ) T β i ∗ ∣ ≤ ∑ j = 1 J ∣ f ′ ( m i j ∗ ) ∣ ∥ α ^ j ∥ ∥ α ^ j − α j ∗ ∥ ∥ β i ∗ ∥ ≤ C 1 C 2 κ 1 ∗ ∑ j = 1 J ∥ α ^ j − α j ∗ ∥ . \begin{aligned}
\|\mathbf{B}_{1,i}(\widehat{\mathcal{A}})\|\le\sum_{j=1}^{J}|f'(m_{ij}^{*})|\|\widehat{\alpha}_{j}(\widehat{\alpha}_{j}-\alpha_{j}^{*})^{\mathsf{T}}\beta_{i}^{*}| \\
&=\sum_{j=1}^{J}|f'(m_{ij}^{*})|\|\widehat{\alpha}_{j}\|\,|(\alpha_{j}-\alpha_{j}^{*})^{\mathsf{T}}\beta_{i}^{*}| \\
&\le\sum_{j=1}^{J}|f'(m_{ij}^{*})|\|\widehat{\alpha}_{j}\|\|\widehat{\alpha}_{j}-\alpha_{j}^{*}\|\|\beta_{i}^{*}\| \\
&\le C_{1}C_{2}\kappa_{1}^{*}\sum_{j=1}^{J}\|\widehat{\alpha}_{j}-\alpha_{j}^{*}\|.
\end{aligned} ∥ B 1 , i ( A ) ∥ ≤ j = 1 ∑ J ∣ f ′ ( m ij ∗ ) ∣∥ α j ( α j − α j ∗ ) T β i ∗ ∣ = j = 1 ∑ J ∣ f ′ ( m ij ∗ ) ∣∥ α j ∥ ∣ ( α j − α j ∗ ) T β i ∗ ∣ ≤ j = 1 ∑ J ∣ f ′ ( m ij ∗ ) ∣∥ α j ∥∥ α j − α j ∗ ∥∥ β i ∗ ∥ ≤ C 1 C 2 κ 1 ∗ j = 1 ∑ J ∥ α j − α j ∗ ∥.
Since the right-hand side does not depend on i i i , taking the maximum over i ∈ [ N ] i\in[N] i ∈ [ N ] gives
max i ∈ [ N ] ∥ B 1 , i ( A ^ ) ∥ ≤ C 1 C 2 κ 1 ∗ ∑ j = 1 J ∥ α ^ j − α j ∗ ∥ . \max_{i\in[N]}\|\mathbf{B}_{1,i}(\widehat{\mathcal{A}})\|\le C_{1}C_{2}\kappa_{1}^{*}\sum_{j=1}^{J}\|\widehat{\alpha}_{j}-\alpha_{j}^{*}\|. i ∈ [ N ] max ∥ B 1 , i ( A ) ∥ ≤ C 1 C 2 κ 1 ∗ j = 1 ∑ J ∥ α j − α j ∗ ∥.
By the Cauchy–Schwarz inequality,
∑ j = 1 J ∥ α ^ j − α j ∗ ∥ ≤ J ( ∑ j = 1 J ∥ α ^ j − α j ∗ ∥ 2 ) 1 / 2 = J ∥ A ^ − A ∗ ∥ F . \sum_{j=1}^{J}\|\widehat{\alpha}_{j}-\alpha_{j}^{*}\|\le\sqrt{J}\left(\sum_{j=1}^{J}\|\widehat{\alpha}_{j}-\alpha_{j}^{*}\|^{2}\right)^{1/2}=\sqrt{J}\|\widehat{\mathcal{A}}-\mathcal{A}^{*}\|_{F}. j = 1 ∑ J ∥ α j − α j ∗ ∥ ≤ J ( j = 1 ∑ J ∥ α j − α j ∗ ∥ 2 ) 1/2 = J ∥ A − A ∗ ∥ F .
Hence,
max i ∈ [ N ] ∥ B 1 , i ( A ^ ) ∥ ≤ C 1 C 2 κ 1 ∗ J ∥ A ^ − A ∗ ∥ F . \max_{i\in[N]}\|\mathbf{B}_{1,i}(\widehat{\mathcal{A}})\|\le C_{1}C_{2}\kappa_{1}^{*}\sqrt{J}\|\widehat{\mathcal{A}}-\mathcal{A}^{*}\|_{F}. i ∈ [ N ] max ∥ B 1 , i ( A ) ∥ ≤ C 1 C 2 κ 1 ∗ J ∥ A − A ∗ ∥ F .
Lemma 7 (Bound for max i ∈ [ N ] β 1 , i ( A ^ ) \max_{i\in[N]}\beta_{1,i}(\widehat{\mathcal{A}}) max i ∈ [ N ] β 1 , i ( A ) ). Let A ∗ = J V ∗ P ^ \mathcal{A}^{*}=\sqrt{J}\mathcal{V}^{*}\widehat{\mathbf{P}} A ∗ = J V ∗ P and B ∗ = J − 1 / 2 U ∗ D ∗ P ^ \mathcal{B}^{*}=J^{-1/2}\mathbf{U}^{*}\mathbf{D}^{*}\widehat{\mathbf{P}} B ∗ = J − 1/2 U ∗ D ∗ P . Assume
∥ A ^ ∥ 2 → ∞ ≤ C 2 \|\widehat{A}\|_{2\to\infty} \le C_2 ∥ A ∥ 2 → ∞ ≤ C 2 and ∥ B ∗ ∥ 2 → ∞ ≤ C 1 \|B^*\|_{2\to\infty} \le C_1 ∥ B ∗ ∥ 2 → ∞ ≤ C 1 . Then
max i ∈ [ N ] β 1 , i ( A ^ ) ≤ C 1 2 C 2 ∥ A ^ − A ∗ ∥ F 2 . (72) \max_{i\in[N]} \beta_{1,i}(\widehat{A}) \le C_1^2 C_2 \|\widehat{A}-A^*\|_F^2. \tag*{(72)} i ∈ [ N ] max β 1 , i ( A ) ≤ C 1 2 C 2 ∥ A − A ∗ ∥ F 2 . (72)
Proof. For each i ∈ [ N ] i\in[N] i ∈ [ N ] ,
β 1 , i ( A ^ ) = sup ∥ u ∥ = 1 ∑ j = 1 J ∣ ( α ^ j − α j ∗ ) T β i ∗ ∣ 2 ∣ α ^ j T u ∣ ≤ C 1 2 C 2 ∑ j = 1 J ∥ α ^ j − α j ∗ ∥ 2 = C 1 2 C 2 ∥ A ^ − A ∗ ∥ F 2 . \begin{aligned}
\beta_{1,i}(\widehat{A}) &= \sup_{\|u\|=1} \sum_{j=1}^{J} \left|(\widehat{\alpha}_j-\alpha_j^*)^\mathsf{T}\beta_i^*\right|^2|\widehat{\alpha}_j^\mathsf{T}u| \\
&\le C_1^2 C_2 \sum_{j=1}^{J}\|\widehat{\alpha}_j-\alpha_j^*\|^2 \\
&= C_1^2 C_2\|\widehat{A}-A^*\|_F^2.
\end{aligned} β 1 , i ( A ) = ∥ u ∥ = 1 sup j = 1 ∑ J ( α j − α j ∗ ) T β i ∗ 2 ∣ α j T u ∣ ≤ C 1 2 C 2 j = 1 ∑ J ∥ α j − α j ∗ ∥ 2 = C 1 2 C 2 ∥ A − A ∗ ∥ F 2 .
Since the right-hand side does not depend on i i i , taking the maximum over i ∈ [ N ] i\in[N] i ∈ [ N ] yields
max i ∈ [ N ] β 1 , i ( A ^ ) ≤ C 1 2 C 2 ∥ A ^ − A ∗ ∥ F 2 . \max_{i\in[N]} \beta_{1,i}(\widehat{A}) \le C_1^2 C_2\|\widehat{A}-A^*\|_F^2. i ∈ [ N ] max β 1 , i ( A ) ≤ C 1 2 C 2 ∥ A − A ∗ ∥ F 2 .
Lemma 8 (Bound for γ 1 , i ( A ^ ) \gamma_{1,i}(\widehat{A}) γ 1 , i ( A ) ). If ∥ A ^ ∥ 2 → ∞ ≤ C 2 \|\widehat{A}\|_{2\to\infty}\le C_2 ∥ A ∥ 2 → ∞ ≤ C 2 , then
γ 1 , i ( A ^ ) ≤ J C 2 3 . (73) \gamma_{1,i}(\widehat{A}) \le J C_2^3. \tag*{(73)} γ 1 , i ( A ) ≤ J C 2 3 . (73) Proof. By the definition of γ 1 , i \gamma_{1,i} γ 1 , i ,
γ 1 , i ( A ^ ) = sup ∥ u ∥ = 1 ∑ j = 1 J ∣ α ^ j T u ∣ 3 ≤ ∑ j = 1 J ∥ α ^ j ∥ 3 ≤ J C 2 3 . \begin{aligned}
\gamma_{1,i}(\widehat{A}) &= \sup_{\|u\|=1}\sum_{j=1}^{J}|\widehat{\alpha}_j^\mathsf{T}u|^3 \\
&\le\sum_{j=1}^{J}\|\widehat{\alpha}_j\|^3 \\
&\le J C_2^3.
\end{aligned} γ 1 , i ( A ) = ∥ u ∥ = 1 sup j = 1 ∑ J ∣ α j T u ∣ 3 ≤ j = 1 ∑ J ∥ α j ∥ 3 ≤ J C 2 3 .
Two-to-Infinity Bounds for the Refined Factors
Lemma 9 (Asymptotic analysis of B ~ \widetilde{\mathcal B} B ). Let A ∗ = J V ∗ P ^ \mathcal A^*=\sqrt{J}\mathcal V^*\widehat{\mathcal P} A ∗ = J V ∗ P and B ∗ = J − 1 / 2 U ∗ D ∗ P ^ \mathcal B^*=J^{-1/2}\mathcal U^*\mathcal D^*\widehat{\mathcal P} B ∗ = J − 1/2 U ∗ D ∗ P . Suppose Assumption 1 holds. Assume that ∥ A ^ − A ∗ ∥ F = O p ( e A , F ) \|\widehat{\mathcal A}-\mathcal A^*\|_{\mathrm F}=O_p(e_{A,F}) ∥ A − A ∗ ∥ F = O p ( e A , F ) , where e A , F e_{A,F} e A , F is deterministic and satisfies
e A , F = Ω ( 1 ) , e A , F = o ( J ( log ( N J ) ) − 1 ) . e_{A,F}=\Omega(1),\qquad e_{A,F}=o\left(\sqrt{J}(\log(NJ))^{-1}\right). e A , F = Ω ( 1 ) , e A , F = o ( J ( log ( N J ) ) − 1 ) . Then, with probability tending to one, for every i ∈ [ N ] i\in[N] i ∈ [ N ] , the equation
S 1 , i ( β i ; A ^ ) = 0 K + 2 S_{1,i}(\beta_i;\widehat{\mathcal A})=\mathbf{0}_{K+2} S 1 , i ( β i ; A ) = 0 K + 2 admits a unique solution β ~ i \widetilde{\beta}_i β i satisfying ∥ β ~ i − β i ∗ ∥ ≤ C 1 \|\widetilde{\beta}_i-\beta_i^*\|\leq C_1 ∥ β i − β i ∗ ∥ ≤ C 1 . Moreover,
∥ B ~ − B ∗ ∥ 2 → ∞ = O p ( J − 1 / 2 log ( N J ) e A , F ) . \|\widetilde{\mathcal B}-\mathcal B^*\|_{2\to\infty}
=O_p\left(J^{-1/2}\log(NJ)e_{A,F}\right). ∥ B − B ∗ ∥ 2 → ∞ = O p ( J − 1/2 log ( N J ) e A , F ) .
Proof. Let L N , J : = log ( N J ) L_{N,J}:=\log(NJ) L N , J := log ( N J ) . Under Assumption 1.2, the centered noises E i j = y i j − f ( m i j ∗ ) \mathcal E_{ij}=y_{ij}-f(m_{ij}^*) E ij = y ij − f ( m ij ∗ ) are uniformly sub-exponential. Hence, the parameters ν \nu ν and b b b in Lemmas 4 and 5 are uniformly bounded. Since K K K is fixed, log ( 2 N 2 J ( K + 2 ) 2 ) ≲ L N , J \log(2N^2J(K+2)^2)\lesssim L_{N,J} log ( 2 N 2 J ( K + 2 ) 2 ) ≲ L N , J . Therefore, Lemma 5 and ∥ A ^ − A ∗ ∥ F = O p ( e A , F ) \|\widehat{\mathcal A}-\mathcal A^*\|_{\mathrm F}=O_p(e_{A,F}) ∥ A − A ∗ ∥ F = O p ( e A , F ) imply
max i ∈ [ N ] ∥ E i ⋅ A ^ ∥ = O p ( J L N , J + J L N , J e A , F ) = O p ( J L N , J e A , F ) , \max_{i\in[N]}\|\mathcal E_{i\cdot}\widehat{\mathcal A}\|
=O_p\left(\sqrt{J}L_{N,J}+\sqrt{J}L_{N,J}e_{A,F}\right)
=O_p\left(\sqrt{J}L_{N,J}e_{A,F}\right), i ∈ [ N ] max ∥ E i ⋅ A ∥ = O p ( J L N , J + J L N , J e A , F ) = O p ( J L N , J e A , F ) ,
where the second equality uses e A , F = Ω ( 1 ) e_{A,F}=\Omega(1) e A , F = Ω ( 1 ) . Similarly, Lemmas 6 and 7 give
max i ∈ [ N ] ∥ B 1 , i ( A ^ ) ∥ = O p ( J e A , F ) , \max_{i\in[N]} \lVert\mathbf{B}_{1,i}(\widehat{\mathcal{A}})\rVert= O_p\left(\sqrt{J}e_{\mathcal{A},F}\right), i ∈ [ N ] max ∥ B 1 , i ( A )∥ = O p ( J e A , F ) ,
and
max i ∈ [ N ] β 1 , i ( A ^ ) = O p ( e A , F 2 ) . \max_{i\in[N]} \beta_{1,i}(\widehat{\mathcal{A}}) = O_p\left(e_{\mathcal{A},F}^{2}\right). i ∈ [ N ] max β 1 , i ( A ) = O p ( e A , F 2 ) .
For each i ∈ [ N ] i \in[N] i ∈ [ N ] , define
T 1 , i : = ∥ E i ⋅ A ^ ∥ + ∥ B 1 , i ( A ^ ) ∥ + β 1 , i ( A ^ ) κ 2 ∗ . T_{1,i} := \lVert\mathcal{E}_i\cdot\widehat{\mathcal{A}}\rVert+ \lVert\mathbf{B}_{1,i}(\widehat{\mathcal{A}})\rVert+ \beta_{1,i}(\widehat{\mathcal{A}})\kappa_2^*. T 1 , i := ∥ E i ⋅ A ∥ + ∥ B 1 , i ( A )∥ + β 1 , i ( A ) κ 2 ∗ .
Since κ 2 ∗ \kappa_2^* κ 2 ∗ is a fixed constant,
max i ∈ [ N ] T 1 , i = O p ( J L N , J e A , F + J e A , F + e A , F 2 ) . \max_{i\in[N]} T_{1,i} = O_p\left(\sqrt{J}L_{N,J}e_{\mathcal{A},F}+\sqrt{J}e_{\mathcal{A},F}+e_{\mathcal{A},F}^{2}\right). i ∈ [ N ] max T 1 , i = O p ( J L N , J e A , F + J e A , F + e A , F 2 ) .
By e A , F = o ( J L N , J − 1 ) e_{\mathcal{A},F}=o\left(\sqrt{J}L_{N,J}^{-1}\right) e A , F = o ( J L N , J − 1 ) , all three terms on the right-hand side are o ( J ) o(J) o ( J ) . Hence,
max i ∈ [ N ] T 1 , i = o p ( J ) . (74) \max_{i\in[N]} T_{1,i}=o_p(J). \tag*{(74)} i ∈ [ N ] max T 1 , i = o p ( J ) . (74)
By Lemma 8,
max i ∈ [ N ] γ 1 , i ( A ^ ) ≲ J . (75) \max_{i\in[N]} \gamma_{1,i}(\widehat{\mathcal{A}})\lesssim J. \tag*{(75)} i ∈ [ N ] max γ 1 , i ( A ) ≲ J . (75)
Next, since A ∗ = J ν ∗ P ^ \mathcal{A}^*=\sqrt{J}\boldsymbol{\nu}^*\widehat{\mathbf{P}} A ∗ = J ν ∗ P , we have σ K + 2 ( A ∗ ) = J \sigma_{K+2}(\mathcal{A}^*)=\sqrt{J} σ K + 2 ( A ∗ ) = J . Moreover,
∥ A ^ − A ∗ ∥ 2 ≤ ∥ A ^ − A ∗ ∥ F = O p ( e A , F ) = o p ( J ) . \lVert\widehat{\mathcal{A}}-\mathcal{A}^*\rVert_2\leq\lVert\widehat{\mathcal{A}}-\mathcal{A}^*\rVert_F=O_p(e_{\mathcal{A},F})=o_p(\sqrt{J}). ∥ A − A ∗ ∥ 2 ≤ ∥ A − A ∗ ∥ F = O p ( e A , F ) = o p ( J ) .
Therefore, Weyl’s inequality yields
σ K + 2 ( A ^ ) ≲ J (76) \sigma_{K+2}(\widehat{\mathcal{A}}) \lesssim\sqrt{J} \tag*{(76)} σ K + 2 ( A ) ≲ J (76)
with probability tending to one. In particular, rank ( A ^ ) = K + 2 \operatorname{rank}(\widehat{\mathcal{A}}) = K + 2 rank ( A ) = K + 2 with probability tending to one.
Since ∣ m i j ∗ ∣ ≤ C 0 |m_{ij}^{*}| \le C_0 ∣ m ij ∗ ∣ ≤ C 0 , Assumption 1.3 gives f ′ ( m i j ∗ ) ≥ δ 1 ∗ > 0 f'(m_{ij}^{*}) \ge\delta_1^{*} > 0 f ′ ( m ij ∗ ) ≥ δ 1 ∗ > 0 . Thus,
I 1 , i ( A ^ ) = ∑ j = 1 J f ′ ( m i j ∗ ) α ^ j α ^ j ⊤ ⪰ δ 1 ∗ A ^ ⊤ A ^ . \mathcal{I}_{1,i}(\widehat{\mathcal{A}}) = \sum_{j=1}^{J} f'(m_{ij}^{*})\widehat{\alpha}_j\widehat{\alpha}_j^{\top} \succeq\delta_1^{*}\widehat{\mathcal{A}}^{\top}\widehat{\mathcal{A}}. I 1 , i ( A ) = j = 1 ∑ J f ′ ( m ij ∗ ) α j α j ⊤ ⪰ δ 1 ∗ A ⊤ A .
Together with (76) , uniformly over i ∈ [ N ] i \in[N] i ∈ [ N ] ,
σ K + 2 ( I 1 , i ( A ^ ) ) ≳ J (77) \sigma_{K+2}(\mathcal{I}_{1,i}(\widehat{\mathcal{A}})) \gtrsim J \tag*{(77)} σ K + 2 ( I 1 , i ( A )) ≳ J (77)
with probability tending to one.
We now verify the two conditions in (61) . By (74) and (77) ,
max i ∈ [ N ] 2 T 1 , i σ K + 2 ( I 1 , i ( A ^ ) ) C 1 = o p ( 1 ) . \max_{i\in[N]} \frac{2T_{1,i}}{\sigma_{K+2}(\mathcal{I}_{1,i}(\widehat{\mathcal{A}}))C_1} = o_p(1). i ∈ [ N ] max σ K + 2 ( I 1 , i ( A )) C 1 2 T 1 , i = o p ( 1 ) .
Hence, with probability tending to one, 2 T 1 , i ≤ σ K + 2 ( I 1 , i ( A ^ ) ) C 1 2T_{1,i} \le\sigma_{K+2}(\mathcal{I}_{1,i}(\widehat{\mathcal{A}}))C_1 2 T 1 , i ≤ σ K + 2 ( I 1 , i ( A )) C 1 simultaneously for all i ∈ [ N ] i \in[N] i ∈ [ N ] .
Moreover, by (75) , (74) , and (77) ,
max i ∈ [ N ] 4 γ 1 , i ( A ^ ) κ 2 ∗ T 1 , i σ K + 2 2 ( I 1 , i ( A ^ ) ) = o p ( 1 ) . \max_{i\in[N]} \frac{4\gamma_{1,i}(\widehat{\mathcal{A}})\kappa_2^{*}T_{1,i}}{\sigma_{K+2}^{2}(\mathcal{I}_{1,i}(\widehat{\mathcal{A}}))} = o_p(1). i ∈ [ N ] max σ K + 2 2 ( I 1 , i ( A )) 4 γ 1 , i ( A ) κ 2 ∗ T 1 , i = o p ( 1 ) .
Thus, with probability tending to one,
4 γ 1 , i ( A ^ ) κ ~ 2 T 1 , i ≤ σ K + 2 2 ( L 1 , i ( A ^ ) ) 4\gamma_{1,i}(\widehat{\mathcal{A}})\widetilde{\kappa}_2 T_{1,i} \le\sigma^2_{K+2}(\mathcal{L}_{1,i}(\widehat{\mathcal{A}})) 4 γ 1 , i ( A ) κ 2 T 1 , i ≤ σ K + 2 2 ( L 1 , i ( A ))
simultaneously for all i ∈ [ N ] i \in[N] i ∈ [ N ] .
Therefore, Lemma 3 applies simultaneously to all i ∈ [ N ] i \in[N] i ∈ [ N ] with probability tending to one.
Hence, for every i ∈ [ N ] i \in[N] i ∈ [ N ] , there exists a unique β ~ i \widetilde{\beta}_i β i satisfying
S 1 , i ( β ~ i ; A ^ ) = 0 K + 2 , ∥ β ~ i − β i ∗ ∥ ≤ C 1 . S_{1,i}(\widetilde{\beta}_i;\widehat{\mathcal{A}}) = 0_{K+2}, \qquad\|\widetilde{\beta}_i-\beta_i^*\| \le C_1. S 1 , i ( β i ; A ) = 0 K + 2 , ∥ β i − β i ∗ ∥ ≤ C 1 .
Furthermore, by (62) and (77) ,
∥ B ~ − B ∗ ∥ 2 → ∞ ≲ J − 1 max i ∈ [ N ] T 1 , i = O p ( J − 1 / 2 L N , J e A , F + J − 1 / 2 e A , F + J − 1 e A , F 2 ) . \begin{aligned}
\|\widetilde{\mathcal{B}}-\mathcal{B}^*\|_{2\to\infty} \lesssim J^{-1}\max_{i\in[N]}T_{1,i} \\
&= O_p\left(J^{-1/2}L_{N,J}e_{A,F}+J^{-1/2}e_{A,F}+J^{-1}e_{A,F}^2\right).
\end{aligned} ∥ B − B ∗ ∥ 2 → ∞ ≲ J − 1 i ∈ [ N ] max T 1 , i = O p ( J − 1/2 L N , J e A , F + J − 1/2 e A , F + J − 1 e A , F 2 ) .
Since L N , J ≥ 1 L_{N,J}\ge1 L N , J ≥ 1 and e A , F = o ( J L N , J − 1 ) e_{A,F}=o(\sqrt{J}L_{N,J}^{-1}) e A , F = o ( J L N , J − 1 ) , the last two terms are dominated by the first one. Therefore,
∥ B ~ − B ∗ ∥ 2 → ∞ = O p ( J − 1 / 2 L N , J e A , F ) . \|\widetilde{\mathcal{B}}-\mathcal{B}^*\|_{2\to\infty}=O_p\left(J^{-1/2}L_{N,J}e_{A,F}\right). ∥ B − B ∗ ∥ 2 → ∞ = O p ( J − 1/2 L N , J e A , F ) .
Since L N , J = log ( N J ) L_{N,J}=\log(NJ) L N , J = log ( N J ) ,
∥ B ~ − B ∗ ∥ 2 → ∞ = O p ( J − 1 / 2 log ( N J ) e A , F ) . \|\widetilde{\mathcal{B}}-\mathcal{B}^*\|_{2\to\infty}=O_p\left(J^{-1/2}\log(NJ)e_{A,F}\right). ∥ B − B ∗ ∥ 2 → ∞ = O p ( J − 1/2 log ( N J ) e A , F ) .
Lemma 10 (Asymptotic analysis of A ~ \widetilde{\mathcal{A}} A . ) Suppose the conditions of Lemma 9 hold, with the
rate condition strengthened to
e A , F = o ( J ( log ( N J ) ) − 2 ) . e_{A,F}=o\left(\sqrt{J}(\log(NJ))^{-2}\right). e A , F = o ( J ( log ( N J ) ) − 2 ) .
Then, with probability tending to one, for every j ∈ [ J ] j\in[J] j ∈ [ J ] , the equation
S 2 , j ( α j ; B ~ ) = 0 K + 2 S_{2,j}(\alpha_j;\widetilde{B})=0_{K+2} S 2 , j ( α j ; B ) = 0 K + 2
admits a unique solution α ~ j \widetilde{\alpha}_j α j satisfying ∥ α ~ j − α j ∗ ∥ ≤ C 2 \|\widetilde{\alpha}_j-\alpha_j^*\|\le C_2 ∥ α j − α j ∗ ∥ ≤ C 2 . Moreover,
∥ A ~ − A ∗ ∥ 2 → ∞ = O p ( N − 1 / 2 log ( N J ) + J − 1 / 2 ( log ( N J ) ) 2 e A , F ) . \|\widetilde{A}-A^*\|_{2\to\infty}=O_p\left(N^{-1/2}\log(NJ)+J^{-1/2}(\log(NJ))^2e_{A,F}\right). ∥ A − A ∗ ∥ 2 → ∞ = O p ( N − 1/2 log ( N J ) + J − 1/2 ( log ( N J ) ) 2 e A , F ) .
Proof. Let L N , J : = log ( N J ) L_{N,J}:=\log(NJ) L N , J := log ( N J ) . By Lemma 9,
∥ B ~ − B ∗ ∥ 2 → ∞ = O p ( J − 1 / 2 L N , J e A , F ) . \|\widetilde{B}-B^*\|_{2\to\infty}=O_p\left(J^{-1/2}L_{N,J}e_{A,F}\right). ∥ B − B ∗ ∥ 2 → ∞ = O p ( J − 1/2 L N , J e A , F ) .
Since e A , F = o ( J L N , J − 2 ) e_{A,F}=o\left(\sqrt{J}L_{N,J}^{-2}\right) e A , F = o ( J L N , J − 2 ) , we have J − 1 / 2 L N , J e A , F = o ( L N , J − 1 ) = o ( 1 ) J^{-1/2}L_{N,J}e_{A,F}=o(L_{N,J}^{-1})=o(1) J − 1/2 L N , J e A , F = o ( L N , J − 1 ) = o ( 1 ) . Hence,
∥ B ~ − B ∗ ∥ 2 → ∞ = o p ( 1 ) . (78) \|\widetilde{B}-B^*\|_{2\to\infty}=o_p(1). \tag*{(78)} ∥ B − B ∗ ∥ 2 → ∞ = o p ( 1 ) . (78)
By the fixed-slack construction above, ∥ B ∗ ∥ 2 → ∞ ≤ C 1 − η B \|B^*\|_{2\to\infty}\le C_1-\eta_B ∥ B ∗ ∥ 2 → ∞ ≤ C 1 − η B . Together with (77), this implies
Pr ( ∥ B ~ − B ∗ ∥ 2 → ∞ ≤ η B ) → 1. \Pr\left(\|\widetilde{B}-B^*\|_{2\to\infty}\le\eta_B\right)\to1. Pr ( ∥ B − B ∗ ∥ 2 → ∞ ≤ η B ) → 1.
Hence, with probability tending to one,
∥ B ~ ∥ 2 → ∞ ≤ ∥ B ∗ ∥ 2 → ∞ + ∥ B ~ − B ∗ ∥ 2 → ∞ ≤ ( C 1 − η B ) + η B = C 1 . (79) \lVert\widetilde{\mathcal{B}}\rVert_{2\to\infty} \le\lVert\mathcal{B}^{*}\rVert_{2\to\infty} + \lVert\widetilde{\mathcal{B}}-\mathcal{B}^{*}\rVert_{2\to\infty}
\le(C_{1}-\eta_{B})+\eta_{B}
= C_{1}. \tag*{(79)} ∥ B ∥ 2 → ∞ ≤ ∥ B ∗ ∥ 2 → ∞ + ∥ B − B ∗ ∥ 2 → ∞ ≤ ( C 1 − η B ) + η B = C 1 . (79)
Furthermore,
∥ B ~ − B ∗ ∥ F ≤ N ∥ B ~ − B ∗ ∥ 2 → ∞ = O p ( N J − 1 / 2 L N , J ℓ A , F ) . (80) \lVert\widetilde{\mathcal{B}}-\mathcal{B}^{*}\rVert_{F} \le\sqrt{N}\lVert\widetilde{\mathcal{B}}-\mathcal{B}^{*}\rVert_{2\to\infty}
= O_{p}\left(\sqrt{N}J^{-1/2}L_{N,J}\ell_{\mathcal{A},F}\right). \tag*{(80)} ∥ B − B ∗ ∥ F ≤ N ∥ B − B ∗ ∥ 2 → ∞ = O p ( N J − 1/2 L N , J ℓ A , F ) . (80)
Applying Lemma 5 to E ⊤ \mathcal{E}^{\top} E ⊤ and interchanging the roles of ( A , J ) (\mathcal{A},J) ( A , J ) and ( B , N ) (\mathcal{B},N) ( B , N ) gives
max j ∈ [ J ] ∥ E j ⊤ B ~ ∥ = O p ( N L N , J + N L N , J ∥ B ~ − B ∗ ∥ F ) = O p ( N L N , J + N J − 1 / 2 L N , J 2 ℓ A , F ) . (81) \max_{j\in[J]}\lVert\mathcal{E}_{j}^{\top}\widetilde{\mathcal{B}}\rVert
= O_{p}\left(\sqrt{N}L_{N,J}+\sqrt{N}L_{N,J}\lVert\widetilde{\mathcal{B}}-\mathcal{B}^{*}\rVert_{F}\right)
= O_{p}\left(\sqrt{N}L_{N,J}+NJ^{-1/2}L_{N,J}^{2}\ell_{\mathcal{A},F}\right). \tag*{(81)} j ∈ [ J ] max ∥ E j ⊤ B ∥ = O p ( N L N , J + N L N , J ∥ B − B ∗ ∥ F ) = O p ( N L N , J + N J − 1/2 L N , J 2 ℓ A , F ) . (81)
By the column-wise analogue of Lemma 6,
max j ∈ [ J ] ∥ B 2 , j ( B ~ ) ∥ = O p ( N J − 1 / 2 L N , J ℓ A , F ) . (82) \max_{j\in[J]}\lVert\mathbf{B}_{2,j}(\widetilde{\mathcal{B}})\rVert
= O_{p}\left(NJ^{-1/2}L_{N,J}\ell_{\mathcal{A},F}\right). \tag*{(82)} j ∈ [ J ] max ∥ B 2 , j ( B )∥ = O p ( N J − 1/2 L N , J ℓ A , F ) . (82)
Similarly, the column-wise analogue of Lemma 7 gives
max j ∈ [ J ] β 2 , j ( B ~ ) = O p ( N J − 1 L N , J 2 ℓ A , F 2 ) . (83) \max_{j\in[J]}\beta_{2,j}(\widetilde{\mathcal{B}})
= O_{p}\left(NJ^{-1}L_{N,J}^{2}\ell_{\mathcal{A},F}^{2}\right). \tag*{(83)} j ∈ [ J ] max β 2 , j ( B ) = O p ( N J − 1 L N , J 2 ℓ A , F 2 ) . (83)
Also, by the column-wise analogue of Lemma 8,
max j ∈ [ J ] γ 2 , j ( B ~ ) ≲ N (84) \max_{j\in[J]}\gamma_{2,j}(\widetilde{\mathcal{B}})\lesssim N \tag*{(84)} j ∈ [ J ] max γ 2 , j ( B ) ≲ N (84)
with probability tending to one.
For each j ∈ [ J ] j \in[J] j ∈ [ J ] , define
T 2 , j : = ∥ E ⋅ j ⊤ B ~ ∥ + ∥ B 2 , j ( B ~ ) ∥ + β 2 , j ( B ~ ) κ 2 ∗ . T_{2,j} := \|\mathcal{E}_{\cdot j}^{\top}\widetilde{\mathcal{B}}\| + \|\mathcal{B}_{2,j}(\widetilde{\mathcal{B}})\| + \beta_{2,j}(\widetilde{\mathcal{B}})\kappa_{2}^{*}. T 2 , j := ∥ E ⋅ j ⊤ B ∥ + ∥ B 2 , j ( B ) ∥ + β 2 , j ( B ) κ 2 ∗ .
Combining (80)–(82), we obtain
max j ∈ [ J ] T 2 , j = O p ( N L N , J + N J − 1 / 2 L N , J 2 e A , F + N J − 1 / 2 L N , J e A , F + N J − 1 L N , J 2 e A , F 2 ) . \max_{j \in[J]} T_{2,j} = O_p\left(\sqrt{N}L_{N,J} + NJ^{-1/2}L_{N,J}^{2}e_{\mathcal{A},F} + NJ^{-1/2}L_{N,J}e_{\mathcal{A},F} + NJ^{-1}L_{N,J}^{2}e_{\mathcal{A},F}^{2}\right). j ∈ [ J ] max T 2 , j = O p ( N L N , J + N J − 1/2 L N , J 2 e A , F + N J − 1/2 L N , J e A , F + N J − 1 L N , J 2 e A , F 2 ) .
Since N ≥ J N \ge J N ≥ J , L N , J = log ( N J ) ≤ 2 log N = o ( N ) L_{N,J} = \log(NJ) \le2\log N = o(\sqrt{N}) L N , J = log ( N J ) ≤ 2 log N = o ( N ) , and therefore N L N , J = o ( N ) \sqrt{N}L_{N,J} = o(N) N L N , J = o ( N ) . Moreover, by e A , F = o ( J L N , J − 2 ) e_{\mathcal{A},F} = o(\sqrt{J}L_{N,J}^{-2}) e A , F = o ( J L N , J − 2 ) , N J − 1 / 2 L N , J 2 e A , F = o ( N ) NJ^{-1/2}L_{N,J}^{2}e_{\mathcal{A},F} = o(N) N J − 1/2 L N , J 2 e A , F = o ( N ) . The remaining two terms are also o ( N ) o(N) o ( N ) . Consequently,
max j ∈ [ J ] T 2 , j = o p ( N ) . (85) \max_{j \in[J]} T_{2,j} = o_p(N). \tag*{(85)} j ∈ [ J ] max T 2 , j = o p ( N ) . (85)
We next control the smallest singular value of B ~ \widetilde{\mathcal{B}} B . Since B ∗ = J − 1 / 2 U ∗ D ∗ P ^ \mathcal{B}^{*} = J^{-1/2}U^{*}D^{*}\widehat{P} B ∗ = J − 1/2 U ∗ D ∗ P , we have σ K + 2 ( B ∗ ) = J − 1 / 2 σ K + 2 ( M ∗ ) \sigma_{K+2}(\mathcal{B}^{*}) = J^{-1/2}\sigma_{K+2}(M^{*}) σ K + 2 ( B ∗ ) = J − 1/2 σ K + 2 ( M ∗ ) . By Assumptions 1.1 and 1.5, σ K + 2 ( M ∗ ) ≍ N J \sigma_{K+2}(M^{*}) \asymp\sqrt{NJ} σ K + 2 ( M ∗ ) ≍ N J , and hence σ K + 2 ( B ∗ ) ≳ N \sigma_{K+2}(\mathcal{B}^{*}) \gtrsim\sqrt{N} σ K + 2 ( B ∗ ) ≳ N . Furthermore, by (79),
∥ B ~ − B ∗ ∥ 2 ≤ ∥ B ~ − B ∗ ∥ F = O p ( N J − 1 / 2 L N , J e A , F ) = o p ( N ) . \begin{aligned}
\|\widetilde{\mathcal{B}}-\mathcal{B}^{*}\|_{2}
\le\|\widetilde{\mathcal{B}}-\mathcal{B}^{*}\|_{F} \\
&= O_p\left(\sqrt{N}J^{-1/2}L_{N,J}e_{\mathcal{A},F}\right) \\
&= o_p(\sqrt{N}).
\end{aligned} ∥ B − B ∗ ∥ 2 ≤ ∥ B − B ∗ ∥ F = O p ( N J − 1/2 L N , J e A , F ) = o p ( N ) .
Therefore, Weyl’s inequality yields
σ K + 2 ( B ~ ) ≳ N (86) \sigma_{K+2}(\widetilde{\mathcal{B}}) \gtrsim\sqrt{N} \tag*{(86)} σ K + 2 ( B ) ≳ N (86)
with probability tending to one. In particular, rank ( B ~ ) = K + 2 \operatorname{rank}(\widetilde{B}) = K + 2 rank ( B ) = K + 2 with probability tending to one.
Since ∣ m i j ∗ ∣ ≤ C 0 |m^*_{ij}| \le C_0 ∣ m ij ∗ ∣ ≤ C 0 , Assumption 1.3 implies f ′ ( m i j ∗ ) ≥ δ 1 ∗ > 0 f'(m^*_{ij}) \ge\delta^*_1 > 0 f ′ ( m ij ∗ ) ≥ δ 1 ∗ > 0 . Thus,
I 2 , j ( B ~ ) = ∑ i = 1 N f ′ ( m i j ∗ ) β ~ i β ~ i ⊤ ⪰ δ 1 ∗ B ~ ⊤ B ~ . \mathcal{I}_{2,j}(\widetilde{B}) = \sum_{i=1}^{N} f'(m^*_{ij})\widetilde{\beta}_i\widetilde{\beta}_i^{\top} \succeq\delta^*_1\widetilde{B}^{\top}\widetilde{B}. I 2 , j ( B ) = i = 1 ∑ N f ′ ( m ij ∗ ) β i β i ⊤ ⪰ δ 1 ∗ B ⊤ B .
Together with (86) , uniformly over j ∈ [ J ] j \in[J] j ∈ [ J ] ,
σ K + 2 ( I 2 , j ( B ~ ) ) ≳ N (87) \sigma_{K+2}\bigl(\mathcal{I}_{2,j}(\widetilde{B})\bigr) \gtrsim N \tag*{(87)} σ K + 2 ( I 2 , j ( B ) ) ≳ N (87)
with probability tending to one.
We now verify the two product-form conditions in the column-wise analogue of (61) . By (84) and (87) ,
max j ∈ [ J ] 2 T 2 , j σ K + 2 ( I 2 , j ( B ~ ) ) C 2 = o p ( 1 ) . \max_{j \in[J]} \frac{2T_{2,j}}{\sigma_{K+2}\bigl(\mathcal{I}_{2,j}(\widetilde{B})\bigr) C_2} = o_p(1). j ∈ [ J ] max σ K + 2 ( I 2 , j ( B ) ) C 2 2 T 2 , j = o p ( 1 ) .
Hence, with probability tending to one, 2 T 2 , j ≤ σ K + 2 ( I 2 , j ( B ~ ) ) C 2 2T_{2,j} \le\sigma_{K+2}\bigl(\mathcal{I}_{2,j}(\widetilde{B})\bigr) C_2 2 T 2 , j ≤ σ K + 2 ( I 2 , j ( B ) ) C 2 simultaneously for all j ∈ [ J ] j \in[J] j ∈ [ J ] .
Furthermore, by (83), (84), and (87) ,
max j ∈ [ J ] 4 γ 2 , j ( B ~ ) κ 2 ∗ T 2 , j σ K + 2 2 ( I 2 , j ( B ~ ) ) = o p ( 1 ) . \max_{j \in[J]} \frac{4\gamma_{2,j}(\widetilde{B})\kappa^*_{2}T_{2,j}}{\sigma^2_{K+2}\bigl(\mathcal{I}_{2,j}(\widetilde{B})\bigr)} = o_p(1). j ∈ [ J ] max σ K + 2 2 ( I 2 , j ( B ) ) 4 γ 2 , j ( B ) κ 2 ∗ T 2 , j = o p ( 1 ) .
Thus, with probability tending to one,
4 γ 2 , j ( B ~ ) κ 2 ∗ T 2 , j ≤ σ K + 2 2 ( I 2 , j ( B ~ ) ) 4\gamma_{2,j}(\widetilde{B})\kappa^*_{2}T_{2,j} \le\sigma^2_{K+2}\bigl(\mathcal{I}_{2,j}(\widetilde{B})\bigr) 4 γ 2 , j ( B ) κ 2 ∗ T 2 , j ≤ σ K + 2 2 ( I 2 , j ( B ) )
simultaneously for all j ∈ [ J ] j \in[J] j ∈ [ J ] .
Therefore, the column-wise analogue of Lemma 3 applies simultaneously to all j ∈ [ J ] j \in[J] j ∈ [ J ] with probability tending to one. Hence, for every j ∈ [ J ] j \in[J] j ∈ [ J ] , there exists a unique α ~ j \widetilde{\alpha}_j α j satisfying
S 2 , j ( α ~ j ; B ~ ) = 0 K + 2 , ∥ α ~ j − α j ∗ ∥ ≤ C 2 . S_{2,j}(\widetilde{\alpha}_j;\widetilde{\mathcal{B}})=\mathbf{0}_{K+2}, \qquad\|\widetilde{\alpha}_j-\alpha_j^*\|\le C_2. S 2 , j ( α j ; B ) = 0 K + 2 , ∥ α j − α j ∗ ∥ ≤ C 2 .
Moreover, ∥ α ~ j − α j ∗ ∥ ≲ N − 1 T 2 , j \|\widetilde{\alpha}_j-\alpha_j^*\|\lesssim N^{-1}T_{2,j} ∥ α j − α j ∗ ∥ ≲ N − 1 T 2 , j . Taking the maximum over j ∈ [ J ] j \in[J] j ∈ [ J ] yields
∥ A ~ − A ∗ ∥ 2 → ∞ = O p ( N − 1 / 2 L N , J + J − 1 / 2 L N , J 2 e A , F + J − 1 / 2 L N , J e A , F + J − 1 L N , J 2 e A , F 2 ) . \|\widetilde{A}-A^*\|_{2\to\infty}=O_p\left(N^{-1/2}L_{N,J}+J^{-1/2}L_{N,J}^2e_{A,F}+J^{-1/2}L_{N,J}e_{A,F}+J^{-1}L_{N,J}^2e_{A,F}^2\right). ∥ A − A ∗ ∥ 2 → ∞ = O p ( N − 1/2 L N , J + J − 1/2 L N , J 2 e A , F + J − 1/2 L N , J e A , F + J − 1 L N , J 2 e A , F 2 ) .
Since L N , J ≥ 1 L_{N,J}\ge1 L N , J ≥ 1 and e A , F = o ( J L N , J − 2 ) e_{A,F}=o(\sqrt{J}L_{N,J}^{-2}) e A , F = o ( J L N , J − 2 ) , the last two terms are dominated by the first two. Therefore,
∥ A ~ − A ∗ ∥ 2 → ∞ = O p ( N − 1 / 2 L N , J + J − 1 / 2 L N , J 2 e A , F ) . \|\widetilde{A}-A^*\|_{2\to\infty}=O_p\left(N^{-1/2}L_{N,J}+J^{-1/2}L_{N,J}^2e_{A,F}\right). ∥ A − A ∗ ∥ 2 → ∞ = O p ( N − 1/2 L N , J + J − 1/2 L N , J 2 e A , F ) .
Since L N , J = log ( N J ) L_{N,J}=\log(NJ) L N , J = log ( N J ) ,
∥ A ~ − A ∗ ∥ 2 → ∞ = O p ( N − 1 / 2 log ( N J ) + J − 1 / 2 ( log ( N J ) ) 2 e A , F ) . \|\widetilde{A}-A^*\|_{2\to\infty}=O_p\left(N^{-1/2}\log(NJ)+J^{-1/2}(\log(NJ))^2e_{A,F}\right). ∥ A − A ∗ ∥ 2 → ∞ = O p ( N − 1/2 log ( N J ) + J − 1/2 ( log ( N J ) ) 2 e A , F ) .
□ \square □
Completion of the Bipartite Proof of Theorem 2
Proof. Let e A , F : = J σ K + 2 − 1 ( M ∗ ) e M , F e_{A,F}:=\sqrt{J}\,\sigma_{K+2}^{-1}(M^*)e_{M,F} e A , F := J σ K + 2 − 1 ( M ∗ ) e M , F .
By Assumptions 1.1 and 1.5, σ K + 2 ( M ∗ ) ≍ N J \sigma_{K+2}(M^*)\asymp\sqrt{NJ} σ K + 2 ( M ∗ ) ≍ N J . Hence,
e A , F ≍ N − 1 / 2 e M , F . (88) e_{A,F}\asymp N^{-1/2}e_{M,F}. \tag*{(88)} e A , F ≍ N − 1/2 e M , F . (88)
We first verify that Lemma 1 is applicable. By the assumption ∥ M i n i t − M ∗ ∥ F = O p ( e M , F ) \|M^{\mathrm{init}}-M^*\|_F=O_p(e_{M,F}) ∥ M init − M ∗ ∥ F = O p ( e M , F ) and e M , F ≪ N J ( log ( N J ) ) − 2 e_{M,F} \ll\sqrt{NJ}(\log(NJ))^{-2} e M , F ≪ N J ( log ( N J ) ) − 2 , we have
∥ M i n i t − M ∗ ∥ F σ K + 2 ( M ∗ ) = O p ( e M , F N J ) = o p ( 1 ) . \frac{\lVert\mathbf{M}^{\mathrm{init}}-\mathbf{M}^*\rVert_F}{\sigma_{K+2}(\mathbf{M}^*)}
= O_p\left(\frac{e_{M,F}}{\sqrt{NJ}}\right)=o_p(1). σ K + 2 ( M ∗ ) ∥ M init − M ∗ ∥ F = O p ( N J e M , F ) = o p ( 1 ) .
Therefore,
Pr ( ∥ M i n i t − M ∗ ∥ F ≤ 1 2 σ K + 2 ( M ∗ ) ) → 1. \Pr\left(\lVert\mathbf{M}^{\mathrm{init}}-\mathbf{M}^*\rVert_F \le\frac{1}{2}\sigma_{K+2}(\mathbf{M}^*)\right)\to1. Pr ( ∥ M init − M ∗ ∥ F ≤ 2 1 σ K + 2 ( M ∗ ) ) → 1.
Lemma 1 thus gives
∥ A ^ − A ∗ ∥ F = O p ( e A , F ) . (89) \lVert\widehat{\mathcal{A}}-\mathcal{A}^*\rVert_F=O_p(e_{A,F}). \tag*{(89)} ∥ A − A ∗ ∥ F = O p ( e A , F ) . (89)
We next verify the rate conditions required by Lemmas 9 and 10. Since N ≲ e M , F \sqrt{N}\lesssim e_{M,F} N ≲ e M , F , (88) implies e A , F = Ω ( 1 ) e_{A,F}=\Omega(1) e A , F = Ω ( 1 ) . Moreover, e M , F ≪ N J ( log ( N J ) ) − 2 e_{M,F}\ll\sqrt{NJ}(\log(NJ))^{-2} e M , F ≪ N J ( log ( N J ) ) − 2 implies
e A , F = o ( J ( log ( N J ) ) − 2 ) . (90) e_{A,F}=o\left(\sqrt{J}(\log(NJ))^{-2}\right). \tag*{(90)} e A , F = o ( J ( log ( N J ) ) − 2 ) . (90)
In particular, e A , F = o ( J ( log ( N J ) ) − 1 ) e_{A,F}=o\left(\sqrt{J}(\log(NJ))^{-1}\right) e A , F = o ( J ( log ( N J ) ) − 1 ) . Therefore, all the conditions of Lemmas 9 and 10 are satisfied.
It follows that
∥ B ~ − B ∗ ∥ 2 → ∞ = O p ( J − 1 / 2 log ( N J ) e A , F ) . (91) \lVert\widetilde{\mathbf{B}}-\mathbf{B}^*\rVert_{2\to\infty}
=O_p\left(J^{-1/2}\log(NJ)e_{A,F}\right). \tag*{(91)} ∥ B − B ∗ ∥ 2 → ∞ = O p ( J − 1/2 log ( N J ) e A , F ) . (91)
and
∥ A ~ − A ∗ ∥ 2 → ∞ = O p ( N − 1 / 2 log ( N J ) + J − 1 / 2 ( log ( N J ) ) 2 e A , F ) . (92) \lVert\widetilde{\mathcal{A}}-\mathcal{A}^*\rVert_{2\to\infty}
=O_p\left(N^{-1/2}\log(NJ)+J^{-1/2}(\log(NJ))^2e_{A,F}\right). \tag*{(92)} ∥ A − A ∗ ∥ 2 → ∞ = O p ( N − 1/2 log ( N J ) + J − 1/2 ( log ( N J ) ) 2 e A , F ) . (92)
By (90) , J − 1 / 2 log ( N J ) e A , F = o ( 1 ) J^{-1/2}\log(NJ)e_{A,F}=o(1) J − 1/2 log ( N J ) e A , F = o ( 1 ) . Together with ∥ B ∗ ∥ 2 → ∞ ≲ 1 \lVert\mathbf{B}^*\rVert_{2\to\infty}\lesssim1 ∥ B ∗ ∥ 2 → ∞ ≲ 1 , (91) yields
∥ B ~ ∥ 2 → ∞ = O p ( 1 ) . (93) \lVert\widetilde{\mathbf{B}}\rVert_{2\to\infty}=O_p(1). \tag*{(93)} ∥ B ∥ 2 → ∞ = O p ( 1 ) . (93)
For bipartite data, by construction,
M ^ r e f = B ~ A ~ ⊤ , M ∗ = B ∗ A ∗ ⊤ . \widehat{\mathbf{M}}^{\mathrm{ref}}=\widetilde{\mathbf{B}}\widetilde{\mathbf{A}}^{\top},\qquad\mathbf{M}^{*}=\mathbf{B}^{*}\mathbf{A}^{*\top}. M ref = B A ⊤ , M ∗ = B ∗ A ∗ ⊤ .
Therefore,
∥ M ^ r e f − M ∗ ∥ max = ∥ B ~ A ~ ⊤ − B ∗ A ∗ ⊤ ∥ max ≤ ∥ ( B ~ − B ∗ ) A ∗ ⊤ ∥ max + ∥ B ~ ( A ~ − A ∗ ) ⊤ ∥ max ≤ ∥ B ~ − B ∗ ∥ 2 → ∞ ∥ A ∗ ∥ 2 → ∞ + ∥ B ~ ∥ 2 → ∞ ∥ A ~ − A ∗ ∥ 2 → ∞ . \begin{aligned}
\left\|\widehat{\mathbf{M}}^{\mathrm{ref}}-\mathbf{M}^{*}\right\|_{\max}
=\left\|\widetilde{\mathbf{B}}\widetilde{\mathbf{A}}^{\top}-\mathbf{B}^{*}\mathbf{A}^{*\top}\right\|_{\max} \\
&\leq\left\|(\widetilde{\mathbf{B}}-\mathbf{B}^{*})\mathbf{A}^{*\top}\right\|_{\max}
+\left\|\widetilde{\mathbf{B}}(\widetilde{\mathbf{A}}-\mathbf{A}^{*})^{\top}\right\|_{\max} \\
&\leq\left\|\widetilde{\mathbf{B}}-\mathbf{B}^{*}\right\|_{2\to\infty}\left\|\mathbf{A}^{*}\right\|_{2\to\infty}
+\left\|\widetilde{\mathbf{B}}\right\|_{2\to\infty}\left\|\widetilde{\mathbf{A}}-\mathbf{A}^{*}\right\|_{2\to\infty}.
\end{aligned} M ref − M ∗ m a x = B A ⊤ − B ∗ A ∗ ⊤ m a x ≤ ( B − B ∗ ) A ∗ ⊤ m a x + B ( A − A ∗ ) ⊤ m a x ≤ B − B ∗ 2 → ∞ ∥ A ∗ ∥ 2 → ∞ + B 2 → ∞ A − A ∗ 2 → ∞ .
Since ∥ A ∗ ∥ 2 → ∞ ≲ 1 \left\|\mathbf{A}^{*}\right\|_{2\to\infty}\lesssim1 ∥ A ∗ ∥ 2 → ∞ ≲ 1 , combining (90)–(92) gives
∥ M ^ r e f − M ∗ ∥ max = O p ( J − 1 / 2 log ( N J ) e A , F + N − 1 / 2 log ( N J ) + J − 1 / 2 ( log ( N J ) ) 2 e A , F ) . \left\|\widehat{\mathbf{M}}^{\mathrm{ref}}-\mathbf{M}^{*}\right\|_{\max}
=O_p\left(
J^{-1/2}\log(NJ)e_{A,F}
+N^{-1/2}\log(NJ)
+J^{-1/2}\bigl(\log(NJ)\bigr)^2e_{A,F}
\right). M ref − M ∗ m a x = O p ( J − 1/2 log ( N J ) e A , F + N − 1/2 log ( N J ) + J − 1/2 ( log ( N J ) ) 2 e A , F ) .
The first term is dominated by the third term. Moreover, by (87),
J − 1 / 2 ( log ( N J ) ) 2 e A , F ≍ ( N J ) − 1 / 2 ( log ( N J ) ) 2 e M , F . J^{-1/2}\bigl(\log(NJ)\bigr)^2e_{A,F}
\asymp(NJ)^{-1/2}\bigl(\log(NJ)\bigr)^2e_{M,F}. J − 1/2 ( log ( N J ) ) 2 e A , F ≍ ( N J ) − 1/2 ( log ( N J ) ) 2 e M , F .
For the remaining term, since e M , F ≳ N e_{M,F}\gtrsim\sqrt{N} e M , F ≳ N and N ≥ J N\geq J N ≥ J ,
N − 1 / 2 log ( N J ) ≲ ( N J ) − 1 / 2 log ( N J ) e M , F ≤ ( N J ) − 1 / 2 ( log ( N J ) ) 2 e M , F . \begin{aligned}
N^{-1/2}\log(NJ)\lesssim(NJ)^{-1/2}\log(NJ)e_{M,F} \\
&\leq(NJ)^{-1/2}\bigl(\log(NJ)\bigr)^2e_{M,F}.
\end{aligned} N − 1/2 log ( N J ) ≲ ( N J ) − 1/2 log ( N J ) e M , F ≤ ( N J ) − 1/2 ( log ( N J ) ) 2 e M , F .
Consequently,
∥ M ^ r e f − M ∗ ∥ max = O p ( ( N J ) − 1 / 2 ( log ( N J ) ) 2 e M , F ) . \left\|\widehat{\mathbf{M}}^{\mathrm{ref}}-\mathbf{M}^{*}\right\|_{\max}
=O_p\left((NJ)^{-1/2}\bigl(\log(NJ)\bigr)^2e_{M,F}\right). M ref − M ∗ m a x = O p ( ( N J ) − 1/2 ( log ( N J ) ) 2 e M , F ) .
This proves Theorem 2 for the bipartite case.
Unipartite One-Step Refinement
We now prove Theorem 2 for unipartite data. The argument follows the same factor-refinement logic as in Section B.2, with three structural modifications: diagonal observations are excluded from the estimating equations, the observation matrix is symmetric, and the final estimator is symmetrized and truncated to rank K + 2 K+2 K + 2 .
Let r : = K + 2 r := K+2 r := K + 2 , L N : = log ( N 2 ) L_N := \log(N^2) L N := log ( N 2 ) , and define, as in the bipartite proof,
P ^ ∈ arg min P ∈ O r ∥ V ^ − V ∗ P ∥ F , \widehat{P} \in\underset{P\in\mathcal{O}_r}{\arg\min}\ \lVert\widehat{\mathcal{V}}-\mathcal{V}^*P\rVert_F, P ∈ P ∈ O r arg min ∥ V − V ∗ P ∥ F ,
together with the aligned factors
A ∗ : = N V ∗ P ^ , B ∗ : = N − 1 / 2 U ∗ D ∗ P ^ . \mathcal{A}^* := \sqrt{N}\mathcal{V}^*\widehat{P}, \qquad\mathcal{B}^* := N^{-1/2}\mathcal{U}^*D^*\widehat{P}. A ∗ := N V ∗ P , B ∗ := N − 1/2 U ∗ D ∗ P .
Then M ∗ = B ∗ A ∗ ⊤ \mathbf{M}^*=\mathcal{B}^*\mathcal{A}^{*\top} M ∗ = B ∗ A ∗ ⊤ . By Assumptions 1.1 and 1.4,
∥ A ∗ ∥ 2 → ∞ ≲ 1 , ∥ B ∗ ∥ 2 → ∞ ≲ 1. \lVert\mathcal{A}^*\rVert_{2\to\infty}\lesssim1,\qquad\lVert\mathcal{B}^*\rVert_{2\to\infty}\lesssim1. ∥ A ∗ ∥ 2 → ∞ ≲ 1 , ∥ B ∗ ∥ 2 → ∞ ≲ 1.
Define e A , F : = N σ r − 1 ( M ∗ ) e M , F e_{A,F}:=\sqrt{N}\sigma_r^{-1}(\mathbf{M}^*)e_{M,F} e A , F := N σ r − 1 ( M ∗ ) e M , F . Since J = N J=N J = N , Assumptions 1.1 and 1.5 imply σ r ( M ∗ ) ≍ N \sigma_r(\mathbf{M}^*)\asymp N σ r ( M ∗ ) ≍ N , and hence
e A , F ≍ N − 1 / 2 e M , F . (94) e_{A,F}\asymp N^{-1/2}e_{M,F}. \tag*{(94)} e A , F ≍ N − 1/2 e M , F . (94)
The assumptions on e M , F e_{M,F} e M , F therefore give
e A , F = Ω ( 1 ) , e A , F = o ( N L N − 2 ) . (95) e_{A,F}=\Omega(1),\qquad e_{A,F}=o\left(\sqrt{N}L_N^{-2}\right). \tag*{(95)} e A , F = Ω ( 1 ) , e A , F = o ( N L N − 2 ) . (95)
Moreover,
∥ M i n i t − M ∗ ∥ F σ r ( M ∗ ) = O p ( e M , F N ) = o p ( 1 ) . \frac{\lVert\mathbf{M}^{\mathrm{init}}-\mathbf{M}^{*}\rVert_{F}}{\sigma_{r}(\mathbf{M}^{*})}
= O_{p}\left(\frac{e_{\mathrm{M},F}}{N}\right)=o_{p}(1). σ r ( M ∗ ) ∥ M init − M ∗ ∥ F = O p ( N e M , F ) = o p ( 1 ) .
Thus, with probability tending to one, ∥ M i n i t − M ∗ ∥ F ≤ 1 2 σ r ( M ∗ ) \lVert\mathbf{M}^{\mathrm{init}}-\mathbf{M}^{*}\rVert_{F}\leq\frac{1}{2}\sigma_{r}(\mathbf{M}^{*}) ∥ M init − M ∗ ∥ F ≤ 2 1 σ r ( M ∗ ) . Applying Lemma 1 with J = N J=N J = N gives
∥ A ^ − A ∗ ∥ F = O p ( e A , F ) . (96) \lVert\widehat{\mathbf{A}}-\mathbf{A}^{*}\rVert_{F}=O_{p}(e_{\mathrm{A},F}). \tag*{(96)} ∥ A − A ∗ ∥ F = O p ( e A , F ) . (96)
We next consider the two refinement steps. The deterministic arguments in Lemmas 2 and 3 remain valid after deleting the single diagonal summand from each estimating equation. The probabilistic arguments in Lemmas 5–8 and in the proofs of Lemmas 9 and 10 also remain valid with J = N J=N J = N . Indeed, for each fixed row i i i , the collection { y i j − f ( m i j ∗ ) : j ≠ i } \{y_{ij}-f(m^{*}_{ij}):j\ne i\} { y ij − f ( m ij ∗ ) : j = i } consists of independent centered sub-exponential random variables, and the same is true for each fixed column j j j . Dependence across different rows or columns caused by symmetry does not affect these arguments, since concentration is applied to each fixed row or column and the resulting bounds are combined by a union bound. We give below the modifications needed to account for the omitted diagonal terms.
For the row refinement, define
S 1 , i u n i ( β i ; A ) : = ∑ j ≠ i { y i j − f ( α j T β i ) } α j . S^{\mathrm{uni}}_{1,i}(\boldsymbol{\beta}_{i};\mathbf{A})
:=\sum_{j\ne i}\{y_{ij}-f(\boldsymbol{\alpha}_{j}^{\mathsf T}\boldsymbol{\beta}_{i})\}\boldsymbol{\alpha}_{j}. S 1 , i uni ( β i ; A ) := j = i ∑ { y ij − f ( α j T β i )} α j .
The analogues of Lemmas 5 –8 , together with (96) , give
max i ∈ [ N ] ∥ ∑ j ≠ i { y i j − f ( m i j ∗ ) } α ^ j ∥ = O p ( N L N e A , F ) , max i ∈ [ N ] ∥ B 1 , i u n i ( A ^ ) ∥ = O p ( N e A , F ) , max i ∈ [ N ] β 1 , i u n i ( A ^ ) = O p ( e A , F 2 ) , max i ∈ [ N ] γ 1 , i u n i ( A ^ ) ≲ N . (97) \begin{aligned}
\max_{i\in[N]}\left\|\sum_{j\ne i}\{y_{ij}-f(m^*_{ij})\}\widehat{\alpha}_j\right\| &= O_p\left(\sqrt{N}L_N e_{A,F}\right),\\
\max_{i\in[N]}\left\|\mathbf{B}^{\mathrm{uni}}_{1,i}(\widehat{\mathcal{A}})\right\| &= O_p\left(\sqrt{N}e_{A,F}\right),\\
\max_{i\in[N]}\beta^{\mathrm{uni}}_{1,i}(\widehat{\mathcal{A}}) &= O_p\left(e_{A,F}^2\right),\\
\max_{i\in[N]}\gamma^{\mathrm{uni}}_{1,i}(\widehat{\mathcal{A}}) &\lesssim N. \tag*{(97)}
\end{aligned} i ∈ [ N ] max j = i ∑ { y ij − f ( m ij ∗ )} α j i ∈ [ N ] max B 1 , i uni ( A ) i ∈ [ N ] max β 1 , i uni ( A ) i ∈ [ N ] max γ 1 , i uni ( A ) = O p ( N L N e A , F ) , = O p ( N e A , F ) , = O p ( e A , F 2 ) , ≲ N . (97)
where the quantities with superscript uni are defined as their bipartite counterparts with the summation restricted to j ≠ i j\ne i j = i .
It remains to check that deleting the i i i th summand does not destroy the information lower bound. By (96) , (95) , and Weyl’s inequality, σ r ( A ^ ) ≍ N \sigma_r(\widehat{\mathcal{A}})\asymp\sqrt{N} σ r ( A ) ≍ N with probability tending to one. Since ∥ A ^ ∥ 2 → ∞ ≤ C 2 \|\widehat{\mathcal{A}}\|_{2\to\infty}\le C_2 ∥ A ∥ 2 → ∞ ≤ C 2 by construction, uniformly over i ∈ [ N ] i\in[N] i ∈ [ N ] ,
λ min ( ∑ j ≠ i α ^ j α ^ j T ) = λ min ( A ^ T A ^ − α ^ i α ^ i T ) ≥ σ r 2 ( A ^ ) − ∥ α ^ i ∥ 2 ≳ N . \lambda_{\min}\left(\sum_{j\ne i}\widehat{\alpha}_j\widehat{\alpha}_j^{\mathsf T}\right)
=\lambda_{\min}\left(\widehat{\mathcal{A}}^{\mathsf T}\widehat{\mathcal{A}}-\widehat{\alpha}_i\widehat{\alpha}_i^{\mathsf T}\right)
\ge\sigma_r^2(\widehat{\mathcal{A}})-\|\widehat{\alpha}_i\|^2\gtrsim N. λ m i n j = i ∑ α j α j T = λ m i n ( A T A − α i α i T ) ≥ σ r 2 ( A ) − ∥ α i ∥ 2 ≳ N .
Since f ′ ( m i j ∗ ) ≥ δ 1 ∗ > 0 f'(m^*_{ij})\ge\delta_1^*>0 f ′ ( m ij ∗ ) ≥ δ 1 ∗ > 0 ,
min i ∈ [ N ] σ r ( T 1 , i u n i ( A ^ ) ) ≳ N (98) \min_{i\in[N]}\sigma_r\left(\mathcal{T}^{\mathrm{uni}}_{1,i}(\widehat{\mathcal{A}})\right)\gtrsim N \tag*{(98)} i ∈ [ N ] min σ r ( T 1 , i uni ( A ) ) ≳ N (98)
with probability tending to one.
Combining (97) , (98) , and (95) verifies the two conditions in the unipartite analogue of (61) , uniformly over i ∈ [ N ] i\in[N] i ∈ [ N ] . Hence, with probability tending to one, every row-wise estimating equation admits a unique solution, and the same argument as in Lemma 9 yields
∥ B ~ − B ∗ ∥ 2 → ∞ = O p ( N − 1 / 2 L N e A , F ) . (99) \|\widetilde{\mathcal{B}}-\mathcal{B}^*\|_{2\to\infty}
=O_p\left(N^{-1/2}L_Ne_{A,F}\right). \tag*{(99)} ∥ B − B ∗ ∥ 2 → ∞ = O p ( N − 1/2 L N e A , F ) . (99)
In view of (94), ∥ B ~ − B ∗ ∥ 2 → ∞ = o p ( 1 ) \lVert\widetilde{B}-B^*\rVert_{2\to\infty}=o_p(1) ∥ B − B ∗ ∥ 2 → ∞ = o p ( 1 ) . Using the fixed slack in the choice of C 1 C_1 C 1 from the bipartite proof, we may therefore work on an event whose probability tends to one on which
∥ B ~ ∥ 2 → ∞ ≤ C 1 . (100) \lVert\widetilde{B}\rVert_{2\to\infty}\le C_1. \tag*{(100)} ∥ B ∥ 2 → ∞ ≤ C 1 . (100)
Furthermore,
∥ B ~ − B ∗ ∥ F ≤ N ∥ B ~ − B ∗ ∥ 2 → ∞ = O p ( L N e A , F ) . (101) \lVert\widetilde{B}-B^*\rVert_F\le\sqrt{N}\lVert\widetilde{B}-B^*\rVert_{2\to\infty}=O_p(L_N e_{A,F}). \tag*{(101)} ∥ B − B ∗ ∥ F ≤ N ∥ B − B ∗ ∥ 2 → ∞ = O p ( L N e A , F ) . (101)
For the column refinement, define
S 2 , j u n i ( α j ; B ) : = ∑ i ≠ j { y i j − f ( α j T β i ) } β i . S^{\mathrm{uni}}_{2,j}(\alpha_j;B):=\sum_{i\ne j}\{y_{ij}-f(\alpha_j^\mathsf{T}\beta_i)\}\beta_i. S 2 , j uni ( α j ; B ) := i = j ∑ { y ij − f ( α j T β i )} β i .
Repeating the proof of Lemma 10, with the diagonal summand omitted and using (100), gives
max j ∈ [ N ] ∥ ∑ i ≠ j { y i j − f ( m i j ∗ ) } β ~ i ∥ = O p ( N L N + N L N 2 e A , F 2 ) , max j ∈ [ N ] ∥ B 2 , j u n i ( B ~ ) ∥ = O p ( N L N e A , F ) , max j ∈ [ N ] β 2 , j u n i ( B ~ ) = O p ( L N 2 e A , F 2 ) . (102) \begin{aligned}
\max_{j\in[N]}\left\lVert\sum_{i\ne j}\{y_{ij}-f(m^*_{ij})\}\widetilde{\beta}_i\right\rVert
&=O_p\left(\sqrt{N}L_N+\sqrt{N}L_N^2e_{A,F}^2\right),\\
\max_{j\in[N]}\lVert B^{\mathrm{uni}}_{2,j}(\widetilde{B})\rVert
&=O_p\left(\sqrt{N}L_Ne_{A,F}\right),\\
\max_{j\in[N]}\beta^{\mathrm{uni}}_{2,j}(\widetilde{B})
&=O_p\left(L_N^2e_{A,F}^2\right). \tag*{(102)}
\end{aligned} j ∈ [ N ] max i = j ∑ { y ij − f ( m ij ∗ )} β i j ∈ [ N ] max ∥ B 2 , j uni ( B )∥ j ∈ [ N ] max β 2 , j uni ( B ) = O p ( N L N + N L N 2 e A , F 2 ) , = O p ( N L N e A , F ) , = O p ( L N 2 e A , F 2 ) . (102)
On the event (99), the column-wise analogue of Lemma 8 further gives
max j ∈ [ N ] γ 2 , j u n i ( B ~ ) ≲ N . \max_{j\in[N]}\gamma^{\mathrm{uni}}_{2,j}(\widetilde{B})\lesssim N. j ∈ [ N ] max γ 2 , j uni ( B ) ≲ N .
Also, σ r ( B ∗ ) = N − 1 / 2 σ r ( M ∗ ) ≍ N \sigma_r(B^*)=N^{-1/2}\sigma_r(M^*)\asymp\sqrt{N} σ r ( B ∗ ) = N − 1/2 σ r ( M ∗ ) ≍ N . By (100) and (94),
∥ B ~ − B ∗ ∥ 2 ≤ ∥ B ~ − B ∗ ∥ F = o p ( N ) , \lVert\widetilde{B}-B^*\rVert_2\le\lVert\widetilde{B}-B^*\rVert_F=o_p(\sqrt{N}), ∥ B − B ∗ ∥ 2 ≤ ∥ B − B ∗ ∥ F = o p ( N ) ,
and hence σ r ( B ~ ) ≲ N \sigma_r(\widetilde{B})\lesssim\sqrt{N} σ r ( B ) ≲ N with probability tending to one. Together with (99), this yields, uniformly over j ∈ [ N ] j \in[N] j ∈ [ N ] ,
λ min ( ∑ i ≠ j β ~ i β ~ i T ) = λ min ( B ~ T B ~ − β ~ j β ~ j T ) ≥ σ r 2 ( B ~ ) − ∥ β ~ j ∥ 2 ≳ N . \lambda_{\min}\left(\sum_{i\ne j}\widetilde{\beta}_i\widetilde{\beta}_i^{\mathsf T}\right)=\lambda_{\min}\left(\widetilde{\mathcal{B}}^{\mathsf T}\widetilde{\mathcal{B}}-\widetilde{\beta}_j\widetilde{\beta}_j^{\mathsf T}\right)
\ge\sigma_r^2(\widetilde{\mathcal{B}})-\|\widetilde{\beta}_j\|^2 \gtrsim N. λ m i n i = j ∑ β i β i T = λ m i n ( B T B − β j β j T ) ≥ σ r 2 ( B ) − ∥ β j ∥ 2 ≳ N .
Therefore,
min j ∈ [ N ] σ r ( T 2 , j u n i ( B ~ ) ) ≳ N (103) \min_{j\in[N]}\sigma_r\left(\mathcal{T}_{2,j}^{\mathrm{uni}}(\widetilde{\mathcal{B}})\right)\gtrsim N \tag*{(103)} j ∈ [ N ] min σ r ( T 2 , j uni ( B ) ) ≳ N (103)
with probability tending to one.
Equations (102) , (103) , and (95) verify the column-wise analogues of the conditions in (61) .
Thus, with probability tending to one, every column-wise estimating equation admits a unique solution, and the same argument as in Lemma 10 gives
∥ A ~ − A ∗ ∥ 2 → ∞ = O p ( N − 1 / 2 L N + N − 1 / 2 L N 2 ϵ A , F ) . (104) \|\widetilde{\mathcal{A}}-\mathcal{A}^*\|_{2\to\infty}
=O_p\left(N^{-1/2}L_N+N^{-1/2}L_N^2\epsilon_{A,F}\right). \tag*{(104)} ∥ A − A ∗ ∥ 2 → ∞ = O p ( N − 1/2 L N + N − 1/2 L N 2 ϵ A , F ) . (104)
We first consider the estimator before the final rank truncation. Let M ~ 0 : = B ~ A ~ T \widetilde{\mathbf{M}}_0:=\widetilde{\mathcal{B}}\widetilde{\mathcal{A}}^{\mathsf T} M 0 := B A T . Since
M ∗ = B ∗ A ∗ T , \mathbf{M}^*=\mathcal{B}^*\mathcal{A}^{*\mathsf T}, M ∗ = B ∗ A ∗ T ,
∥ M ~ 0 − M ∗ ∥ max ≤ ∥ B ~ − B ∗ ∥ 2 → ∞ ∥ A ∗ ∥ 2 → ∞ + ∥ B ~ ∥ 2 → ∞ ∥ A ~ − A ∗ ∥ 2 → ∞ . \|\widetilde{\mathbf{M}}_0-\mathbf{M}^*\|_{\max}
\le\|\widetilde{\mathcal{B}}-\mathcal{B}^*\|_{2\to\infty}\|\mathcal{A}^*\|_{2\to\infty}
+\|\widetilde{\mathcal{B}}\|_{2\to\infty}\|\widetilde{\mathcal{A}}-\mathcal{A}^*\|_{2\to\infty}. ∥ M 0 − M ∗ ∥ m a x ≤ ∥ B − B ∗ ∥ 2 → ∞ ∥ A ∗ ∥ 2 → ∞ + ∥ B ∥ 2 → ∞ ∥ A − A ∗ ∥ 2 → ∞ .
Combining (99) , (100) , and (104) yields
∥ M ~ 0 − M ∗ ∥ max = O p ( N − 1 / 2 L N + N − 1 / 2 L N 2 ϵ A , F ) . \|\widetilde{\mathbf{M}}_0-\mathbf{M}^*\|_{\max}
=O_p\left(N^{-1/2}L_N+N^{-1/2}L_N^2\epsilon_{A,F}\right). ∥ M 0 − M ∗ ∥ m a x = O p ( N − 1/2 L N + N − 1/2 L N 2 ϵ A , F ) .
By (94) , N − 1 / 2 L N 2 ϵ A , F ≍ N − 1 L N 2 ϵ M , F N^{-1/2}L_N^2\epsilon_{A,F}\asymp N^{-1}L_N^2\epsilon_{M,F} N − 1/2 L N 2 ϵ A , F ≍ N − 1 L N 2 ϵ M , F , whereas ϵ M , F ≳ N \epsilon_{M,F}\gtrsim\sqrt{N} ϵ M , F ≳ N implies N − 1 / 2 L N ≲ N − 1 L N ϵ M , F N^{-1/2}L_N\lesssim N^{-1}L_N\epsilon_{M,F} N − 1/2 L N ≲ N − 1 L N ϵ M , F . Consequently,
∥ M ~ 0 − M ∗ ∥ max = O p ( ϵ N ) , ϵ N : = N − 1 L N 2 ϵ M , F = o ( 1 ) . (105) \|\widetilde{\mathbf{M}}_0-\mathbf{M}^*\|_{\max}=O_p(\epsilon_N),\qquad
\epsilon_N:=N^{-1}L_N^2\epsilon_{M,F}=o(1). \tag*{(105)} ∥ M 0 − M ∗ ∥ m a x = O p ( ϵ N ) , ϵ N := N − 1 L N 2 ϵ M , F = o ( 1 ) . (105)
Define the symmetrized estimator
M ‾ : = B ~ A ~ ⊤ + A ~ B ~ ⊤ 2 . \overline{\mathbf{M}} := \frac{\widetilde{\mathbf{B}}\widetilde{\mathbf{A}}^{\top}+\widetilde{\mathbf{A}}\widetilde{\mathbf{B}}^{\top}}{2}. M := 2 B A ⊤ + A B ⊤ .
Since M ∗ \mathbf{M}^{*} M ∗ is symmetric,
∥ M ‾ − M ∗ ∥ max ≤ ∥ M ~ 0 − M ∗ ∥ max = O p ( ε N ) . (106) \left\|\overline{\mathbf{M}}-\mathbf{M}^{*}\right\|_{\max}\leq\left\|\widetilde{\mathbf{M}}_{0}-\mathbf{M}^{*}\right\|_{\max}=O_{p}(\varepsilon_{N}). \tag*{(106)} M − M ∗ m a x ≤ M 0 − M ∗ m a x = O p ( ε N ) . (106)
It remains to verify that the final rank-r r r truncation preserves this entrywise rate. We use the following deterministic lemma.
Lemma 11. Let M , M ~ ∈ R N × N \mathbf{M},\widetilde{\mathbf{M}}\in\mathbb{R}^{N\times N} M , M ∈ R N × N be symmetric and suppose that rank ( M ) = r \operatorname{rank}(\mathbf{M})=r rank ( M ) = r , where r r r is fixed. Assume
σ r ( M ) ≳ N , ∥ M ∥ 2 ≲ N . \sigma_{r}(\mathbf{M})\gtrsim N,\qquad\|\mathbf{M}\|_{2}\lesssim N. σ r ( M ) ≳ N , ∥ M ∥ 2 ≲ N . Let U ∈ R N × r \mathbf{U}\in\mathbb{R}^{N\times r} U ∈ R N × r contain orthonormal eigenvectors spanning the nonzero eigenspace of M \mathbf{M} M , and suppose
∥ U ∥ 2 → ∞ ≲ N − 1 / 2 . \|\mathbf{U}\|_{2\to\infty}\lesssim N^{-1/2}. ∥ U ∥ 2 → ∞ ≲ N − 1/2 . If, for some deterministic sequence ε N = o ( 1 ) \varepsilon_{N}=o(1) ε N = o ( 1 ) ,
∥ M ~ − M ∥ max ≲ ε N , \|\widetilde{\mathbf{M}}-\mathbf{M}\|_{\max}\lesssim\varepsilon_{N}, ∥ M − M ∥ m a x ≲ ε N , then
∥ SVD r ( M ~ ) − M ∥ max ≲ ε N . \|\operatorname{SVD}_{r}(\widetilde{\mathbf{M}})-\mathbf{M}\|_{\max}\lesssim\varepsilon_{N}. ∥ SVD r ( M ) − M ∥ m a x ≲ ε N .
Proof. Let Δ M : = M ~ − M \Delta_{\mathbf{M}}:=\widetilde{\mathbf{M}}-\mathbf{M} Δ M := M − M . Then
∥ Δ M ∥ 2 ≤ ∥ Δ M ∥ F ≤ N ∥ Δ M ∥ max ≲ N ε N = o ( N ) . \|\Delta_{\mathbf{M}}\|_{2}\leq\|\Delta_{\mathbf{M}}\|_{F}\leq N\|\Delta_{\mathbf{M}}\|_{\max}\lesssim N\varepsilon_{N}=o(N). ∥ Δ M ∥ 2 ≤ ∥ Δ M ∥ F ≤ N ∥ Δ M ∥ m a x ≲ N ε N = o ( N ) .
By Weyl’s inequality, the r r r singular values of M ~ \widetilde{M} M corresponding to the signal are of order N N N , whereas σ r + 1 ( M ~ ) ≤ ∥ Δ M ∥ 2 = o ( N ) \sigma_{r+1}(\widetilde{M}) \le\lVert\Delta_M\rVert_2 = o(N) σ r + 1 ( M ) ≤ ∥ Δ M ∥ 2 = o ( N ) .
Let U ^ ∈ R N × r \widehat{U} \in\mathbb{R}^{N \times r} U ∈ R N × r contain the eigenvectors corresponding to the r r r eigenvalues of M ~ \widetilde{M} M with largest absolute values, and let Λ ^ \widehat{\Lambda} Λ be the corresponding signed eigenvalue matrix. Define
Π : = U U T , Π ^ : = U ^ U ^ T . \Pi:= UU^{\mathsf{T}}, \qquad\widehat{\Pi} := \widehat{U}\widehat{U}^{\mathsf{T}}. Π := U U T , Π := U U T .
Then
SVD r ( M ~ ) = Π ^ M ~ Π ^ , ∥ Λ ^ − 1 ∥ 2 ≲ N − 1 . \operatorname{SVD}_r(\widetilde{M}) = \widehat{\Pi}\widetilde{M}\widehat{\Pi}, \qquad\lVert\widehat{\Lambda}^{-1}\rVert_2 \lesssim N^{-1}. SVD r ( M ) = Π M Π , ∥ Λ − 1 ∥ 2 ≲ N − 1 .
We first control U ^ \widehat{U} U row-wisely. Since M ~ U ^ = U ^ Λ ^ \widetilde{M}\widehat{U} = \widehat{U}\widehat{\Lambda} M U = U Λ , for every i ∈ [ N ] i \in[N] i ∈ [ N ] ,
∥ e i T U ^ ∥ ≤ ( ∥ e i T M U ^ ∥ + ∥ e i T Δ M U ^ ∥ ) ∥ Λ ^ − 1 ∥ 2 ≤ ( ∥ e i T U ∥ ∥ M ∥ 2 + ∥ e i T Δ M ∥ ) ∥ Λ ^ − 1 ∥ 2 \begin{aligned}
\lVert e_i^{\mathsf{T}}\widehat{U}\rVert\le\left(\lVert e_i^{\mathsf{T}}M\widehat{U}\rVert+ \lVert e_i^{\mathsf{T}}\Delta_M\widehat{U}\rVert\right)\lVert\widehat{\Lambda}^{-1}\rVert_2 \\
&\le\left(\lVert e_i^{\mathsf{T}}U\rVert\lVert M\rVert_2 + \lVert e_i^{\mathsf{T}}\Delta_M\rVert\right)\lVert\widehat{\Lambda}^{-1}\rVert_2
\end{aligned} ∥ e i T U ∥ ≤ ( ∥ e i T M U ∥ + ∥ e i T Δ M U ∥ ) ∥ Λ − 1 ∥ 2 ≤ ( ∥ e i T U ∥ ∥ M ∥ 2 + ∥ e i T Δ M ∥ ) ∥ Λ − 1 ∥ 2
≲ ( N + N ε N ) N − 1 ≲ N − 1 / 2 . \lesssim\left(\sqrt{N}+\sqrt{N}\,\varepsilon_N\right)N^{-1}\lesssim N^{-1/2}. ≲ ( N + N ε N ) N − 1 ≲ N − 1/2 .
Hence
∥ U ^ ∥ 2 → ∞ ≲ N − 1 / 2 . (107) \lVert\widehat{U}\rVert_{2\to\infty} \lesssim N^{-1/2}. \tag*{(107)} ∥ U ∥ 2 → ∞ ≲ N − 1/2 . (107)
Next, because ( I N − Π ) M = 0 (I_N-\Pi)M=0 ( I N − Π ) M = 0 ,
( I N − Π ) U ^ = ( I N − Π ) Δ M U ^ Λ ^ − 1 . (I_N-\Pi)\widehat{U}=(I_N-\Pi)\Delta_M\widehat{U}\widehat{\Lambda}^{-1}. ( I N − Π ) U = ( I N − Π ) Δ M U Λ − 1 .
Since
∥ e i T Δ M ∥ ≤ N ∥ Δ M ∥ max ≲ N ε N \lVert e_i^{\mathsf{T}}\Delta_M\rVert\le\sqrt{N}\lVert\Delta_M\rVert_{\max}\lesssim\sqrt{N}\,\varepsilon_N ∥ e i T Δ M ∥ ≤ N ∥ Δ M ∥ m a x ≲ N ε N
and ∥ e i T Π ∥ = ∥ e i T U ∥ ≲ N − 1 / 2 \lVert e_i^{\mathsf{T}}\Pi\rVert=\lVert e_i^{\mathsf{T}}\mathcal{U}\rVert\lesssim N^{-1/2} ∥ e i T Π ∥ = ∥ e i T U ∥ ≲ N − 1/2 , we obtain
∥ e i T ( I N − Π ) U ^ ∥ ≤ { ∥ e i T Δ M ∥ + ∥ e i T Π ∥ ∥ Δ M ∥ 2 } ∥ Λ ^ − 1 ∥ 2 ≲ N − 1 / 2 ε N . \lVert e_i^{\mathsf{T}}(I_N-\Pi)\widehat{\mathcal{U}}\rVert\le\left\{\lVert e_i^{\mathsf{T}}\Delta_M\rVert+\lVert e_i^{\mathsf{T}}\Pi\rVert\lVert\Delta_M\rVert_2\right\}\lVert\widehat{\Lambda}^{-1}\rVert_2
\lesssim N^{-1/2}\varepsilon_N. ∥ e i T ( I N − Π ) U ∥ ≤ { ∥ e i T Δ M ∥ + ∥ e i T Π ∥ ∥ Δ M ∥ 2 } ∥ Λ − 1 ∥ 2 ≲ N − 1/2 ε N .
Therefore,
∥ ( I N − Π ) Π ^ ∥ 2 → ∞ ≲ N − 1 / 2 ε N . (108) \lVert(I_N-\Pi)\widehat{\Pi}\rVert_{2\to\infty}\lesssim N^{-1/2}\varepsilon_N. \tag*{(108)} ∥( I N − Π ) Π ∥ 2 → ∞ ≲ N − 1/2 ε N . (108)
On the other hand, Wedin’s perturbation bound gives
∥ ( I N − Π ^ ) Π ∥ 2 ≲ ∥ Δ M ∥ 2 σ r ( M ) ≲ ε N . \lVert(I_N-\widehat{\Pi})\Pi\rVert_2\lesssim\frac{\lVert\Delta_M\rVert_2}{\sigma_r(M)}\lesssim\varepsilon_N. ∥( I N − Π ) Π ∥ 2 ≲ σ r ( M ) ∥ Δ M ∥ 2 ≲ ε N .
Consequently,
∥ Π ( I N − Π ^ ) ∥ 2 → ∞ = max i ∈ [ N ] ∥ e i T U U T ( I N − Π ^ ) ∥ ≤ ∥ U ∥ 2 → ∞ ∥ U T ( I N − Π ^ ) ∥ 2 ≲ N − 1 / 2 ε N . \lVert\Pi(I_N-\widehat{\Pi})\rVert_{2\to\infty}=\max_{i\in[N]}\lVert e_i^{\mathsf{T}}\mathcal{U}\mathcal{U}^{\mathsf{T}}(I_N-\widehat{\Pi})\rVert
\le\lVert\mathcal{U}\rVert_{2\to\infty}\lVert\mathcal{U}^{\mathsf{T}}(I_N-\widehat{\Pi})\rVert_2
\lesssim N^{-1/2}\varepsilon_N. ∥ Π ( I N − Π ) ∥ 2 → ∞ = i ∈ [ N ] max ∥ e i T U U T ( I N − Π )∥ ≤ ∥ U ∥ 2 → ∞ ∥ U T ( I N − Π ) ∥ 2 ≲ N − 1/2 ε N .
Since
Π ^ − Π = ( I N − Π ) Π ^ − Π ( I N − Π ^ ) , \widehat{\Pi}-\Pi=(I_N-\Pi)\widehat{\Pi}-\Pi(I_N-\widehat{\Pi}), Π − Π = ( I N − Π ) Π − Π ( I N − Π ) ,
we conclude that
∥ Π ^ − Π ∥ 2 → ∞ ≲ N − 1 / 2 ε N . (109) \lVert\widehat{\Pi}-\Pi\rVert_{2\to\infty}\lesssim N^{-1/2}\varepsilon_N. \tag*{(109)} ∥ Π − Π ∥ 2 → ∞ ≲ N − 1/2 ε N . (109)
Together with (106), this also gives ∥ Π ^ ∥ 2 → ∞ ≲ N − 1 / 2 \lVert\widehat{\Pi}\rVert_{2\to\infty}\lesssim N^{-1/2} ∥ Π ∥ 2 → ∞ ≲ N − 1/2 .
Let Δ Π : = Π ^ − Π \Delta_\Pi:=\widehat{\Pi}-\Pi Δ Π := Π − Π . Since Π M = M Π = M \Pi M=M\Pi=M Π M = M Π = M ,
SVD r ( M ~ ) − M = Π ^ Δ M Π ^ + Δ Π M + M Δ Π + Δ Π M Δ Π . \operatorname{SVD}_r(\widetilde{M})-M=\widehat{\Pi}\Delta_M\widehat{\Pi}+\Delta_\Pi M+M\Delta_\Pi+\Delta_\Pi M\Delta_\Pi. SVD r ( M ) − M = Π Δ M Π + Δ Π M + M Δ Π + Δ Π M Δ Π .
For the first term,
∥ Π ^ Δ M Π ^ ∥ max ≤ ∥ Π ^ ∥ 2 → ∞ 2 ∥ Δ M ∥ 2 ≲ ε N . \lVert\widehat{\Pi}\Delta_M\widehat{\Pi}\rVert_{\max} \le\lVert\widehat{\Pi}\rVert_{2\to\infty}^{2}\lVert\Delta_M\rVert_2 \lesssim\varepsilon_N. ∥ Π Δ M Π ∥ m a x ≤ ∥ Π ∥ 2 → ∞ 2 ∥ Δ M ∥ 2 ≲ ε N .
Moreover,
max j ∈ [ N ] ∥ M e j ∥ ≤ ∥ M ∥ 2 ∥ U ∥ 2 → ∞ ≲ N . \max_{j\in[N]} \lVert M e_j\rVert\le\lVert M\rVert_2\lVert U\rVert_{2\to\infty} \lesssim\sqrt{N}. j ∈ [ N ] max ∥ M e j ∥ ≤ ∥ M ∥ 2 ∥ U ∥ 2 → ∞ ≲ N .
Thus, by (109) ,
∥ Δ Π M ∥ max + ∥ M Δ Π ∥ max ≲ ε N . \lVert\Delta_{\Pi}M\rVert_{\max}+\lVert M\Delta_{\Pi}\rVert_{\max}\lesssim\varepsilon_N. ∥ Δ Π M ∥ m a x + ∥ M Δ Π ∥ m a x ≲ ε N .
Finally,
∥ Δ Π M Δ Π ∥ max ≤ ∥ Δ Π ∥ 2 → ∞ 2 ∥ M ∥ 2 ≲ ε N 2 ≲ ε N . \lVert\Delta_{\Pi}M\Delta_{\Pi}\rVert_{\max}\le\lVert\Delta_{\Pi}\rVert_{2\to\infty}^{2}\lVert M\rVert_2\lesssim\varepsilon_N^2\lesssim\varepsilon_N. ∥ Δ Π M Δ Π ∥ m a x ≤ ∥ Δ Π ∥ 2 → ∞ 2 ∥ M ∥ 2 ≲ ε N 2 ≲ ε N .
Combining the preceding bounds gives ∥ SVD r ( M ~ ) − M ∥ max ≲ ε N \lVert\operatorname{SVD}_r(\widetilde{M})-M\rVert_{\max}\lesssim\varepsilon_N ∥ SVD r ( M ) − M ∥ m a x ≲ ε N , which completes the proof. □
We apply Lemma 11 with M ~ = M ‾ \widetilde{M}=\overline{M} M = M , M = M ∗ M=M^* M = M ∗ . Assumption 1.4 gives ∥ U ∗ ∥ 2 → ∞ ≲ N − 1 / 2 \lVert U^*\rVert_{2\to\infty}\lesssim N^{-1/2} ∥ U ∗ ∥ 2 → ∞ ≲ N − 1/2 , whereas Assumptions 1.1 and 1.5 imply σ r ( M ∗ ) ≳ N \sigma_r(M^*)\gtrsim N σ r ( M ∗ ) ≳ N , ∥ M ∗ ∥ 2 ≲ N \lVert M^*\rVert_2\lesssim N ∥ M ∗ ∥ 2 ≲ N . Together with (106) , Lemma 11 yields
∥ SVD r ( M ‾ ) − M ∗ ∥ max = O p ( ε N ) . \lVert\operatorname{SVD}_r(\overline{M})-M^*\rVert_{\max}=O_p(\varepsilon_N). ∥ SVD r ( M ) − M ∗ ∥ m a x = O p ( ε N ) .
By Algorithm 1, M ^ r e f = SVD r ( M ‾ ) \widehat{M}^{\mathrm{ref}}=\operatorname{SVD}_r(\overline{M}) M ref = SVD r ( M ) , and hence
∥ M ^ r e f − M ∗ ∥ max = O p ( N − 1 ( log ( N 2 ) ) 2 e M , F ) . \lVert\widehat{M}^{\mathrm{ref}}-M^*\rVert_{\max}=O_p\left(N^{-1}(\log(N^2))^2e_{M,F}\right). ∥ M ref − M ∗ ∥ m a x = O p ( N − 1 ( log ( N 2 ) ) 2 e M , F ) .
Since J = N J=N J = N , N − 1 = ( N J ) − 1 / 2 N^{-1}=(NJ)^{-1/2} N − 1 = ( N J ) − 1/2 , log ( N 2 ) = log ( N J ) \log(N^2)=\log(NJ) log ( N 2 ) = log ( N J ) , and therefore
∥ M ^ r e f − M ∗ ∥ max = O p ( ( N J ) − 1 / 2 ( log ( N J ) ) 2 e M , F ) . \lVert\widehat{M}^{\mathrm{ref}}-M^*\rVert_{\max}=O_p\left((NJ)^{-1/2}(\log(NJ))^2e_{M,F}\right). ∥ M ref − M ∗ ∥ m a x = O p ( ( N J ) − 1/2 ( log ( N J ) ) 2 e M , F ) .
This proves Theorem 2 for the unipartite case.
Proof of Corollary 2
Proof. We apply Theorem 2 with M i n i t = M ^ S V D \mathbf{M}^{\mathrm{init}}=\widehat{\mathbf{M}}^{\mathrm{SVD}} M init = M SVD .
By Corollary 1,
∥ M ^ S V D − M ∗ ∥ F = O p ( e M , F ) , \left\|\widehat{\mathbf{M}}^{\mathrm{SVD}}-\mathbf{M}^{*}\right\|_{F}=O_{p}(e_{M,F}), M SVD − M ∗ F = O p ( e M , F ) ,
where
e M , F = N J J − 1 2 ( K + 2 ) ( log J ) K + 4 2 ( K + 2 ) . e_{M,F}=\sqrt{NJ}J^{-\frac{1}{2(K+2)}}(\log J)^{\frac{K+4}{2(K+2)}}. e M , F = N J J − 2 ( K + 2 ) 1 ( log J ) 2 ( K + 2 ) K + 4 .
Since N = J O ( 1 ) N=J^{O(1)} N = J O ( 1 ) , we have log ( N J ) = O ( log J ) \log(NJ)=O(\log J) log ( N J ) = O ( log J ) , and therefore
N ≲ e M , F ≪ N J { log ( N J ) } − 2 . \sqrt{N}\lesssim e_{M,F}\ll\sqrt{NJ}\{\log(NJ)\}^{-2}. N ≲ e M , F ≪ N J { log ( N J ) } − 2 .
Hence, Theorem 2 gives
∥ M ^ R S D − M ∗ ∥ max = O p ( ( log ( N J ) ) 2 J − 1 2 ( K + 2 ) ( log J ) K + 4 2 ( K + 2 ) ) . \left\|\widehat{\mathbf{M}}^{\mathrm{RSD}}-\mathbf{M}^{*}\right\|_{\max}
=O_{p}\left((\log(NJ))^{2}J^{-\frac{1}{2(K+2)}}(\log J)^{\frac{K+4}{2(K+2)}}\right). M RSD − M ∗ m a x = O p ( ( log ( N J ) ) 2 J − 2 ( K + 2 ) 1 ( log J ) 2 ( K + 2 ) K + 4 ) .
If the response supports are uniformly bounded, Corollary 1 gives
∥ M ^ S V D − M ∗ ∥ F = O p ( N J J − 1 2 ( K + 2 ) ) . \left\|\widehat{\mathbf{M}}^{\mathrm{SVD}}-\mathbf{M}^{*}\right\|_{F}
=O_{p}\left(\sqrt{NJ}J^{-\frac{1}{2(K+2)}}\right). M SVD − M ∗ F = O p ( N J J − 2 ( K + 2 ) 1 ) .
Thus, we may take
e M , F = N J J − 1 2 ( K + 2 ) . e_{M,F}=\sqrt{NJ}J^{-\frac{1}{2(K+2)}}. e M , F = N J J − 2 ( K + 2 ) 1 .
The conditions of Theorem 2 remain satisfied, and therefore
∥ M ^ R S D − M ∗ ∥ max = O p ( ( log ( N J ) ) 2 J − 1 2 ( K + 2 ) ) . \left\|\widehat{\mathbf{M}}^{\mathrm{RSD}}-\mathbf{M}^{*}\right\|_{\max}
=O_{p}\left((\log(NJ))^{2}J^{-\frac{1}{2(K+2)}}\right). M RSD − M ∗ m a x = O p ( ( log ( N J ) ) 2 J − 2 ( K + 2 ) 1 ) .
For unipartite data, the same arguments apply with J = N J=N J = N , under the additional condition f ( τ 0 ) ∈ [ a N N , b N N ] f(\tau_0) \in[a_{NN}, b_{NN}] f ( τ 0 ) ∈ [ a N N , b N N ] .
Proofs for Configuration Recovery
The configuration recovery arguments differ substantially between the two observation structures. We therefore present the unipartite proof first, followed by the auxiliary results and perturbation analysis required for the bipartite case.
Throughout Appendix D, the configuration recovery results are treated as deterministic perturbation statements. Let ξ = ξ N , J > 0 \xi= \xi_{N,J} > 0 ξ = ξ N , J > 0 denote a deterministic sequence such that ∥ M ^ − M ∗ ∥ max ≤ ξ \lVert\widehat{\mathbf{M}} - \mathbf{M}^* \rVert_{\max} \le\xi ∥ M − M ∗ ∥ m a x ≤ ξ , ξ = o ( 1 ) \xi= o(1) ξ = o ( 1 ) . For the bipartite parameter space Γ \Gamma Γ , we write C Γ C_{\Gamma} C Γ for the fixed radius appearing in its definition, so that ∥ γ ∥ ≤ C Γ \lVert\gamma\rVert\le C_{\Gamma} ∥ γ ∥ ≤ C Γ for every γ ∈ Γ \gamma\in\Gamma γ ∈ Γ .
Unipartite Configuration Recovery
Proof. Let
B ∗ = 1 2 J N M ∗ J N = Θ ∗ Θ ∗ T , B ^ = 1 2 J N M ^ J N , \mathbf{B}^* = \frac{1}{2}\mathbf{J}_N\mathbf{M}^*\mathbf{J}_N = \mathbf{\Theta}^*\mathbf{\Theta}^{*\mathsf{T}}, \qquad\widehat{\mathbf{B}} = \frac{1}{2}\mathbf{J}_N\widehat{\mathbf{M}}\mathbf{J}_N, B ∗ = 2 1 J N M ∗ J N = Θ ∗ Θ ∗ T , B = 2 1 J N M J N ,
where the first equality follows from 1 N T Θ ∗ = 0 \mathbf{1}_N^{\mathsf{T}}\mathbf{\Theta}^* = 0 1 N T Θ ∗ = 0 . Set Δ B = B ^ − B ∗ \Delta_{\mathbf{B}} = \widehat{\mathbf{B}} - \mathbf{B}^* Δ B = B − B ∗ . Since ∥ M ^ − M ∗ ∥ max ≤ ξ \lVert\widehat{\mathbf{M}} - \mathbf{M}^* \rVert_{\max} \le\xi ∥ M − M ∗ ∥ m a x ≤ ξ , double centering gives
∥ Δ B ∥ max ≲ ξ , ∥ Δ B ∥ ≲ N ξ , ∥ Δ B ∥ 2 → ∞ ≲ N ξ . (110) \lVert\Delta_{\mathbf{B}} \rVert_{\max} \lesssim\xi, \qquad\lVert\Delta_{\mathbf{B}} \rVert\lesssim N\xi, \qquad\lVert\Delta_{\mathbf{B}} \rVert_{2\to\infty} \lesssim\sqrt{N}\,\xi. \tag*{(110)} ∥ Δ B ∥ m a x ≲ ξ , ∥ Δ B ∥ ≲ N ξ , ∥ Δ B ∥ 2 → ∞ ≲ N ξ . (110)
Let G = Θ ∗ T Θ ∗ \mathbf{G} = \mathbf{\Theta}^{*\mathsf{T}}\mathbf{\Theta}^* G = Θ ∗ T Θ ∗ . By assumption, λ min ( G ) ≳ N \lambda_{\min}(\mathbf{G}) \gtrsim N λ m i n ( G ) ≳ N , while Assumption 1.1 gives λ max ( G ) ≲ N \lambda_{\max}(\mathbf{G}) \lesssim N λ m a x ( G ) ≲ N . Thus the K K K nonzero eigenvalues of B ∗ \mathbf{B}^* B ∗ are of order N N N .
Write
Eigen K ( B ^ ) = U ^ Λ ^ U ^ T , Λ ^ = diag ( λ ^ 1 , … , λ ^ K ) . \operatorname{Eigen}_K(\widehat{\mathbf{B}}) = \widehat{\mathbf{U}}\widehat{\mathbf{\Lambda}}\widehat{\mathbf{U}}^{\mathsf{T}}, \qquad\widehat{\mathbf{\Lambda}} = \operatorname{diag}(\widehat{\lambda}_1,\ldots,\widehat{\lambda}_K). Eigen K ( B ) = U Λ U T , Λ = diag ( λ 1 , … , λ K ) .
By Weyl’s inequality and (110) , since ξ = o ( 1 ) \xi=o(1) ξ = o ( 1 ) , λ ^ K ≳ N \widehat{\lambda}_K\gtrsim N λ K ≳ N for all sufficiently large N N N . Hence ( λ ^ k ) + = λ ^ k (\widehat{\lambda}_k)_+=\widehat{\lambda}_k ( λ k ) + = λ k for k ∈ [ K ] k\in[K] k ∈ [ K ] , and the configuration estimator in § 3.2 can be written as
Θ ^ = U ^ Λ ^ 1 / 2 . \widehat{\boldsymbol{\Theta}}=\widehat{\mathbf{U}}\widehat{\mathbf{\Lambda}}^{1/2}. Θ = U Λ 1/2 .
For k > K k>K k > K , the corresponding population eigenvalues of B ∗ \mathbf{B}^* B ∗ are zero. Hence Weyl’s inequality gives ∣ λ ^ k ∣ ≤ ∥ Δ B ∥ |\widehat{\lambda}_k|\leq\|\Delta_B\| ∣ λ k ∣ ≤ ∥ Δ B ∥ , k > K k>K k > K . Therefore,
∥ Θ ^ Θ ^ ⊤ − B ∗ ∥ = ∥ Eigen K ( B ^ ) − B ∗ ∥ ≤ ∥ Eigen K ( B ^ ) − B ^ ∥ + ∥ B ^ − B ∗ ∥ ≲ ∥ Δ B ∥ . \begin{aligned}
\left\|\widehat{\boldsymbol{\Theta}}\widehat{\boldsymbol{\Theta}}^{\top}-\mathbf{B}^*\right\|
&=\left\|\operatorname{Eigen}_K(\widehat{\mathbf{B}})-\mathbf{B}^*\right\|\\
&\leq\left\|\operatorname{Eigen}_K(\widehat{\mathbf{B}})-\widehat{\mathbf{B}}\right\|+\left\|\widehat{\mathbf{B}}-\mathbf{B}^*\right\|\\
&\lesssim\|\Delta_B\|.
\end{aligned} Θ Θ ⊤ − B ∗ = Eigen K ( B ) − B ∗ ≤ Eigen K ( B ) − B + B − B ∗ ≲ ∥ Δ B ∥.
Together with (110) , this yields
∥ Θ ^ Θ ^ ⊤ − B ∗ ∥ ≲ N ξ . (111) \left\|\widehat{\boldsymbol{\Theta}}\widehat{\boldsymbol{\Theta}}^{\top}-\mathbf{B}^*\right\|\lesssim N\xi. \tag*{(111)} Θ Θ ⊤ − B ∗ ≲ N ξ . (111)
Let P ∗ = Θ ∗ G − 1 Θ ∗ ⊤ \mathbf{P}^*=\boldsymbol{\Theta}^*\mathbf{G}^{-1}\boldsymbol{\Theta}^{*\top} P ∗ = Θ ∗ G − 1 Θ ∗ ⊤ be the orthogonal projector onto the column space of Θ ∗ \boldsymbol{\Theta}^* Θ ∗ . We first control the component of Θ ^ \widehat{\boldsymbol{\Theta}} Θ orthogonal to this space. Since ( I N − P ∗ ) B ∗ = 0 (\mathbf{I}_N-\mathbf{P}^*)\mathbf{B}^*=0 ( I N − P ∗ ) B ∗ = 0 and B ^ Θ ^ = Θ ^ Λ ^ \widehat{\mathbf{B}}\widehat{\boldsymbol{\Theta}}=\widehat{\boldsymbol{\Theta}}\widehat{\mathbf{\Lambda}} B Θ = Θ Λ , we have
( I N − P ∗ ) Θ ^ = ( I N − P ∗ ) Δ B Θ ^ Λ ^ − 1 . (\mathbf{I}_N-\mathbf{P}^*)\widehat{\boldsymbol{\Theta}}=(\mathbf{I}_N-\mathbf{P}^*)\Delta_B\widehat{\boldsymbol{\Theta}}\widehat{\mathbf{\Lambda}}^{-1}. ( I N − P ∗ ) Θ = ( I N − P ∗ ) Δ B Θ Λ − 1 .
Furthermore,
∥ Θ ^ Λ ^ − 1 ∥ = ∥ U ^ Λ ^ − 1 / 2 ∥ ≲ N − 1 / 2 . \left\|\widehat{\boldsymbol{\Theta}}\widehat{\mathbf{\Lambda}}^{-1}\right\|=\left\|\widehat{\mathbf{U}}\widehat{\mathbf{\Lambda}}^{-1/2}\right\|\lesssim N^{-1/2}. Θ Λ − 1 = U Λ − 1/2 ≲ N − 1/2 .
Since
∥ e i ⊤ P ∗ ∥ ≤ ∥ θ i ∗ ∥ ∥ G − 1 Θ ∗ ⊤ ∥ ≲ N − 1 / 2 , \left\|\mathbf{e}_i^{\top}\mathbf{P}^*\right\|\leq\left\|\boldsymbol{\theta}_i^*\right\|\left\|\mathbf{G}^{-1}\boldsymbol{\Theta}^{*\top}\right\|\lesssim N^{-1/2}, e i ⊤ P ∗ ≤ ∥ θ i ∗ ∥ G − 1 Θ ∗ ⊤ ≲ N − 1/2 ,
we have
∥ e i ⊤ ( I N − P ∗ ) Δ B ∥ ≤ ∥ e i ⊤ Δ B ∥ + ∥ e i ⊤ P ∗ ∥ ∥ Δ B ∥ ≲ N ξ . \begin{aligned}
\left\|\mathbf{e}_i^{\top}(\mathbf{I}_N-\mathbf{P}^*)\Delta_B\right\|
&\leq\left\|\mathbf{e}_i^{\top}\Delta_B\right\|+\left\|\mathbf{e}_i^{\top}\mathbf{P}^*\right\|\|\Delta_B\|\\
&\lesssim\sqrt{N}\xi.
\end{aligned} e i ⊤ ( I N − P ∗ ) Δ B ≤ e i ⊤ Δ B + e i ⊤ P ∗ ∥ Δ B ∥ ≲ N ξ .
Therefore, ∥ ( I N − P ∗ ) Δ B ∥ 2 → ∞ ≲ N ξ \lVert(I_N-P^*)\Delta B\rVert_{2\to\infty}\lesssim\sqrt{N}\,\xi ∥( I N − P ∗ ) Δ B ∥ 2 → ∞ ≲ N ξ , and consequently
∥ ( I N − P ∗ ) Θ ^ ∥ 2 → ∞ ≲ ξ . (112) \lVert(I_N-P^*)\widehat{\Theta}\rVert_{2\to\infty}\lesssim\xi. \tag*{(112)} ∥( I N − P ∗ ) Θ ∥ 2 → ∞ ≲ ξ . (112)
It remains to control the component within the column space of Θ ∗ \Theta^* Θ ∗ . Define H = G − 1 Θ ∗ ⊤ Θ ^ \mathcal{H}=\mathbf{G}^{-1}\Theta^{*\top}\widehat{\Theta} H = G − 1 Θ ∗ ⊤ Θ . Then P ∗ Θ ^ = Θ ∗ H P^*\widehat{\Theta}=\Theta^*\mathcal{H} P ∗ Θ = Θ ∗ H . Moreover,
H H ⊤ − I K = G − 1 Θ ∗ ⊤ ( Θ ^ Θ ^ ⊤ − Θ ∗ Θ ∗ ⊤ ) Θ ∗ G − 1 . \mathcal{H}\mathcal{H}^{\top}-I_K=\mathbf{G}^{-1}\Theta^{*\top}\left(\widehat{\Theta}\widehat{\Theta}^{\top}-\Theta^*\Theta^{*\top}\right)\Theta^*\mathbf{G}^{-1}. H H ⊤ − I K = G − 1 Θ ∗ ⊤ ( Θ Θ ⊤ − Θ ∗ Θ ∗ ⊤ ) Θ ∗ G − 1 .
Since
∥ G − 1 Θ ∗ ⊤ ∥ = λ min ( G ) − 1 / 2 ≲ N − 1 / 2 , \lVert\mathbf{G}^{-1}\Theta^{*\top}\rVert=\lambda_{\min}(\mathbf{G})^{-1/2}\lesssim N^{-1/2}, ∥ G − 1 Θ ∗ ⊤ ∥ = λ m i n ( G ) − 1/2 ≲ N − 1/2 ,
(110) yields
∥ H H ⊤ − I K ∥ ≲ ξ . (113) \lVert\mathcal{H}\mathcal{H}^{\top}-I_K\rVert\lesssim\xi. \tag*{(113)} ∥ H H ⊤ − I K ∥ ≲ ξ . (113)
Thus, for all sufficiently large N N N , H \mathcal{H} H is nonsingular. Let Q = ( H H ⊤ ) − 1 / 2 H ∈ O K \mathbf{Q}=(\mathcal{H}\mathcal{H}^{\top})^{-1/2}\mathcal{H}\in\mathcal{O}_K Q = ( H H ⊤ ) − 1/2 H ∈ O K be its polar factor. By (112),
∥ H − Q ∥ ≲ ξ . (114) \lVert\mathcal{H}-\mathbf{Q}\rVert\lesssim\xi. \tag*{(114)} ∥ H − Q ∥ ≲ ξ . (114)
Finally,
Θ ^ − Θ ∗ Q = ( I N − P ∗ ) Θ ^ + Θ ∗ ( H − Q ) . \widehat{\Theta}-\Theta^*\mathbf{Q}=(I_N-P^*)\widehat{\Theta}+\Theta^*(\mathcal{H}-\mathbf{Q}). Θ − Θ ∗ Q = ( I N − P ∗ ) Θ + Θ ∗ ( H − Q ) .
Therefore, by (111), (113), and Assumption 1.1,
∥ Θ ^ − Θ ∗ Q ∥ 2 → ∞ ≲ ξ . \lVert\widehat{\Theta}-\Theta^*\mathbf{Q}\rVert_{2\to\infty}\lesssim\xi. ∥ Θ − Θ ∗ Q ∥ 2 → ∞ ≲ ξ .
Since Q ∈ O K \mathbf{Q} \in\mathcal{O}_{K} Q ∈ O K , taking this Q \mathbf{Q} Q and zero translation in the definition of d c d_c d c gives
d c ( Θ ^ , Θ ∗ ) ≲ ξ . d_c(\widehat{\boldsymbol{\Theta}},\boldsymbol{\Theta}^{*}) \lesssim\xi. d c ( Θ , Θ ∗ ) ≲ ξ .
This completes the proof.
Bipartite Configuration Recovery
We first collect the auxiliary results needed for the perturbation analysis and then prove Theorem 4.
Auxiliary Lemmas and Proofs
Lemma 12. Suppose Assumption 1.1 and Assumption 2.1 hold. Then, for all sufficiently large N N N and J J J , there exists Q Z ∈ S K \mathbf{Q}_{Z} \in\mathcal{S}_{K} Q Z ∈ S K such that
∥ Z ^ Q Z − Z ∗ ∥ 2 → ∞ ≲ ξ . \left\|\widehat{\mathbf{Z}}\mathbf{Q}_{Z}-\mathbf{Z}^{*}\right\|_{2\to\infty} \lesssim\xi. Z Q Z − Z ∗ 2 → ∞ ≲ ξ . Moreover, the optimization problems based on Z ^ \widehat{\mathbf{Z}} Z and Z ^ Q Z \widehat{\mathbf{Z}}\mathbf{Q}_{Z} Z Q Z have the same minimum value, and their global minimizers are in one-to-one correspondence under
( τ , vec ( F ) T , b T ) T ↦ ( τ , vec ( Q Z F ) T , b T ) T . (\tau,\operatorname{vec}(\mathbf{F})^{\mathsf{T}},\mathbf{b}^{\mathsf{T}})^{\mathsf{T}} \mapsto(\tau,\operatorname{vec}(\mathbf{Q}_{Z}\mathbf{F})^{\mathsf{T}},\mathbf{b}^{\mathsf{T}})^{\mathsf{T}}. ( τ , vec ( F ) T , b T ) T ↦ ( τ , vec ( Q Z F ) T , b T ) T . In particular, if
γ ^ = ( τ ^ , vec ( F ^ ) T , b ^ T ) T \widehat{\gamma}=\left(\widehat{\tau},\operatorname{vec}(\widehat{\mathbf{F}})^{\mathsf{T}},\widehat{\mathbf{b}}^{\mathsf{T}}\right)^{\mathsf{T}} γ = ( τ , vec ( F ) T , b T ) T is a global minimizer of (10), then, for s = 1 , 2 s=1,2 s = 1 , 2 ,
Θ P s ( ( τ ^ , vec ( Q Z F ^ ) T , b ^ T ) T ; Z ^ Q Z ) = Θ P s ( γ ^ ; Z ^ ) . \Theta_{\mathcal{P}_{s}}\left(\left(\widehat{\tau},\operatorname{vec}(\mathbf{Q}_{Z}\widehat{\mathbf{F}})^{\mathsf{T}},\widehat{\mathbf{b}}^{\mathsf{T}}\right)^{\mathsf{T}};\widehat{\mathbf{Z}}\mathbf{Q}_{Z}\right)=\Theta_{\mathcal{P}_{s}}(\widehat{\gamma};\widehat{\mathbf{Z}}). Θ P s ( ( τ , vec ( Q Z F ) T , b T ) T ; Z Q Z ) = Θ P s ( γ ; Z ) .
Proof. Let M c ∗ = 1 2 J N M ∗ J J = U ∗ Σ ∗ V ∗ ⊤ M_c^* = \frac{1}{2}J_NM^*J_J = U^*\Sigma^*V^{*\top} M c ∗ = 2 1 J N M ∗ J J = U ∗ Σ ∗ V ∗ ⊤ and M ^ c = 1 2 J N M ^ J J \widehat{M}_c = \frac{1}{2}J_N\widehat{M}J_J M c = 2 1 J N M J J . Set Δ c = M ^ c − M c ∗ \Delta_c = \widehat{M}_c - M_c^* Δ c = M c − M c ∗ . Since
∥ M ^ − M ∗ ∥ max ≤ ξ , \lVert\widehat{M} - M^* \rVert_{\max} \le\xi, ∥ M − M ∗ ∥ m a x ≤ ξ ,
∥ Δ c ∥ max ≲ ξ , ∥ Δ c ∥ ≲ N J ξ . \lVert\Delta_c \rVert_{\max} \lesssim\xi, \qquad\lVert\Delta_c \rVert\lesssim\sqrt{NJ}\,\xi. ∥ Δ c ∥ m a x ≲ ξ , ∥ Δ c ∥ ≲ N J ξ .
By Assumption 2.1, the relevant population singular values are separated by gaps of order N J \sqrt{NJ} N J . Therefore, the left and right singular vectors can be matched individually. The only remaining ambiguity for each matched singular-vector pair is its sign. Hence, by Weyl’s inequality and Wedin’s theorem, for all sufficiently large N N N and J J J , there exists Q Z ∈ S K Q_Z \in\mathcal{S}_K Q Z ∈ S K such that
∥ U ^ Q Z − U ∗ ∥ + ∥ V ^ Q Z − V ∗ ∥ ≲ ξ , ∥ Σ ^ − Σ ∗ ∥ ≲ N J ξ . \lVert\widehat{U}Q_Z - U^* \rVert+ \lVert\widehat{V}Q_Z - V^* \rVert\lesssim\xi, \qquad\lVert\widehat{\Sigma} - \Sigma^* \rVert\lesssim\sqrt{NJ}\,\xi. ∥ U Q Z − U ∗ ∥ + ∥ V Q Z − V ∗ ∥ ≲ ξ , ∥ Σ − Σ ∗ ∥ ≲ N J ξ .
In particular, ∥ Σ ^ − 1 ∥ ≲ ( N J ) − 1 / 2 \lVert\widehat{\Sigma}^{-1} \rVert\lesssim(NJ)^{-1/2} ∥ Σ − 1 ∥ ≲ ( N J ) − 1/2 and ∥ Σ ^ − 1 − Σ ∗ − 1 ∥ ≲ ξ N J \lVert\widehat{\Sigma}^{-1} - \Sigma^{*-1} \rVert\lesssim\frac{\xi}{\sqrt{NJ}} ∥ Σ − 1 − Σ ∗− 1 ∥ ≲ N J ξ .
Since Q Z Q_Z Q Z commutes with the diagonal matrix Σ ^ \widehat{\Sigma} Σ ,
U ^ Q Z − U ∗ = Δ c V ^ Q Z Σ ^ − 1 + M c ∗ ( V ^ Q Z − V ∗ ) Σ ^ − 1 + U ∗ Σ ∗ ( Σ ^ − 1 − Σ ∗ − 1 ) . \widehat{U}Q_Z - U^* = \Delta_c\widehat{V}Q_Z\widehat{\Sigma}^{-1} + M_c^*(\widehat{V}Q_Z - V^*)\widehat{\Sigma}^{-1}
+ U^*\Sigma^*(\widehat{\Sigma}^{-1} - \Sigma^{*-1}). U Q Z − U ∗ = Δ c V Q Z Σ − 1 + M c ∗ ( V Q Z − V ∗ ) Σ − 1 + U ∗ Σ ∗ ( Σ − 1 − Σ ∗− 1 ) .
Under Assumption 1.1, ∥ M c ∗ ∥ 2 → ∞ ≲ J \lVert M_c^*\rVert_{2\to\infty} \lesssim\sqrt{J} ∥ M c ∗ ∥ 2 → ∞ ≲ J and ∥ U ∗ ∥ 2 → ∞ ≲ N − 1 / 2 \lVert U^*\rVert_{2\to\infty} \lesssim N^{-1/2} ∥ U ∗ ∥ 2 → ∞ ≲ N − 1/2 . Moreover, ∥ e i ⊤ Δ c ∥ ≲ J ξ \lVert e_i^\top\Delta_c\rVert\lesssim\sqrt{J}\,\xi ∥ e i ⊤ Δ c ∥ ≲ J ξ . Hence N ∥ U ^ Q Z − U ∗ ∥ 2 → ∞ ≲ ξ \sqrt{N}\lVert\widehat{U}Q_Z - U^*\rVert_{2\to\infty} \lesssim\xi N ∥ U Q Z − U ∗ ∥ 2 → ∞ ≲ ξ .
Applying the same argument to M ^ c ⊤ \widehat{M}_c^\top M c ⊤ gives J ∥ V ^ Q Z − V ∗ ∥ 2 → ∞ ≲ ξ \sqrt{J}\lVert\widehat{V}Q_Z - V^*\rVert_{2\to\infty} \lesssim\xi J ∥ V Q Z − V ∗ ∥ 2 → ∞ ≲ ξ . Since
W ^ = 1 N J V ^ Σ ^ , W ∗ = 1 N J V ∗ Σ ∗ , \widehat{W} = \frac{1}{\sqrt{NJ}}\widehat{V}\widehat{\Sigma}, \qquad W^* = \frac{1}{\sqrt{NJ}}V^*\Sigma^*, W = N J 1 V Σ , W ∗ = N J 1 V ∗ Σ ∗ ,
we further obtain J ∥ W ^ Q Z − W ∗ ∥ 2 → ∞ ≲ ξ \sqrt{J}\lVert\widehat{W}Q_Z - W^*\rVert_{2\to\infty} \lesssim\xi J ∥ W Q Z − W ∗ ∥ 2 → ∞ ≲ ξ . Therefore, ∥ Z ^ Q Z − Z ∗ ∥ 2 → ∞ ≲ ξ \lVert\widehat{Z}Q_Z - Z^*\rVert_{2\to\infty} \lesssim\xi ∥ Z Q Z − Z ∗ ∥ 2 → ∞ ≲ ξ .
Since Q Z ⊤ = Q Z − 1 = Q Z Q_Z^\top= Q_Z^{-1} = Q_Z Q Z ⊤ = Q Z − 1 = Q Z , direct substitution into (8) gives
N U ^ Q Z ( Q Z F ^ ) = N U ^ F ^ \sqrt{N}\widehat{U}Q_Z(Q_Z\widehat{F}) = \sqrt{N}\widehat{U}\widehat{F} N U Q Z ( Q Z F ) = N U F
and
J W ^ Q Z ( Q Z F ^ ) − ⊤ + 1 J b ^ ⊤ = J W ^ F ^ − ⊤ + 1 J b ^ ⊤ . \sqrt{J}\widehat{\mathbf{W}}\mathbf{Q}_{Z}(\mathbf{Q}_{Z}\widehat{\mathbf{F}})^{-\top}+\mathbf{1}_{J}\widehat{\mathbf{b}}^{\top}=\sqrt{J}\widehat{\mathbf{W}}\widehat{\mathbf{F}}^{-\top}+\mathbf{1}_{J}\widehat{\mathbf{b}}^{\top}. J W Q Z ( Q Z F ) − ⊤ + 1 J b ⊤ = J W F − ⊤ + 1 J b ⊤ .
Thus, for s = 1 , 2 s=1,2 s = 1 , 2 ,
Θ P s ( ( τ ^ , vec ( Q Z F ^ ) ⊤ , b ^ ⊤ ) ⊤ ; Z ^ Q Z ) = Θ P s ( γ ^ ; Z ^ ) , \Theta_{P_s}\left(\left(\widehat{\tau},\operatorname{vec}(\mathbf{Q}_{Z}\widehat{\mathbf{F}})^{\top},\widehat{\mathbf{b}}^{\top}\right)^{\top};\widehat{\mathbf{Z}}\mathbf{Q}_{Z}\right)=\Theta_{P_s}(\widehat{\gamma};\widehat{\mathbf{Z}}), Θ P s ( ( τ , vec ( Q Z F ) ⊤ , b ⊤ ) ⊤ ; Z Q Z ) = Θ P s ( γ ; Z ) ,
and consequently
M ( ( τ ^ , vec ( Q Z F ^ ) ⊤ , b ^ ⊤ ) ⊤ ; Z ^ Q Z ) = M ( γ ^ ; Z ^ ) . M\left(\left(\widehat{\tau},\operatorname{vec}(\mathbf{Q}_{Z}\widehat{\mathbf{F}})^{\top},\widehat{\mathbf{b}}^{\top}\right)^{\top};\widehat{\mathbf{Z}}\mathbf{Q}_{Z}\right)=M(\widehat{\gamma};\widehat{\mathbf{Z}}). M ( ( τ , vec ( Q Z F ) ⊤ , b ⊤ ) ⊤ ; Z Q Z ) = M ( γ ; Z ) .
Moreover, ( Q Z F ) ⊤ ( Q Z F ) = F ⊤ F (\mathbf{Q}_{Z}\mathbf{F})^{\top}(\mathbf{Q}_{Z}\mathbf{F})=\mathbf{F}^{\top}\mathbf{F} ( Q Z F ) ⊤ ( Q Z F ) = F ⊤ F , so the transformation preserves Γ \Gamma Γ . Since the transformation is its own inverse, it defines a bijection between the feasible sets of the two problems while preserving the objective value. The claimed one-to-one correspondence of global minimizers follows. □
Lemma 13. Suppose Assumption 1.1 and Assumption 2 hold, and suppose
ξ 2 = o ( c N J ) , ρ N J = o ( q N J N J ) . \xi^{2}=o(c_{NJ}),\qquad\rho_{NJ}=o\left(\frac{q_{NJ}}{\sqrt{NJ}}\right). ξ 2 = o ( c N J ) , ρ N J = o ( N J q N J ) . Then, for all sufficiently large N N N and J J J , the minimum in (10) is attained.
Moreover, for any global minimizer
γ ^ = ( τ ^ , vec ( F ^ ) ⊤ , b ^ ⊤ ) ⊤ \widehat{\gamma}=\left(\widehat{\tau},\operatorname{vec}(\widehat{\mathbf{F}})^{\top},\widehat{\mathbf{b}}^{\top}\right)^{\top} γ = ( τ , vec ( F ) ⊤ , b ⊤ ) ⊤ and the sign matrix Q Z \mathbf{Q}_{Z} Q Z in Lemma 12,
inf γ ~ ∈ [ γ 0 ] ∥ ( τ ^ , vec ( Q Z F ^ ) ⊤ , b ^ ⊤ ) ⊤ − γ ~ ∥ < ρ N J . (115) \inf_{\widetilde{\gamma}\in[\gamma_{0}]}\left\|\left(\widehat{\tau},\operatorname{vec}(\mathbf{Q}_{Z}\widehat{\mathbf{F}})^{\top},\widehat{\mathbf{b}}^{\top}\right)^{\top}-\widetilde{\gamma}\right\|<\rho_{NJ}. \tag*{(115)} γ ∈ [ γ 0 ] inf ( τ , vec ( Q Z F ) ⊤ , b ⊤ ) ⊤ − γ < ρ N J . (115)
Furthermore, there exists Q ∈ S K \mathbf{Q} \in\mathcal{S}_K Q ∈ S K attaining the preceding infimum such that
∣ τ ^ − τ 0 ∣ + ∥ Q Z F ^ − F 0 Q ∥ F + ∥ b ^ − Q T b 0 ∥ ≲ N J q N J ξ . |\widehat{\tau}-\tau_0|+\|\mathbf{Q}_Z\widehat{\mathbf{F}}-\mathbf{F}_0\mathbf{Q}\|_F+\|\widehat{\mathbf{b}}-\mathbf{Q}^{\mathsf{T}}\mathbf{b}_0\|\lesssim\frac{\sqrt{NJ}}{q_{NJ}}\xi. ∣ τ − τ 0 ∣ + ∥ Q Z F − F 0 Q ∥ F + ∥ b − Q T b 0 ∥ ≲ q N J N J ξ .
Proof. Let D 0 = ∂ M ( γ 0 ; Z ∗ ) \mathbf{D}_0=\partial\mathbf{M}(\gamma_0;\mathbf{Z}^*) D 0 = ∂ M ( γ 0 ; Z ∗ ) . Since the parameter dimension is fixed and the rows of D 0 \mathbf{D}_0 D 0 are uniformly bounded, ∥ D 0 ∥ ≲ N J \|\mathbf{D}_0\|\lesssim\sqrt{NJ} ∥ D 0 ∥ ≲ N J . Assumption 2.2 therefore implies q N J ≲ N J q_{NJ}\lesssim\sqrt{NJ} q N J ≲ N J . Hence
ρ N J = o ( q N J N J ) = o ( 1 ) . \rho_{NJ}=o\left(\frac{q_{NJ}}{\sqrt{NJ}}\right)=o(1). ρ N J = o ( N J q N J ) = o ( 1 ) .
We first show that minimizing sequences stay uniformly away from singular F \mathbf{F} F . By Lemma 12 and Assumption 2.1,
λ min { ( W ^ Q Z ) T ( W ^ Q Z ) } = σ ^ K 2 N J ≳ 1. \lambda_{\min}\left\{(\widehat{\mathbf{W}}\mathbf{Q}_Z)^{\mathsf{T}}(\widehat{\mathbf{W}}\mathbf{Q}_Z)\right\}=\frac{\widehat{\sigma}_K^2}{NJ}\gtrsim1. λ m i n { ( W Q Z ) T ( W Q Z ) } = N J σ K 2 ≳ 1.
Since the columns of U ^ Q Z \widehat{\mathbf{U}}\mathbf{Q}_Z U Q Z and W ^ Q Z \widehat{\mathbf{W}}\mathbf{Q}_Z W Q Z are centered, for any γ ∈ Γ \gamma\in\Gamma γ ∈ Γ ,
1 N J ∑ i , j ∥ θ i ( γ ; Z ^ Q Z ) − θ N + j ( γ ; Z ^ Q Z ) ∥ 2 ≥ tr { F − 1 ( W ^ Q Z ) T ( W ^ Q Z ) F − T } ≳ σ min − 2 ( F ) . \frac{1}{NJ}\sum_{i,j}\left\|\theta_i(\gamma;\widehat{\mathbf{Z}}\mathbf{Q}_Z)-\theta_{N+j}(\gamma;\widehat{\mathbf{Z}}\mathbf{Q}_Z)\right\|^2
\geq\operatorname{tr}\left\{\mathbf{F}^{-1}(\widehat{\mathbf{W}}\mathbf{Q}_Z)^{\mathsf{T}}(\widehat{\mathbf{W}}\mathbf{Q}_Z)\mathbf{F}^{-\mathsf{T}}\right\}
\gtrsim\sigma_{\min}^{-2}(\mathbf{F}). N J 1 i , j ∑ θ i ( γ ; Z Q Z ) − θ N + j ( γ ; Z Q Z ) 2 ≥ tr { F − 1 ( W Q Z ) T ( W Q Z ) F − T } ≳ σ m i n − 2 ( F ) .
Since ∣ τ ∣ ≤ ∥ γ ∥ ≤ C Γ |\tau|\leq\|\gamma\|\leq C_\Gamma ∣ τ ∣ ≤ ∥ γ ∥ ≤ C Γ , Jensen’s inequality gives
1 N J ∥ M ( γ ; Z ^ Q Z ) ∥ F 2 ⟶ ∞ \frac{1}{NJ}\|\mathbf{M}(\gamma;\widehat{\mathbf{Z}}\mathbf{Q}_Z)\|_F^2\longrightarrow\infty N J 1 ∥ M ( γ ; Z Q Z ) ∥ F 2 ⟶ ∞
uniformly as σ min ( F ) → 0 \sigma_{\min}(\mathbf{F})\to0 σ m i n ( F ) → 0 .
Since F 0 \mathbf{F}_0 F 0 and F 0 − 1 \mathbf{F}_0^{-1} F 0 − 1 are uniformly bounded, Lemma 12 and the smoothness of (8) –(9) give ∥ M ( γ 0 ; Z ^ Q Z ) − M ∗ ∥ F ≲ N J ξ \|\mathbf{M}(\gamma_0;\widehat{\mathbf{Z}}\mathbf{Q}_Z)-\mathbf{M}^*\|_F\lesssim\sqrt{NJ}\xi ∥ M ( γ 0 ; Z Q Z ) − M ∗ ∥ F ≲ N J ξ . Hence
1 N J l ( γ 0 ; Z ^ Q Z , M ^ ) ≲ ξ 2 , \frac{1}{NJ}l(\gamma_0;\widehat{\mathbf{Z}}\mathbf{Q}_Z,\widehat{\mathbf{M}})\lesssim\xi^2, N J 1 l ( γ 0 ; Z Q Z , M ) ≲ ξ 2 ,
e n d s m i d = 1 {ends_mid=1} e n d s m i d = 1
while Assumption 1.1 and ξ = o ( 1 ) \xi= o(1) ξ = o ( 1 ) imply 1 N J ∥ M ^ ∥ F 2 ≲ 1 \frac{1}{NJ}\|\widehat{\mathbf M}\|_{F}^{2} \lesssim1 N J 1 ∥ M ∥ F 2 ≲ 1 . Therefore, there exists a fixed c F > 0 c_F > 0 c F > 0 such that every minimizing sequence eventually satisfies σ min ( F ) ≥ c F \sigma_{\min}(\mathbf F) \ge c_F σ m i n ( F ) ≥ c F . The corresponding subset of Γ \Gamma Γ is compact. Hence the aligned problem attains its minimum. By the bijection in Lemma 12 , the original problem (10) also attains its minimum.
On the set σ min ( F ) ≥ c F \sigma_{\min}(\mathbf F) \ge c_F σ m i n ( F ) ≥ c F , ∥ F ∥ + ∥ F − 1 ∥ ≲ 1 \|\mathbf F\| + \|\mathbf F^{-1}\| \lesssim1 ∥ F ∥ + ∥ F − 1 ∥ ≲ 1 . Thus, Lemma 12 and smoothness imply
sup γ ∈ Γ : σ min ( F ) ≥ c F ∥ M ( γ ; Z Q Z ^ ) − M ( γ ; Z ∗ ) ∥ F ≲ N J ξ . (116) \sup_{\substack{\gamma\in\Gamma: \\ \sigma_{\min}(\mathbf F) \ge c_F}} \|\mathbf M(\gamma; \widehat{\mathbf Z Q_Z}) - \mathbf M(\gamma; \mathbf Z^*)\|_F \lesssim\sqrt{NJ}\,\xi. \tag*{(116)} γ ∈ Γ : σ m i n ( F ) ≥ c F sup ∥ M ( γ ; Z Q Z ) − M ( γ ; Z ∗ ) ∥ F ≲ N J ξ . (116)
By Lemma 12 ,
γ ‾ : = ( τ ^ , vec ( Q Z F ^ ) T , b ^ T ) T \overline{\gamma} := \left(\widehat{\tau}, \operatorname{vec}(\mathbf Q_Z\widehat{\mathbf F})^{\mathsf T}, \widehat{\mathbf b}^{\mathsf T}\right)^{\mathsf T} γ := ( τ , vec ( Q Z F ) T , b T ) T
is a global minimizer of the aligned problem. Hence, by optimality,
∥ M ( γ ‾ ; Z Q Z ^ ) − M ^ ∥ F ≲ N J ξ . (117) \|\mathbf M(\overline{\gamma};\widehat{\mathbf Z Q_Z})-\widehat{\mathbf M}\|_F \lesssim\sqrt{NJ}\,\xi. \tag*{(117)} ∥ M ( γ ; Z Q Z ) − M ∥ F ≲ N J ξ . (117)
We next localize this minimizer. Suppose, to the contrary, that
inf γ ~ ∈ [ γ 0 ] ∥ γ ‾ − γ ~ ∥ ≥ ρ N J . \inf_{\widetilde{\gamma}\in[\gamma_0]}\|\overline{\gamma}-\widetilde{\gamma}\| \ge\rho_{NJ}. γ ∈ [ γ 0 ] inf ∥ γ − γ ∥ ≥ ρ N J .
By Assumption 2.3, there exists a fixed constant c 1 > 0 c_1 > 0 c 1 > 0 such that, for all sufficiently large N N N and J J J , ∥ M ( γ ‾ ; Z ∗ ) − M ∗ ∥ F ≥ c 1 N J c N J \|\mathbf M(\overline{\gamma};\mathbf Z^*)-\mathbf M^*\|_F \ge c_1\sqrt{NJ}c_{NJ} ∥ M ( γ ; Z ∗ ) − M ∗ ∥ F ≥ c 1 N J c N J . By (116) and ∥ M ^ − M ∗ ∥ F ≤ N J ξ \|\widehat{\mathbf M}-\mathbf M^*\|_F \le\sqrt{NJ}\,\xi ∥ M − M ∗ ∥ F ≤ N J ξ , there exists a fixed constant c 2 > 0 c_2 > 0 c 2 > 0 such that
∥ M ( γ ˉ ; Z Q Z ^ ) − M ^ ∥ F ≥ ∥ M ( γ ˉ ; Z ∗ ) − M ∗ ∥ F − ∥ M ( γ ˉ ; Z Q Z ^ ) − M ( γ ˉ ; Z ∗ ) ∥ F − ∥ M ^ − M ∗ ∥ F ≥ c 1 N J c N J − c 2 N J ξ . \begin{aligned}
\left\|\mathbf{M}(\bar{\gamma};\widehat{\mathbf{ZQ_Z}})-\widehat{\mathbf{M}}\right\|_F \ge\left\|\mathbf{M}(\bar{\gamma};\mathbf{Z}^*)-\mathbf{M}^*\right\|_F - \left\|\mathbf{M}(\bar{\gamma};\widehat{\mathbf{ZQ_Z}})-\mathbf{M}(\bar{\gamma};\mathbf{Z}^*)\right\|_F - \left\|\widehat{\mathbf{M}}-\mathbf{M}^*\right\|_F \\
&\ge c_1\sqrt{N J c_{N J}}-c_2\sqrt{N J}\,\xi.
\end{aligned} M ( γ ˉ ; Z Q Z ) − M F ≥ ∥ M ( γ ˉ ; Z ∗ ) − M ∗ ∥ F − M ( γ ˉ ; Z Q Z ) − M ( γ ˉ ; Z ∗ ) F − M − M ∗ F ≥ c 1 N J c N J − c 2 N J ξ .
Since ξ 2 = o ( c N J ) \xi^2=o(c_{N J}) ξ 2 = o ( c N J ) , we have ξ = o ( c N J ) \xi=o(\sqrt{c_{N J}}) ξ = o ( c N J ) . Thus, the preceding lower bound is of order N J c N J \sqrt{N J c_{N J}} N J c N J , which is asymptotically larger than the upper bound O ( N J ξ ) O(\sqrt{N J}\xi) O ( N J ξ ) in (117) . This is a contradiction. Therefore,
inf γ ~ ∈ [ γ 0 ] ∥ γ ˉ − γ ~ ∥ < ρ N J . (118) \inf_{\widetilde{\gamma}\in[\gamma_0]}\|\bar{\gamma}-\widetilde{\gamma}\|<\rho_{N J}. \tag*{(118)} γ ∈ [ γ 0 ] inf ∥ γ ˉ − γ ∥ < ρ N J . (118)
It remains to establish the local growth bound. We first prove that, uniformly for γ ∈ Γ \gamma\in\Gamma γ ∈ Γ satisfying ∥ γ − γ 0 ∥ ≤ ρ N J \|\gamma-\gamma_0\|\le\rho_{N J} ∥ γ − γ 0 ∥ ≤ ρ N J ,
∥ M ( γ ; Z ∗ ) − M ∗ ∥ F ≳ q N J ∥ γ − γ 0 ∥ . (119) \|\mathbf{M}(\gamma;\mathbf{Z}^*)-\mathbf{M}^*\|_F\gtrsim q_{N J}\|\gamma-\gamma_0\|. \tag*{(119)} ∥ M ( γ ; Z ∗ ) − M ∗ ∥ F ≳ q N J ∥ γ − γ 0 ∥. (119)
Let h = γ − γ 0 \mathbf{h}=\gamma-\gamma_0 h = γ − γ 0 and write its F \mathbf{F} F -component as H \mathcal{H} H . Rotational invariance gives
N 0 = { ( 0 , vec ( F 0 R ) ⊤ , ( − R b 0 ) ⊤ ) ⊤ : R ⊤ = − R } ⊆ Null ( D 0 ) . \mathcal{N}_0=\left\{(0,\operatorname{vec}(\mathbf{F}_0\mathbf{R})^\top,(-\mathbf{R}\mathbf{b}_0)^\top)^\top:\mathbf{R}^\top=-\mathbf{R}\right\}\subseteq\operatorname{Null}(\mathbf{D}_0). N 0 = { ( 0 , vec ( F 0 R ) ⊤ , ( − R b 0 ) ⊤ ) ⊤ : R ⊤ = − R } ⊆ Null ( D 0 ) .
The parameter dimension is 1 + K 2 + K 1+K^2+K 1 + K 2 + K , whereas dim ( N 0 ) = K ( K − 1 ) 2 \dim(\mathcal{N}_0)=\frac{K(K-1)}{2} dim ( N 0 ) = 2 K ( K − 1 ) . Assumption 2.2 gives rank ( D 0 ) ≥ ( K + 1 ) ( K + 2 ) 2 \operatorname{rank}(\mathbf{D}_0)\ge\frac{(K+1)(K+2)}{2} rank ( D 0 ) ≥ 2 ( K + 1 ) ( K + 2 ) . Since N 0 ⊆ Null ( D 0 ) \mathcal{N}_0\subseteq\operatorname{Null}(\mathbf{D}_0) N 0 ⊆ Null ( D 0 ) , the reverse dimension bound also holds, and hence
rank ( D 0 ) = ( K + 1 ) ( K + 2 ) 2 , Null ( D 0 ) = N 0 . \operatorname{rank}(\mathbf{D}_0)=\frac{(K+1)(K+2)}{2},\qquad\operatorname{Null}(\mathbf{D}_0)=\mathcal{N}_0. rank ( D 0 ) = 2 ( K + 1 ) ( K + 2 ) , Null ( D 0 ) = N 0 .
Let
T 0 = { ( δ τ , vec ( H ) ⊤ , δ b ⊤ ) ⊤ : Δ ( F 0 ⊤ H + H ⊤ F 0 ) = 0 } \mathcal{T}_0=\left\{(\delta\tau,\operatorname{vec}(\mathcal{H})^\top,\delta b^\top)^\top:\Delta(\mathbf{F}_0^\top\mathcal{H}+\mathcal{H}^\top\mathbf{F}_0)=0\right\} T 0 = { ( δ τ , vec ( H ) ⊤ , δ b ⊤ ) ⊤ : Δ ( F 0 ⊤ H + H ⊤ F 0 ) = 0 }
be the tangent space of the constraint Δ ( F ⊤ F ) = 0 \Delta(\mathbf{F}^\top\mathbf{F})=0 Δ ( F ⊤ F ) = 0 at γ 0 \gamma_0 γ 0 . Since γ 0 ∈ Γ \gamma_0\in\Gamma γ 0 ∈ Γ , write F 0 ⊤ F 0 = diag ( d 1 , N , J , … , d K , N , J ) \mathbf{F}_0^\top\mathbf{F}_0=\operatorname{diag}(d_{1,N,J},\ldots,d_{K,N,J}) F 0 ⊤ F 0 = diag ( d 1 , N , J , … , d K , N , J ) . Assumption 2.1 gives d k , N , J ⟶ d k d_{k,N,J}\longrightarrow d_k d k , N , J ⟶ d k , k ∈ [ K ] k\in[K] k ∈ [ K ] , where d 1 > ⋯ > d K > 0 d_1>\cdots>d_K>0 d 1 > ⋯ > d K > 0 . Hence, for all sufficiently large N , J N,J N , J ,
min k d k , N , J ≳ 1 , min p ≠ q ∣ d p , N , J − d q , N , J ∣ ≳ 1. \min_k d_{k,N,J}\gtrsim1,\qquad\min_{p\ne q}|d_{p,N,J}-d_{q,N,J}|\gtrsim1. k min d k , N , J ≳ 1 , p = q min ∣ d p , N , J − d q , N , J ∣ ≳ 1.
For
( 0 , vec ( F 0 R ) ⊤ , ( − R b 0 ) ⊤ ) ⊤ ∈ N 0 , (0,\operatorname{vec}(\mathbf{F}_0R)^\top,(-R\mathbf{b}_0)^\top)^\top\in\mathcal{N}_0, ( 0 , vec ( F 0 R ) ⊤ , ( − R b 0 ) ⊤ ) ⊤ ∈ N 0 ,
we have, for p ≠ q p\ne q p = q ,
[ Δ { F 0 ⊤ F 0 R − R F 0 ⊤ F 0 } ] p q = ( d p , N , J − d q , N , J ) R p q . \left[\Delta\{\mathbf{F}_0^\top\mathbf{F}_0R-R\mathbf{F}_0^\top\mathbf{F}_0\}\right]_{pq}=(d_{p,N,J}-d_{q,N,J})R_{pq}. [ Δ { F 0 ⊤ F 0 R − R F 0 ⊤ F 0 } ] pq = ( d p , N , J − d q , N , J ) R pq .
Consequently,
∥ Δ { F 0 ⊤ F 0 R − R F 0 ⊤ F 0 } ∥ F ≳ ∥ R ∥ F . \left\|\Delta\{\mathbf{F}_0^\top\mathbf{F}_0R-R\mathbf{F}_0^\top\mathbf{F}_0\}\right\|_F\gtrsim\|R\|_F. Δ { F 0 ⊤ F 0 R − R F 0 ⊤ F 0 } F ≳ ∥ R ∥ F .
Since F 0 \mathbf{F}_0 F 0 is uniformly nonsingular and b 0 \mathbf{b}_0 b 0 is uniformly bounded, the norm of the corresponding element of N 0 \mathcal{N}_0 N 0 is equivalent to ∥ R ∥ F \|R\|_F ∥ R ∥ F . Thus T 0 ∩ N 0 = { 0 } \mathcal{T}_0\cap\mathcal{N}_0=\{0\} T 0 ∩ N 0 = { 0 } , and the smallest principal angle between T 0 \mathcal{T}_0 T 0 and N 0 \mathcal{N}_0 N 0 is uniformly bounded away from zero.
If F = F 0 + H \mathbf{F}=\mathbf{F}_0+\mathcal{H} F = F 0 + H is feasible, then Δ ( F 0 ⊤ H + H ⊤ F 0 ) = − Δ ( H ⊤ H ) \Delta(\mathbf{F}_0^\top\mathcal{H}+\mathcal{H}^\top\mathbf{F}_0)=-\Delta(\mathcal{H}^\top\mathcal{H}) Δ ( F 0 ⊤ H + H ⊤ F 0 ) = − Δ ( H ⊤ H ) . Let S = Δ ( F 0 ⊤ H + H ⊤ F 0 ) S=\Delta(\mathbf{F}_0^\top\mathcal{H}+\mathcal{H}^\top\mathbf{F}_0) S = Δ ( F 0 ⊤ H + H ⊤ F 0 ) and define a symmetric matrix A \mathcal{A} A with zero diagonal by A p q = S p q d p , N , J + d q , N , J \mathcal{A}_{pq}=\frac{S_{pq}}{d_{p,N,J}+d_{q,N,J}} A pq = d p , N , J + d q , N , J S pq , p ≠ q p\ne q p = q . Since min k d k , N , J ≳ 1 \min_k d_{k,N,J}\gtrsim1 min k d k , N , J ≳ 1 , ∥ A ∥ F ≲ ∥ H ∥ F 2 \|\mathcal{A}\|_F\lesssim\|\mathcal{H}\|_F^2 ∥ A ∥ F ≲ ∥ H ∥ F 2 , and Δ { F 0 ⊤ ( F 0 A ) + ( F 0 A ) ⊤ F 0 } = S \Delta\{\mathbf{F}_0^\top(\mathbf{F}_0\mathcal{A})+(\mathbf{F}_0\mathcal{A})^\top\mathbf{F}_0\}=S Δ { F 0 ⊤ ( F 0 A ) + ( F 0 A ) ⊤ F 0 } = S . Therefore, replacing H \mathcal{H} H by H − F 0 A \mathcal{H}-\mathbf{F}_0\mathcal{A} H − F 0 A produces an element h 0 ∈ T 0 \mathbf{h}_0\in\mathcal{T}_0 h 0 ∈ T 0 satisfying ∥ h − h 0 ∥ ≲ ∥ h ∥ 2 \|\mathbf{h}-\mathbf{h}_0\|\lesssim\|\mathbf{h}\|^2 ∥ h − h 0 ∥ ≲ ∥ h ∥ 2 . By the uniform transversality of T 0 \mathcal{T}_0 T 0 and N 0 \mathcal{N}_0 N 0 , dist ( h 0 , N 0 ) ≳ ∥ h 0 ∥ \operatorname{dist}(\mathbf{h}_0,\mathcal{N}_0) \gtrsim\|\mathbf{h}_0\| dist ( h 0 , N 0 ) ≳ ∥ h 0 ∥ . Hence
dist ( h , N 0 ) ≥ dist ( h 0 , N 0 ) − ∥ h − h 0 ∥ \operatorname{dist}(\mathbf{h},\mathcal{N}_0) \ge\operatorname{dist}(\mathbf{h}_0,\mathcal{N}_0) - \|\mathbf{h}-\mathbf{h}_0\| dist ( h , N 0 ) ≥ dist ( h 0 , N 0 ) − ∥ h − h 0 ∥
≳ ∥ h ∥ − O ( ∥ h ∥ 2 ) \gtrsim\|\mathbf{h}\| - O(\|\mathbf{h}\|^2) ≳ ∥ h ∥ − O ( ∥ h ∥ 2 )
≳ ∥ h ∥ \gtrsim\|\mathbf{h}\| ≳ ∥ h ∥
for all sufficiently small ∥ h ∥ \|\mathbf{h}\| ∥ h ∥ . Equivalently, ∥ Π N 0 ⊥ h ∥ ≳ ∥ h ∥ \|\Pi_{\mathcal{N}_0^\perp}\mathbf{h}\| \gtrsim\|\mathbf{h}\| ∥ Π N 0 ⊥ h ∥ ≳ ∥ h ∥ .
Since Null ( D 0 ) = N 0 \operatorname{Null}(\mathbf{D}_0)=\mathcal{N}_0 Null ( D 0 ) = N 0 , Assumption 2.2 gives
∥ D 0 h ∥ = ∥ D 0 Π N 0 ⊥ h ∥ ≳ q N J ∥ h ∥ . \|\mathbf{D}_0\mathbf{h}\|=\|\mathbf{D}_0\Pi_{\mathcal{N}_0^\perp}\mathbf{h}\|\gtrsim q_{NJ}\|\mathbf{h}\|. ∥ D 0 h ∥ = ∥ D 0 Π N 0 ⊥ h ∥ ≳ q N J ∥ h ∥.
The second derivatives of m i j ( γ ; Z ∗ ) m_{ij}(\gamma; \mathbf{Z}^*) m ij ( γ ; Z ∗ ) are uniformly bounded in a fixed neighborhood of γ 0 \gamma_0 γ 0 , so Taylor’s theorem yields
∥ vec ( M ( γ ; Z ∗ ) − M ∗ ) − D 0 h ∥ ≲ N J ∥ h ∥ 2 . \|\operatorname{vec}(\mathbf{M}(\gamma;\mathbf{Z}^*)-\mathbf{M}^*)-\mathbf{D}_0\mathbf{h}\|\lesssim\sqrt{NJ}\|\mathbf{h}\|^2. ∥ vec ( M ( γ ; Z ∗ ) − M ∗ ) − D 0 h ∥ ≲ N J ∥ h ∥ 2 .
Since ∥ h ∥ ≤ ρ N J \|\mathbf{h}\|\le\rho_{NJ} ∥ h ∥ ≤ ρ N J and ρ N J = o ( q N J N J ) \rho_{NJ}=o\left(\frac{q_{NJ}}{\sqrt{NJ}}\right) ρ N J = o ( N J q N J ) , the remainder is o ( q N J ∥ h ∥ ) o(q_{NJ}\|\mathbf{h}\|) o ( q N J ∥ h ∥ ) uniformly. This proves (118).
Since the sign transformation defining [ γ 0 ] [\gamma_0] [ γ 0 ] is an isometry, preserves Γ \Gamma Γ , and leaves M ( γ ; Z ∗ ) \mathbf{M}(\gamma;\mathbf{Z}^*) M ( γ ; Z ∗ ) invariant, the same bound holds in a ρ N J \rho_{NJ} ρ N J -neighborhood of every element of [ γ 0 ] [\gamma_0] [ γ 0 ] .
Finally, (115), (116), and ∥ M ^ − M ∗ ∥ F ≤ N J ξ \|\widehat{\mathbf{M}}-\mathbf{M}^*\|_F\le\sqrt{NJ}\,\xi ∥ M − M ∗ ∥ F ≤ N J ξ imply
∥ M ( γ ~ ; Z ∗ ) − M ∗ ∥ F ≲ N J ξ . \|\mathbf{M}(\widetilde{\gamma};\mathbf{Z}^*)-\mathbf{M}^*\|_F\lesssim\sqrt{NJ}\,\xi. ∥ M ( γ ; Z ∗ ) − M ∗ ∥ F ≲ N J ξ .
Combining this with (117) and (118) gives
q N J inf γ ~ ∈ [ γ 0 ] ∥ γ ~ − γ ~ 0 ∥ ≲ N J ξ . q_{NJ}\inf_{\widetilde{\gamma}\in[\gamma_0]}\|\widetilde{\gamma}-\widetilde{\gamma}_0\|\lesssim\sqrt{NJ}\,\xi. q N J γ ∈ [ γ 0 ] inf ∥ γ − γ 0 ∥ ≲ N J ξ .
Since [ γ 0 ] [\gamma_0] [ γ 0 ] is finite, the infimum is attained by some γ ~ 0 ∈ [ γ 0 ] \widetilde{\gamma}_0\in[\gamma_0] γ 0 ∈ [ γ 0 ] . By the definition of the sign-equivalence class, there exists Q ∈ S K \mathbf{Q} \in\mathcal{S}_K Q ∈ S K such that
γ ~ 0 = ( τ 0 , vec ( F 0 Q ) T , ( Q T b 0 ) T ) T . \widetilde{\gamma}_0 = \left(\tau_0, \operatorname{vec}(\mathbf{F}_0\mathbf{Q})^{\mathsf{T}},(\mathbf{Q}^{\mathsf{T}}\mathbf{b}_0)^{\mathsf{T}}\right)^{\mathsf{T}}. γ 0 = ( τ 0 , vec ( F 0 Q ) T , ( Q T b 0 ) T ) T .
Hence,
∥ γ ‾ − γ ~ 0 ∥ ≲ N J q N J ξ , ∥ γ ‾ − γ ~ 0 ∥ < ρ N J . \lVert\overline{\gamma}-\widetilde{\gamma}_0\rVert\lesssim\frac{\sqrt{NJ}}{q_{NJ}}\xi,\qquad\lVert\overline{\gamma}-\widetilde{\gamma}_0\rVert< \rho_{NJ}. ∥ γ − γ 0 ∥ ≲ q N J N J ξ , ∥ γ − γ 0 ∥ < ρ N J .
Since K K K is fixed, equivalence of norms yields
∣ τ ^ − τ 0 ∣ + ∥ Q Z ^ F ^ − F 0 Q ∥ F + ∥ b ^ − Q T b 0 ∥ ≲ N J q N J ξ . |\widehat{\tau}-\tau_0|+\lVert\mathbf{Q}\widehat{\mathbf{Z}}\widehat{\mathbf{F}}-\mathbf{F}_0\mathbf{Q}\rVert_F+\lVert\widehat{\mathbf{b}}-\mathbf{Q}^{\mathsf{T}}\mathbf{b}_0\rVert\lesssim\frac{\sqrt{NJ}}{q_{NJ}}\xi. ∣ τ − τ 0 ∣ + ∥ Q Z F − F 0 Q ∥ F + ∥ b − Q T b 0 ∥ ≲ q N J N J ξ .
This proves the lemma.
Lemma 14. Suppose Assumption 1.1 and Assumption 2.1 hold. Then there exist fixed constants η 0 > 0 \eta_0 > 0 η 0 > 0 and C T > 0 C_T > 0 C T > 0 such that, for any
γ = ( τ , vec ( F ) T , b T ) T , Z , Q ∈ S K \gamma= \left(\tau,\operatorname{vec}(\mathbf{F})^{\mathsf{T}},\mathbf{b}^{\mathsf{T}}\right)^{\mathsf{T}},\qquad\mathbf{Z},\qquad\mathbf{Q} \in\mathcal{S}_K γ = ( τ , vec ( F ) T , b T ) T , Z , Q ∈ S K satisfying
∣ τ − τ 0 ∣ + ∥ F − F 0 Q ∥ F + ∥ b − Q T b 0 ∥ + ∥ Z − Z ∗ ∥ 2 → ∞ ≤ η 0 , |\tau-\tau_0|+\lVert\mathbf{F}-\mathbf{F}_0\mathbf{Q}\rVert_F+\lVert\mathbf{b}-\mathbf{Q}^{\mathsf{T}}\mathbf{b}_0\rVert+\lVert\mathbf{Z}-\mathbf{Z}^{*}\rVert_{2\to\infty}\leq\eta_0, ∣ τ − τ 0 ∣ + ∥ F − F 0 Q ∥ F + ∥ b − Q T b 0 ∥ + ∥ Z − Z ∗ ∥ 2 → ∞ ≤ η 0 , we have
max s = 1 , 2 ∥ Θ P s ( γ ; Z ) − Θ P s Q ∗ ∥ 2 → ∞ ≤ C T { ∥ F − F 0 Q ∥ F + ∥ b − Q T b 0 ∥ + ∥ Z − Z ∗ ∥ 2 → ∞ } . \max_{s=1,2}\lVert\boldsymbol{\Theta}_{P_s}(\gamma;\mathbf{Z})-\boldsymbol{\Theta}^{*}_{P_s\mathbf{Q}}\rVert_{2\to\infty}\leq C_T\left\{\lVert\mathbf{F}-\mathbf{F}_0\mathbf{Q}\rVert_F+\lVert\mathbf{b}-\mathbf{Q}^{\mathsf{T}}\mathbf{b}_0\rVert+\lVert\mathbf{Z}-\mathbf{Z}^{*}\rVert_{2\to\infty}\right\}. s = 1 , 2 max ∥ Θ P s ( γ ; Z ) − Θ P s Q ∗ ∥ 2 → ∞ ≤ C T { ∥ F − F 0 Q ∥ F + ∥ b − Q T b 0 ∥ + ∥ Z − Z ∗ ∥ 2 → ∞ } .
Proof. Write Z = ( N U T , J W T ) T \mathbf{Z}=\left(\sqrt{N}\mathbf{U}^{\mathsf{T}},\sqrt{J}\mathbf{W}^{\mathsf{T}}\right)^{\mathsf{T}} Z = ( N U T , J W T ) T . For η 0 \eta_0 η 0 sufficiently small, F \mathbf{F} F and F − 1 \mathbf{F}^{-1} F − 1 are uniformly bounded. By (8) ,
∥ Θ P 1 ( γ ; Z ) − Θ P 1 ∗ Q ∥ 2 → ∞ = ∥ N U F − N U ∗ F 0 Q ∥ 2 → ∞ ≲ ∥ Z − Z ∗ ∥ 2 → ∞ + ∥ F − F 0 Q ∥ F . \left\|\boldsymbol{\Theta}_{\mathcal{P}_1}(\gamma;\boldsymbol{Z})-\boldsymbol{\Theta}_{\mathcal{P}_1}^{*}\boldsymbol{Q}\right\|_{2\to\infty}
=
\left\|\sqrt{N}\boldsymbol{U}\boldsymbol{F}-\sqrt{N}\boldsymbol{U}^{*}\boldsymbol{F}_0\boldsymbol{Q}\right\|_{2\to\infty}
\lesssim
\left\|\boldsymbol{Z}-\boldsymbol{Z}^{*}\right\|_{2\to\infty}
+
\left\|\boldsymbol{F}-\boldsymbol{F}_0\boldsymbol{Q}\right\|_{\mathrm{F}}. Θ P 1 ( γ ; Z ) − Θ P 1 ∗ Q 2 → ∞ = N U F − N U ∗ F 0 Q 2 → ∞ ≲ ∥ Z − Z ∗ ∥ 2 → ∞ + ∥ F − F 0 Q ∥ F .
For the second configuration, since ( F 0 Q ) − T = F 0 − T Q (\boldsymbol{F}_0\boldsymbol{Q})^{-\mathsf{T}}=\boldsymbol{F}_0^{-\mathsf{T}}\boldsymbol{Q} ( F 0 Q ) − T = F 0 − T Q and ( Q T b 0 ) T = b 0 T Q (\boldsymbol{Q}^{\mathsf{T}}\boldsymbol{b}_0)^{\mathsf{T}}=\boldsymbol{b}_0^{\mathsf{T}}\boldsymbol{Q} ( Q T b 0 ) T = b 0 T Q , we have
∥ Θ P 2 ( γ ; Z ) − Θ P 2 ∗ Q ∥ 2 → ∞ ≤ J ∥ W − W ∗ ∥ 2 → ∞ ∥ F − T ∥ + J ∥ W ∗ ∥ 2 → ∞ ∥ F − T − ( F 0 Q ) − T ∥ + ∥ b − Q T b 0 ∥ . \left\|\boldsymbol{\Theta}_{\mathcal{P}_2}(\gamma;\boldsymbol{Z})-\boldsymbol{\Theta}_{\mathcal{P}_2}^{*}\boldsymbol{Q}\right\|_{2\to\infty}
\le
\sqrt{J}\left\|\boldsymbol{W}-\boldsymbol{W}^{*}\right\|_{2\to\infty}\left\|\boldsymbol{F}^{-\mathsf{T}}\right\|
+
\sqrt{J}\left\|\boldsymbol{W}^{*}\right\|_{2\to\infty}\left\|\boldsymbol{F}^{-\mathsf{T}}-(\boldsymbol{F}_0\boldsymbol{Q})^{-\mathsf{T}}\right\|
+
\left\|\boldsymbol{b}-\boldsymbol{Q}^{\mathsf{T}}\boldsymbol{b}_0\right\|. Θ P 2 ( γ ; Z ) − Θ P 2 ∗ Q 2 → ∞ ≤ J ∥ W − W ∗ ∥ 2 → ∞ F − T + J ∥ W ∗ ∥ 2 → ∞ F − T − ( F 0 Q ) − T + b − Q T b 0 .
Since F \boldsymbol{F} F and F 0 Q \boldsymbol{F}_0\boldsymbol{Q} F 0 Q are uniformly nonsingular, ∥ F − T − ( F 0 Q ) − T ∥ ≲ ∥ F − F 0 Q ∥ F \left\|\boldsymbol{F}^{-\mathsf{T}}-(\boldsymbol{F}_0\boldsymbol{Q})^{-\mathsf{T}}\right\|\lesssim\left\|\boldsymbol{F}-\boldsymbol{F}_0\boldsymbol{Q}\right\|_{\mathrm{F}} F − T − ( F 0 Q ) − T ≲ ∥ F − F 0 Q ∥ F . The result follows.
Proof of Theorem 4
Proof. Let Q Z ∈ S K \boldsymbol{Q}_{Z}\in S_{K} Q Z ∈ S K be given by Lemma 12 . By Lemma 13 , the minimum in (10) is attained. Let
γ ^ = ( τ ^ , vec ( F ^ ) T , b ^ T ) T \widehat{\gamma}=\left(\widehat{\tau},\operatorname{vec}(\widehat{\boldsymbol{F}})^{\mathsf{T}},\widehat{\boldsymbol{b}}^{\mathsf{T}}\right)^{\mathsf{T}} γ = ( τ , vec ( F ) T , b T ) T
be any global minimizer, and define the aligned parameter
γ ‾ = ( τ ^ , vec ( Q Z F ^ ) T , b ^ T ) T . \overline{\gamma}=\left(\widehat{\tau},\operatorname{vec}(\boldsymbol{Q}_{Z}\widehat{\boldsymbol{F}})^{\mathsf{T}},\widehat{\boldsymbol{b}}^{\mathsf{T}}\right)^{\mathsf{T}}. γ = ( τ , vec ( Q Z F ) T , b T ) T .
Lemma 13 gives a Q ∈ S K \boldsymbol{Q}\in S_{K} Q ∈ S K , attaining the infimum in (115) , such that
∥ γ ‾ − ( τ 0 , vec ( F 0 Q ) T , ( Q T b 0 ) T ) T ∥ < ρ N J = o ( 1 ) , \left\|\overline{\gamma}-\left(\tau_0,\operatorname{vec}(\boldsymbol{F}_0\boldsymbol{Q})^{\mathsf{T}},(\boldsymbol{Q}^{\mathsf{T}}\boldsymbol{b}_0)^{\mathsf{T}}\right)^{\mathsf{T}}\right\|<\rho_{NJ}=o(1), γ − ( τ 0 , vec ( F 0 Q ) T , ( Q T b 0 ) T ) T < ρ N J = o ( 1 ) ,
and
∣ τ ^ − τ 0 ∣ + ∥ Q Z F ^ − F 0 Q ∥ F + ∥ b ^ − Q T b 0 ∥ ≲ N J q N J ξ . |\widehat{\tau}-\tau_0|+\lVert Q_Z\widehat{F}-F_0Q\rVert_F+\lVert\widehat{b}-Q^{\mathsf{T}}b_0\rVert\lesssim\frac{\sqrt{NJ}}{q_{NJ}}\xi. ∣ τ − τ 0 ∣ + ∥ Q Z F − F 0 Q ∥ F + ∥ b − Q T b 0 ∥ ≲ q N J N J ξ .
Moreover, Lemma 12 gives ∥ Z ^ Q Z − Z ∗ ∥ 2 → ∞ ≲ ξ = o ( 1 ) \lVert\widehat{Z}Q_Z-Z^*\rVert_{2\to\infty}\lesssim\xi=o(1) ∥ Z Q Z − Z ∗ ∥ 2 → ∞ ≲ ξ = o ( 1 ) . Therefore, for all sufficiently large N N N and J J J , the localization bound and the alignment bound place γ ‾ \overline{\gamma} γ and Z ^ Q Z \widehat{Z}Q_Z Z Q Z inside the fixed neighborhood required by Lemma 14.
Applying Lemma 14 with Z = Z ^ Q Z Z=\widehat{Z}Q_Z Z = Z Q Z , F = Q Z F ^ F=Q_Z\widehat{F} F = Q Z F , we obtain
max s = 1 , 2 ∥ Θ P s ( γ ‾ ; Z ^ Q Z ) − Θ P s ∗ Q ∥ 2 → ∞ ≲ ξ + N J q N J ξ . \max_{s=1,2}\left\lVert\Theta_{\mathcal{P}_s}\left(\overline{\gamma};\widehat{Z}Q_Z\right)-\Theta_{\mathcal{P}_s}^*Q\right\rVert_{2\to\infty}\lesssim\xi+\frac{\sqrt{NJ}}{q_{NJ}}\xi. s = 1 , 2 max Θ P s ( γ ; Z Q Z ) − Θ P s ∗ Q 2 → ∞ ≲ ξ + q N J N J ξ .
Since the rows of ∂ M ( γ 0 ; Z ∗ ) \partial M(\gamma_0;Z^*) ∂ M ( γ 0 ; Z ∗ ) are uniformly bounded and the parameter dimension is fixed, Assumption 2.2 implies q N J ≲ N J q_{NJ}\lesssim\sqrt{NJ} q N J ≲ N J . Therefore, ξ ≲ N J q N J ξ \xi\lesssim\frac{\sqrt{NJ}}{q_{NJ}}\xi ξ ≲ q N J N J ξ , and hence
max s = 1 , 2 ∥ Θ P s ( γ ‾ ; Z ^ Q Z ) − Θ P s ∗ Q ∥ 2 → ∞ ≲ N J q N J ξ . \max_{s=1,2}\left\lVert\Theta_{\mathcal{P}_s}\left(\overline{\gamma};\widehat{Z}Q_Z\right)-\Theta_{\mathcal{P}_s}^*Q\right\rVert_{2\to\infty}\lesssim\frac{\sqrt{NJ}}{q_{NJ}}\xi. s = 1 , 2 max Θ P s ( γ ; Z Q Z ) − Θ P s ∗ Q 2 → ∞ ≲ q N J N J ξ .
Finally, Lemma 12 gives, for s = 1 , 2 s=1,2 s = 1 , 2 ,
Θ P s ( γ ‾ ; Z ^ Q Z ) = Θ P s ( γ ^ ; Z ^ ) . \Theta_{\mathcal{P}_s}\left(\overline{\gamma};\widehat{Z}Q_Z\right)=\Theta_{\mathcal{P}_s}\left(\widehat{\gamma};\widehat{Z}\right). Θ P s ( γ ; Z Q Z ) = Θ P s ( γ ; Z ) .
Thus,
∥ Θ ( γ ^ ; Z ^ ) − Θ ∗ Q ∥ 2 → ∞ = max s = 1 , 2 ∥ Θ P s ( γ ^ ; Z ^ ) − Θ P s ∗ Q ∥ 2 → ∞ ≲ N J q N J ξ . \left\lVert\Theta(\widehat{\gamma};\widehat{Z})-\Theta^*Q\right\rVert_{2\to\infty}
=\max_{s=1,2}\left\lVert\Theta_{\mathcal{P}_s}(\widehat{\gamma};\widehat{Z})-\Theta_{\mathcal{P}_s}^*Q\right\rVert_{2\to\infty}
\lesssim\frac{\sqrt{NJ}}{q_{NJ}}\xi. Θ ( γ ; Z ) − Θ ∗ Q 2 → ∞ = s = 1 , 2 max Θ P s ( γ ; Z ) − Θ P s ∗ Q 2 → ∞ ≲ q N J N J ξ .
Since Q ∈ S K ⊂ O K Q\in S_K\subset O_K Q ∈ S K ⊂ O K , taking this Q Q Q and zero translation in the definition of d c d_c d c gives
d c ( Θ ( γ ^ ; Z ^ ) , Θ ∗ ) ≲ N J q N J ξ . d_c\left(\Theta(\widehat{\gamma};\widehat{Z}),\Theta^*\right)\lesssim\frac{\sqrt{NJ}}{q_{NJ}}\xi. d c ( Θ ( γ ; Z ) , Θ ∗ ) ≲ q N J N J ξ .
This completes the proof.
Proof of Corollary 3
Proof. Let M ^ \widehat{\mathbf{M}} M denote the output of Algorithm 1. By Theorem 1 when f ( x ) = x f(x)=x f ( x ) = x and Corollary 2 when f f f is nonlinear,
∥ M ^ − M ∗ ∥ max = O p ( δ N J ) . \left\|\widehat{\mathbf{M}}-\mathbf{M}^{*}\right\|_{\max}=O_{p}(\delta_{NJ}). M − M ∗ m a x = O p ( δ N J ) .
Under the standing condition N = J O ( 1 ) N=J^{O(1)} N = J O ( 1 ) , δ N J = o ( 1 ) \delta_{NJ}=o(1) δ N J = o ( 1 ) in both cases.
Fix any ε > 0 \varepsilon>0 ε > 0 . By the definition of O p ( δ N J ) O_{p}(\delta_{NJ}) O p ( δ N J ) , there exists a constant M ε > 0 M_{\varepsilon}>0 M ε > 0 such that, for all sufficiently large N N N and J J J ,
Pr ( ∥ M ^ − M ∗ ∥ max ≤ M ε δ N J ) ≥ 1 − ε . \Pr\left(\left\|\widehat{\mathbf{M}}-\mathbf{M}^{*}\right\|_{\max}\leq M_{\varepsilon}\delta_{NJ}\right)\geq1-\varepsilon. Pr ( M − M ∗ m a x ≤ M ε δ N J ) ≥ 1 − ε .
Denote this event by E N , J , ε \mathcal{E}_{N,J,\varepsilon} E N , J , ε .
For unipartite data, on E N , J , ε \mathcal{E}_{N,J,\varepsilon} E N , J , ε , Theorem 3 gives
d c ( Θ ^ , Θ ∗ ) ≲ M ε δ N J . d_{c}(\widehat{\boldsymbol{\Theta}},\boldsymbol{\Theta}^{*})\lesssim M_{\varepsilon}\delta_{NJ}. d c ( Θ , Θ ∗ ) ≲ M ε δ N J .
Since ε > 0 \varepsilon>0 ε > 0 is arbitrary,
d c ( Θ ^ , Θ ∗ ) = O p ( δ N J ) . d_{c}(\widehat{\boldsymbol{\Theta}},\boldsymbol{\Theta}^{*})=O_{p}(\delta_{NJ}). d c ( Θ , Θ ∗ ) = O p ( δ N J ) .
For bipartite data, δ N J 2 = o ( c N J ) \delta_{NJ}^{2}=o(c_{NJ}) δ N J 2 = o ( c N J ) implies ( M ε δ N J ) 2 = o ( c N J ) (M_{\varepsilon}\delta_{NJ})^{2}=o(c_{NJ}) ( M ε δ N J ) 2 = o ( c N J ) . Together with ρ N J = o ( q N J / N J ) \rho_{NJ}=o(q_{NJ}/\sqrt{NJ}) ρ N J = o ( q N J / N J ) , Theorem 4 gives, on E N , J , ε \mathcal{E}_{N,J,\varepsilon} E N , J , ε ,
d c ( Θ ^ , Θ ∗ ) ≲ M ε N J q N J δ N J . d_{c}(\widehat{\boldsymbol{\Theta}},\boldsymbol{\Theta}^{*})\lesssim M_{\varepsilon}\frac{\sqrt{NJ}}{q_{NJ}}\delta_{NJ}. d c ( Θ , Θ ∗ ) ≲ M ε q N J N J δ N J .
Therefore,
d c ( Θ ^ , Θ ∗ ) = O p ( N J q N J δ N J ) . d_{c}(\widehat{\boldsymbol{\Theta}},\boldsymbol{\Theta}^{*})=O_{p}\left(\frac{\sqrt{NJ}}{q_{NJ}}\delta_{NJ}\right). d c ( Θ , Θ ∗ ) = O p ( q N J N J δ N J ) .
If the response supports are uniformly bounded, the sharper nonlinear-link rate follows directly from Corollary 2.