betaFeedback

WorksMathematics

Empirical Variational Bayes Factorization for Longitudinal Matrix Data

Abstract

Collecting high-dimensional data over time for multiple individuals is increasingly common in biomedical research, particularly in studies of disease progression or molecular time-course experiments. Often data are missing or the timepoints are irregularly spaced across individuals. There is a lack of flexible methodology to explore latent patterns of change over time and account for missing data. We propose an empirical variational Bayes (EVB) approach to factorize high-dimensional longitudinal data, in which the underlying factors have temporal correlation over time. The EVB approach is computationally efficient and accommodates flexible covariance structures over time with automatic determination of all hyper-parameters (including the underlying rank, i.e., number of factors). Simulations demonstrate that the approach captures the underlying signal more precisely than other strategies, and performs well with respect to missing data imputation under different scenarios. An application to a gene time-course data for multiple sclerosis treatment provides a factorization with interpretable components over time and improved imputation accuracy for missing data.

keywords: Dimension reduction; Gaussian process; Low-rank approximation; Temporal correlation; Variational Bayes

Introduction

Matrix factorization methods have long been critical to analyze and summarize high-dimensional data for diverse tasks such as latent structure inference, data compression, and missing data imputation. Recently, empirical variational Bayes has been shown to be a powerful, flexible, and efficient approach to low-rank matrix factorization. Under a variational Bayes approach the true posterior for a Bayesian model is approximated by a variational distribution with simplified assumptions [1], under EVB the model hyperparameters are also empirically determined. Previous work describes a simple EVB approach to matrix factorization that results in an efficient closed-form low-rank solution [2]. Empirical Bayes matrix factorization flexibly incorporates different forms of priors, allowing novel combinations of classical models such as sparse matrix factorization and non-negative matrix factorization [3]. In the context of multiple linked matrices, EVB has been used to decompose signal into low-rank approximations for shared and unshared structure [4].

Advancements in biotechnology have enabled high-throughput data collection (e.g., for “omics” or imaging modalities) from multiple timepoints, motivating longitudinal studies. However, in most cases it is difficult to obtain complete data, often caused by patient unavailability or dropout. This can lead to several issues, such as biased results and a reduction in statistical power, compromising the validity of the study and the reliability of its findings. As a motivating example, consider a study that investigates genomic responses of multiple sclerosis (MS) patients to the genetic therapy recombinant human interferon beta (rIFNβ\beta) [5]. The study involves analyzing a gene time course dataset that measures 76 gene expression levels in 53 subjects, at 7 timepoints over a 2-year followup period. However, 10.5% of the data are missing across all timepoints, limiting the validity of inference.

Several classes of methods have been developed to apply factorization models to longitudinal data settings. One popular area of research are tensor factorization methods [6].

The tensor framework generalizes the two-way matrix to higher-order arrays. Tensor factorization methods enjoy many desirable properties of matrix factorization, such as low rank representation of the data and interpretable additive components. They can conveniently accommodate longitudinally collected high-dimensional matrices by adding time as a third dimension, forming a third-order tensor array. Tensor factorization methods have been developed specifically to accommodate temporal structure, by incorporating functional representations or smoothing along the temporal dimension [7, 8]. Other tensor methods have been developed for imputation in a longitudinal setting using a tensor factorization [9]. However, treating time as an additional dimension within a traditional tensor framework assumes that timepoints are aligned across individuals.

For matrices, a related class of methods incorporate auxiliary side information on rows or columns to inform matrix factorization. For example, [10] considers additional matrices of covariates for the rows and columns, and borrows information to factorize all simultaneously. Variational Bayes has enabled computationally efficient posterior approximation for matrix factorization under structured Bayesian priors that incorporate side information for rows or columns [11]; however, this requires model-specific hyperparameters that are not empirically determined. More recently, [12] describe an empirical variational Bayes approach in which auxiliary covariates inform the latent factor means. While time can be incorporated as a covariate in such methods to capture mean trends over time, they do not inherently accommodate within-individual correlation structures over time that are common for longitudinal studies.

A small but growing number of matrix factorization models are explicitly designed for longitudinal settings. For example, [13] use a factor analysis approach and models the scores using a multivariate linear mixed model. This induces correlation within individual observations, but limits flexibility to model the correlation to the specific form under a linear mixed model. Bayesian methods, on the other hand, can deal with more complex correlation structures by naturally incorporating flexible priors for the factors. For example, [14] uses a factor model for random effects within a mixed modeling framework, where the scores are employed with an infinite mixture prior. However, this work centered on the task of clustering instead of signal estimation or imputation, which is the focus of this paper. Alternatively, [15] propose a longitudinal factor model for disease progression from high-dimensional data, but assume a transition between two latent states over time rather than a more general correlation structure.

We propose an empirical variation Bayes factor model for high-dimensional longitudinal data that simultaneously addresses estimation and imputation, even when timepoints are inconsistently measured. Our contribution has four main components. First, we introduce a general repeated measures model for high-dimensional data using latent factors that flexibly accommodates a range of correlation structures over time. Second, we use an EVB framework for estimation that does not require any user-specified tuning parameters, and that achieves automatic rank selection—an important advantage since manual rank selection can be difficult or computationally prohibitive for large datasets. Third, we develop a novel empirical Bayes random process model (EBRPM), which extends prior work to allow efficient computation and unified estimation of both priors and variationally approximated posteriors. This formulation leverages empirical Bayes principles to maximize efficiency while retaining the ability to recover complex signals in high-dimensional settings. Fourth, our framework naturally handles missing data, including both entrywise and blockwise missingness, by retaining a key Bayesian property: the sharing of information across individuals to estimate a common prior distribution. This feature is especially important for blockwise missingness, where data for an individual may be entirely absent at a timepoint and imputation without borrowing information and temporal correlation is impossible.

Methods

Model Construction

Here we describe our model for aligned measurement times across all subjects; extensions to scenarios with irregularly measured timepoints across subjects are straightforward, and are described in § 10.1 of the Supplementary Materials.

Assume there are nn subjects, each with pp variables measured at t1,…,tDt_1,\ldots,t_D timepoints. We define the rank RR model as

Yt=UVtT+Et, t=t1,…,tD(1)\mathbf{Y}_t = U\mathbf{V}_t^T + \mathbf{E}_t,\ t=t_1,\ldots,t_D \tag*{(1)}

where Yt∈Rp×n\mathbf{Y}_t \in\mathbb{R}^{p \times n} are the observed data at each timepoint, U∈Rp×RU \in\mathbb{R}^{p \times R}, Vt∈Rn×R\mathbf{V}_t \in\mathbb{R}^{n \times R} are factor matrices and Et\mathbf{E}_t is an error matrix at time tt. Following standard terminology, we refer to UU as the loading matrix, VV as the score matrix, and RR as the rank/number of factors. The model can be expanded as the sum RR rank-1 components, via the outer products between columns of UU and Vt\mathbf{V}_t,

Yt=∑r=1RurvtrT+Et(2)\mathbf{Y}_t = \sum_{r=1}^{R} \mathbf{u}_r\mathbf{v}_{tr}^{T} + \mathbf{E}_t \tag*{(2)}

where ur\mathbf{u}_r and vtr\mathbf{v}_{tr} denote the rr-th column of UU and Vt\mathbf{V}_t, respectively. Following a Bayesian framework, we give independent normal priors to each column of UU, and a joint normal prior to v~ri:=[vt1ri,…,vtDri]\tilde{\mathbf{v}}_{ri} := [v_{t_1ri},\ldots,v_{t_Dri}], the corresponding entries of Vt\mathbf{V}_t across timepoints:

urj∼iidN(0,ηr2), v~ri∼iidmvN⁡(0,Kr), Etji∼iidN(0,σtji2)(3)u_{rj} \overset{\mathrm{iid}}{\sim} N(0,\eta_r^2),\ \tilde{\mathbf{v}}_{ri} \overset{\mathrm{iid}}{\sim} \operatorname{mvN}(0,K_r),\ E_{tji} \overset{\mathrm{iid}}{\sim} N(0,\sigma_{tji}^2) \tag*{(3)}

where KrK_r captures component-specific temporal correlation in the factor scores, as detailed in § 2.4. The use of normal priors with factor-specific variances can be considered an automatic relevance determination (ARD) prior [16, 17, 18, 19].

Empirical Variational Bayes Estimation

We begin by describing the estimation algorithm for the rank 1 model, and generalize to the multiple rank scenario in § 2.3. In a standard empirical Bayes framework, model estimation proceeds by first obtaining point estimates for the prior parameters η2\eta^2, K\mathbf{K}, σ2\sigma^2, e.g., by maximizing the marginal log-likelihood under estimated priors g(⋅)g(\cdot):

(η^,K^,σ^2)=arg⁡max⁡η,K,σ2log⁡∫ ⁣ ⁣∫∏t=t1tDp(Yt∣u,vt,σ2)g(vt∣K)g(u∣η) dv du,(4)(\hat{\eta},\hat{\mathbf{K}},\hat{\sigma}^2)=\underset{\eta,\mathbf{K},\sigma^2}{\arg\max}\log\int\!\!\int\prod_{t=t_1}^{t_D}p(\mathbf{Y}_t\mid\mathbf{u},\mathbf{v}_t,\sigma^2)g(\mathbf{v}_t\mid\mathbf{K})g(\mathbf{u}\mid\eta)\,d\mathbf{v}\,d\mathbf{u}, \tag*{(4)}

then performing inference on u\mathbf{u}, v\mathbf{v} via the posterior distribution

p(u,v∣Y,η^,K^,σ^2).(5)p(\mathbf{u},\mathbf{v}\mid\mathbf{Y},\hat{\eta},\hat{\mathbf{K}},\hat{\sigma}^2). \tag*{(5)}

Approximate Inference

The integrals involved in both (4) and (5) are intractable due to marginalizing over latent variables. Therefore, we must resort to approximate inference techniques. Markov Chain Monte Carlo (MCMC) [20] is the most common technique where approximate samples are drawn from the true posterior, however it often suffers from high computational demands for large-scale problems. Instead, we propose a computationally attractive variational Bayes solution for our setting.

In variational Bayes, posterior estimation is posed as an optimization problem, where we search for a factor distribution q(u,v)q(\mathbf{u},\mathbf{v}) that gives the smallest KL divergence with the true posterior:

q∗(u,v)=arg⁡min⁡q∈Q KL(q(u,v)∥p(u,v∣Y))q^*(\mathbf{u},\mathbf{v})=\underset{q\in\mathcal{Q}}{\arg\min}\ \mathrm{KL}(q(\mathbf{u},\mathbf{v})\|p(\mathbf{u},\mathbf{v}\mid\mathbf{Y})) [21]. This is done by maximizing the evidence lower bound (ELBO),

FELBO(q,η,K,σ):=Eq[log⁡p(Y,u,v;η,K,σ)q(u,v)]=l(η,K,σ)−KL(qu,v∥pu,v∣Y,η,K,σ)(6)F_{ELBO}(q,\eta,\mathbf{K},\sigma):=E_q\left[\log\frac{p(\mathbf{Y},\mathbf{u},\mathbf{v};\eta,\mathbf{K},\sigma)}{q(\mathbf{u},\mathbf{v})}\right] = l(\eta,\mathbf{K},\sigma)-\mathrm{KL}(q_{\mathbf{u},\mathbf{v}}\|p_{\mathbf{u},\mathbf{v}\mid\mathbf{Y},\eta,\mathbf{K},\sigma}) \tag*{(6)}

which is a lower bound for the log marginal likelihood, as the KL divergence is non-negative. Since l(η,K,σ)l(\eta,\mathbf{K},\boldsymbol{\sigma}) is constant with respect to q(u,v)q(\mathbf{u},\mathbf{v}),

arg⁡max⁡q F(q,η,K,σ)=arg⁡min⁡q KL⁡(qu,v∥pu,v∣Y,η,K,σ).\underset{q}{\arg\max}\ F(q,\eta,\mathbf{K},\boldsymbol{\sigma}) = \underset{q}{\arg\min}\ \operatorname{KL}(q_{\mathbf{u},\mathbf{v}}\Vert p_{\mathbf{u},\mathbf{v}\mid\mathbf{Y},\eta,\mathbf{K},\boldsymbol{\sigma}}).

However, without further constraints on qq, this optimization is difficult. We restrict the space of factor distributions to the mean field variational family,

Q={q(u,v):q(u,v)=∏j=1pquj(uj)∏i=1nqv~i(v~i)}(7)\mathcal{Q} = \left\{q(\mathbf{u},\mathbf{v}):q(\mathbf{u},\mathbf{v})=\prod_{j=1}^{p}q_{u_j}(u_j)\prod_{i=1}^{n}q_{\tilde{v}_i}(\tilde{v}_i)\right\} \tag*{(7)}

Under this factorization assumption, we develop a coordinate ascent algorithm to solve for q∗(u,v)q^*(\mathbf{u},\mathbf{v}) given by Algorithm 1. To allow for modeling of flexible correlation structure, we define a simpler, more general empirical Bayes model with optimization in the following section that can be used to solve step 4 and step 5 of Algorithm 1.

Algorithm 1 Coordinate Ascent Model Estimation with Rank-1 Model

Require: Initialize qu(u)(0)q_u(\mathbf{u})^{(0)}, qv(v)(0)q_v(\mathbf{v})^{(0)}, η(0)\eta^{(0)}, K(0)\mathbf{K}^{(0)}, σ(0)\boldsymbol{\sigma}^{(0)}

  1. repeat

  2. m←m+1m \leftarrow m+1

  3. σ(m)←arg⁡max⁡σ FELBO(qu(m−1),qv(m−1),η(m−1),K(m−1),σ)\boldsymbol{\sigma}^{(m)} \leftarrow\underset{\boldsymbol{\sigma}}{\arg\max}\ F_{ELBO}\left(q_u^{(m-1)},q_v^{(m-1)},\eta^{(m-1)},\mathbf{K}^{(m-1)},\boldsymbol{\sigma}\right)

  4. qu(m),η(m)←arg⁡max⁡qu,η FELBO(qu,qv(m−1),η,K(m−1),σ(m))q_u^{(m)},\eta^{(m)} \leftarrow\underset{q_u,\eta}{\arg\max}\ F_{ELBO}\left(q_u,q_v^{(m-1)},\eta,\mathbf{K}^{(m-1)},\boldsymbol{\sigma}^{(m)}\right)

  5. qv(m),K(m)←arg⁡max⁡qv,K FELBO(qu(m),qv,η(m),K,σ(m))q_v^{(m)},\mathbf{K}^{(m)} \leftarrow\underset{q_v,\mathbf{K}}{\arg\max}\ F_{ELBO}\left(q_u^{(m)},q_v,\eta^{(m)},\mathbf{K},\boldsymbol{\sigma}^{(m)}\right)

  6. until convergence criterion met

  7. Return qu(m),qv(m),η(m),K(m),σ(m)q_u^{(m)},q_v^{(m)},\eta^{(m)},\mathbf{K}^{(m)},\boldsymbol{\sigma}^{(m)}

Optimizing via Empirical Bayes Random Process Means (EBRPM) Model

In this section, we define the Empirical Bayes Random Process Means Model (EBRPM). This model is closely related to the proposed model, in the sense that there is an equivalence with respect to u\mathbf{u} or v\mathbf{v} when the other is fixed. Suppose that we have noisy observations x1,…,xnx_1,\ldots,x_n of true values θ1,…,θn\theta_1,\ldots,\theta_n, each xi∈Rdix_i\in\mathbb{R}^{d_i} can potentially be of different dimension:

xi∣θi∼indNdi(θi,Si)(8)x_i\mid\theta_i \mathrel{\overset{\mathrm{ind}}{\sim}} \mathcal{N}_{d_i}(\theta_i,S_i) \tag*{(8)}

The SiS_i’s are known diagonal matrices reflecting independent noise and θi\theta_i are generated from a random process. Here we focus on modeling correlation within the xix_i’s and restrict our prior family for θi\theta_i to zero-mean Gaussian processes,

θi∼iidGP(0,K(⋅,⋅))(9)\theta_i \mathrel{\overset{\mathrm{iid}}{\sim}} \mathcal{GP}(0, \mathbf{K}(\cdot,\cdot)) \tag*{(9)}

since there are many well studied covariance functions and this distribution generally have good computational properties. We are interested in estimating the θi\theta_i’s and K\mathbf{K} via an empirical Bayes approach, and this is performed in two steps:

  1. Estimate K\mathbf{K} by type II maximum likelihood:

K^=arg⁡max⁡K ∫∏i=1np(xi∣θi)g(θi∣K) dθ=arg⁡max⁡K ∏i=1nNdi(xi∣0,Si+K)(10)\widehat{\mathbf{K}} = \underset{\mathbf{K}}{\arg\max}\ \int\prod_{i=1}^{n} p(x_i \mid\theta_i)g(\theta_i \mid\mathbf{K})\,\mathrm{d}\theta= \underset{\mathbf{K}}{\arg\max}\ \prod_{i=1}^{n} \mathcal{N}_{d_i}(x_i \mid0, \mathbf{S}_i + \mathbf{K}) \tag*{(10)}
  1. Calculate the posteriors for θi\theta_i’s using the estimated prior K^\widehat{\mathbf{K}} from the first step:

p(θi∣x,g^)∝p(xi∣θi)g(θi∣K^)(11)p(\theta_i \mid x, \widehat{g}) \propto p(x_i \mid\theta_i)g(\theta_i \mid\widehat{\mathbf{K}}) \tag*{(11)}

This model is a special case of the empirical Bayes multivariate normal means model (EBMNM) [22] with multivariate Normal priors when di=djd_i=d_j for all i,ji,j realized at the same timepoints, and the empirical Bayes normal means model (EBNM) [3] with normal priors when di=1d_i=1 for all ii. Following the EBNM notation, we summarize the estimation procedure as a function, with an input of (X,S~)(\mathbf{X},\widetilde{\mathbf{S}}) and output (pθ∣x,K^)(p_{\theta\mid x},\widehat{\mathbf{K}}), where X=[x1,…,xn]\mathbf{X}=[x_1,\ldots,x_n], S~=[S1,…,Sn]\widetilde{\mathbf{S}}=[S_1,\ldots,S_n]. In Theorem 2.1 we establish that steps 4 and 5 of Algorithm 1 can be solved by an EBRPM problem.

Theorem 2.1. Step 4 and 5 of Algorithm 1 are equivalent to the following EBRPM problems:

arg⁡max⁡qu,η FELBO(qu,qv,η,K,σ)=EBRPM(xj=∑t,iYtjiv‾ti/σt2∑t,iv‾ti 2/σt2, Sj=1∑t,iv‾ti 2/σt2)(12)\underset{q_u,\eta}{\arg\max}\ F_{ELBO}(q_u,q_v,\eta,\mathbf{K},\boldsymbol{\sigma}) = EBRPM\left(x_j=\frac{\sum_{t,i}Y_{tji}\overline{v}_{ti}/\sigma_t^2}{\sum_{t,i}\overline{v}_{ti}^{\,2}/\sigma_t^2},\ S_j=\frac{1}{\sum_{t,i}\overline{v}_{ti}^{\,2}/\sigma_t^2}\right) \tag*{(12)}

and

arg⁡max⁡qu,K FELBO(qu,qv,η,K,σ)=EBRPM(xi=∑j=1pu‾jyji∑j=1pu‾j2,Si=1∑j=1pu‾j2Σ)(13)\underset{\mathbf{q}_u,\mathbf{K}}{\arg\max}\ F_{ELBO}(\mathbf{q}_u,\mathbf{q}_v,\eta,\mathbf{K},\boldsymbol{\sigma}) = EBRPM\left(x_i = \frac{\sum_{j=1}^{p}\overline{u}_j y_{ji}}{\sum_{j=1}^{p}\overline{u}_j^2}, S_i = \frac{1}{\sum_{j=1}^{p}\overline{u}_j^2}\boldsymbol{\Sigma}\right) \tag*{(13)}

Where yji=[Yt1ji,…,YtDji]T\mathbf{y}_{ji} = [Y_{t_1ji},\ldots,Y_{t_Dji}]^T, Σ=diag⁡(σt12,…,σtD2)\boldsymbol{\Sigma} = \operatorname{diag}(\sigma_{t_1}^2,\ldots,\sigma_{t_D}^2), u‾,v‾\overline{u},\overline{v} represent the first and second moments of u,v\mathbf{u},\mathbf{v} under qu,qvq_u,q_v.

The proof of Theorem 2.1 is provided in Section S2 of the Supplementary Materials, and additional computational details are provided in § 10.3 and § 10.4.

Extension to Rank RR Model

For the rank RR model (R>1R > 1), we simply extend the variational approximation by setting factor distributions for all ur,v~ri\mathbf{u}_r,\widetilde{\mathbf{v}}_{ri} as in (2). FELBOF_{ELBO} is then maximized over (qr,ηr,Kr)r=1R,σ(q_r,\eta_r,\mathbf{K}_r)_{r=1}^{R},\boldsymbol{\sigma} and Algorithm 1 is modified accordingly to iteratively update each factor component while keeping all others constant. The update steps for each ur,v~ri\mathbf{u}_r,\widetilde{\mathbf{v}}_{ri}, r=1,…,Rr=1,\ldots,R follows Theorem 2.1, with Yt\mathbf{Y}_t replaced by Yt−∑r′≠rur′vtr′T\mathbf{Y}_t-\sum_{r'\ne r}\mathbf{u}_{r'}\mathbf{v}_{tr'}^{T}. We summarize the estimation procedure in Algorithm 2.

Algorithm 2 Extension to Rank RR Model

Require: Initialize qur(ur)(0)q_{u_r}(\mathbf{u}_r)^{(0)}, qvr(vr)(0)q_{v_r}(\mathbf{v}_r)^{(0)}, ηr(0)\eta_r^{(0)}, Kr(0)\mathbf{K}_r^{(0)}, σ(0)\boldsymbol{\sigma}^{(0)}, r=1,…,Rr=1,\ldots,R

  1. repeat

  2. m←m+1m \leftarrow m+1

  3. σ(m)←arg⁡max⁡σ F(qu(m−1),qv(m−1),η(m−1),K(m−1),σ)\boldsymbol{\sigma}^{(m)} \leftarrow\underset{\boldsymbol{\sigma}}{\arg\max}\ F(q_u^{(m-1)},q_v^{(m-1)},\eta^{(m-1)},\mathbf{K}^{(m-1)},\boldsymbol{\sigma})

  4. for r=1,…,Rr=1,\ldots,R do

  5. qur(m),ηr(m)←arg⁡max⁡qur,ηr F(qur,qv(m−1),η,K(m−1),σ(m))q_{u_r}^{(m)},\eta_r^{(m)} \leftarrow\underset{q_{u_r},\eta_r}{\arg\max}\ F(q_{u_r},q_v^{(m-1)},\eta,\mathbf{K}^{(m-1)},\boldsymbol{\sigma}^{(m)})

  6. qvr(m),Kr(m)←arg⁡max⁡qvr,Kr F(qu(m),qvr,η(m),K,σ(m))q_{v_r}^{(m)},\mathbf{K}_r^{(m)} \leftarrow\underset{q_{v_r},\mathbf{K}_r}{\arg\max}\ F(q_u^{(m)},q_{v_r},\eta^{(m)},\mathbf{K},\boldsymbol{\sigma}^{(m)})

  7. end for

  8. until convergence criterion met

  9. Return qur,qvr,ηr,Kr,σq_{u_r},q_{v_r},\eta_r,\mathbf{K}_r,\boldsymbol{\sigma}, r=1,…,Rr=1,\ldots,R

When RR is large, this algorithm can become computationally intensive, as all RR factors are updated at each iteration. To alleviate this issue, we implement a greedy algorithm, where rank-1 models are sequentially fitted to the residuals from the sum of previous estimated models, effectively adding one column to the factor matrices at a time, as described in Algorithm 3.

Algorithm 3 Greedy Algorithm for Initialization

Require: Initialize R=1R = 1 and St=0S_t = 0 for t=t1,…,tDt = t_1,\ldots,t_D

  1. repeat

  2. Set Rt=Yt−StR_t = Y_t - S_t for t=t1,…,tDt = t_1,\ldots,t_D

  3. Perform Algorithm 1 with Yt=RtY_t = R_t for t=t1,…,tDt = t_1,\ldots,t_D

  4. Set u^R=Equ[U]\hat{u}_R = E_{q_u}[U], v^tR=Eqv[Vt]\hat{v}_{tR} = E_{q_v}[V_t] for t=t1,…,tDt = t_1,\ldots,t_D

  5. Set U=[u1⋯uR]U = [u_1 \cdots u_R], Vt=[vt1⋯vtR]V_t = [v_{t1} \cdots v_{tR}], St=UVtTS_t = UV_t^T for t=t1,…,tDt = t_1,\ldots,t_D

  6. R=R+1R = R + 1

  7. until convergence criterion met

  8. Return U,Vt,StU,V_t,S_t for t=t1,…,tDt = t_1,\ldots,t_D

In practice, Algorithm 2 often achieves significantly better fit, given that it optimizes FELBO\mathit{FELBO} over a larger space. Combining both by using the greedy algorithm as an initialization for the coordinate ascent algorithm results in significantly less iterations required to achieve convergence, thus reducing computational cost without sacrificing model fit.

Covariance Functions

Several covariance functions for the Gaussian process prior of K\mathbf{K} have been developed for a wide range of scenarios. We implement several common covariance functions, as follows:

White noise kernel (Indep):K(s,t)=σv21(s=t)(14)\textit{White noise kernel (Indep)}:\quad\mathbf{K}(s,t)=\sigma_v^2\mathbb{1}(s=t) \tag*{(14)}
Exchangable covariance (EC):K(s,t)=ρ1(s≠t)σv2(15)\textit{Exchangable covariance (EC)}:\quad\mathbf{K}(s,t)=\rho^{\mathbb{1}(s\ne t)}\sigma_v^2 \tag*{(15)}
Radial Basis Function kernel (RBF):K(s,t)=σv2exp⁡(−∥s−t∥22α2)(16)\textit{Radial Basis Function kernel (RBF)}:\quad\mathbf{K}(s,t)=\sigma_v^2\exp\left(-\frac{\lVert s-t\rVert^2}{2\alpha^2}\right) \tag*{(16)}
Free covariance (Free):K(s,t)=σst2, K∈U+(17)\textit{Free covariance (Free)}:\quad\mathbf{K}(s,t)=\sigma_{st}^2,\ \mathbf{K}\in\mathcal{U}_+ \tag*{(17)}

where U+\mathcal{U}_+ denotes the set of positive definite matrices. We consider the model with independence over time as a benchmark to quantify the performance gain of modeling covariance. This is a special case of the model described in [2] and [4], which has a closed form analytic solution to the empirical variational Bayes approximation. Thus we opt to use their code for faster implementation. Covariance (17) is a free structure with a different covariance for each pair of timepoints. It is impractical at highly irregular timepoints, but is the most flexible structure for the regular timepoint model, and is likely to perform well when the true model structure is unclear. The white noise and exchangeable covariances (14) and (15) have a closed form solution for their parameters in the optimization sub-step (10), while the other covariance structures may be estimated via gradient-based techniques (see § 10.3 of the Supplementary Materials for more details).

The choice of covariance structure may depend on context and nature of the data. For example the RBF covariance (16) accounts for distance between timepoints and is well-motivated when observations are irregularly spaced over time. Otherwise, our results show that the free covariance model is generally a robust choice to misspecification of the true covariance structure (see simulations in § 3 and Section S6.2), and is a flexible option. Alternatively, cross-validation of imputation accuracy may be used to select a covariance structure, as done for our application in § 4. The objective function FELBOF_{ELBO} itself is not recommended as a general criteria to select covariance, as the empirical estimation will always favor the free model because it subsumes the other models.

Model Dimension Selection

Variational Bayes matrix factorization methods have the advantage of automatically selecting the rank concurrently with the estimation framework [3, 17]. This comes from the fact that FELBOF_{ELBO} can be viewed as a balance between model fit and prior resemblence [21]:

FELBO∝∑t=t1tD−12σt2Eq[∥Yt−UVtT∥F2]−KL⁡(q(U,V)∥g(U,V))(18)F_{ELBO} \propto\sum_{t=t_1}^{t_D} -\frac{1}{2\sigma_t^2} E_q[\lVert Y_t - UV_t^T \rVert_F^2] - \operatorname{KL}(q(U,V)\|g(U,V)) \tag*{(18)}

Let U∈Rp×R1U \in\mathbb{R}^{p \times R_1}, Vt∈Rn×R1V_t \in\mathbb{R}^{n \times R_1} and a larger model U′∈Rp×R2U' \in\mathbb{R}^{p \times R_2}, Vt′∈Rn×R2V'_t \in\mathbb{R}^{n \times R_2}. It is clear that when R2>R1R_2 > R_1, KL⁡(qU′,V′∥gU′,V′)>KL⁡(qU,V∥gU,V)\operatorname{KL}(q_{U',V'}\|g_{U',V'}) > \operatorname{KL}(q_{U,V}\|g_{U,V}). So when the decrease in expected squared residuals is less than the penalty incurred by the KL divergence between the added factor distributions and their priors, FELBOF_{ELBO} cannot increase and prefers the lower rank model, equivalent to optimizing q(U,V)q(U,V) towards point masses at zero. Thus, setting RR large initially, the backfit algorithm “prunes” the model by setting redundant factors to zero.

Further, rank sparsity is also achieved through the empirical Bayes step, which can minimize the prior density g(⋅)g(\cdot) to a point mass at zero. As noted in [3] and [17], it is possible that the marginal likelihood is maximized for g(⋅)g(\cdot) at zero, and the following lemma from [3] shows that FELBOF_{ELBO} is guaranteed to be maximized at g(⋅)g(\cdot) if the marginal likelihood is, which can be applied to our model as well.

Lemma. For a rank 1 model, if

arg⁡max⁡gu,gv l(qu,qv,gu,gv,σ)=(δgu,δgv),\underset{g_u,g_v}{\arg\max}\ l(q_u,q_v,g_u,g_v,\boldsymbol{\sigma})=(\delta_{g_u},\delta_{g_v}),

then

arg⁡max⁡gu,gv F(qu,qv,gu,gv,σ)=(δgu,δgv)\underset{g_u,g_v}{\arg\max}\ F(q_u,q_v,g_u,g_v,\boldsymbol{\sigma})=(\delta_{g_u},\delta_{g_v})

In our context, this means that η\eta, K\mathsf{K} are maximized at 00. This justifies the procedure of the greedy algorithm, as columns will be added to the factor matrices until eventually adding a column of zeros which terminates the algorithm.

Identifiability

As in other matrix factorization models, the loading and score components are only identifiable up to a scaling factor, and permutations [3, 23]. Specifically, for any cr≠0c_r \ne0, rescaling ur↦cruru_r \mapsto c_r u_r and v~ri↦v~ri/cr\tilde{v}_{ri} \mapsto\tilde{v}_{ri}/c_r, together with ηr↦∣cr∣ηr\eta_r \mapsto|c_r|\eta_r, Kr↦Kr/cr2\mathsf{K}_r \mapsto\mathsf{K}_r/c_r^2, leaves both FELBOF_{ELBO} and the estimated signal Eq[UVT]E_q[UV^{T}] invariant. This is generally not an issue as signal estimation and imputation are unaffected. A scale convention can be imposed by normalizing each loading estimate to norm 1 and adjust the scale of the corresponding scores accordingly.

Missing Data Imputation

Here we consider incomplete data and the task of imputing missing elements. Denote the indices of missing entries as M={(t,j,i):Ytji not observed}\mathcal{M} = \{(t,j,i):Y_{tji}\text{ not observed}\} and YM={Ytji:(t,j,i)∈M}Y_{\mathcal{M}} = \{Y_{tji}:(t,j,i) \in\mathcal{M}\}, we propose an Expectation-Maximization type approach, summarized in Algorithm 4.

Algorithm 4 Imputation

  1. Set Ytji(0)=Ytji⋅1((t,j,i)∉M)Y_{tji}^{(0)} = Y_{tji} \cdot1((t,j,i) \notin\mathcal{M})

  2. repeat

  3. k←k+1k \leftarrow k+1

  4. Perform the complete data algorithm on Yt(k−1)Y_t^{(k-1)} and denote S^t(k)=Eq[UVtT]\hat{S}_t^{(k)} = E_q[UV_t^{\mathsf{T}}] for t=t1,…,tDt=t_1,\ldots,t_D

  5. Impute Ytji(k)=S^tji(k)Y_{tji}^{(k)} = \hat{S}_{tji}^{(k)}, (t,j,i)∈M(t,j,i) \in\mathcal{M}

  6. until ∑(t,j,i)∈M(S^tji(k)−S^tji(k−1))2<ϵ\sum_{(t,j,i)\in\mathcal{M}}(\hat{S}_{tji}^{(k)}-\hat{S}_{tji}^{(k-1)})^2 < \epsilon for some small ϵ\epsilon

  7. Return Yt1(k),…,YtD(k)Y_{t_1}^{(k)},\ldots,Y_{t_D}^{(k)}

This approach is analogous to previous methodologies including [4] and [24]. Note that step 4 and 5 are respectively the “maximization” and “expectation” steps, as the former maximizes FELBOF_{ELBO} according to Theorem 2.1 and Algorithm 2, and the latter replaces the missing entries with the expected value under current estimates of the factor distributions.

This algorithm is justified by Theorem 2.2, which shows the imputation algorithm optimizes FELBOF_{ELBO} over missing entries YMY_{\mathcal{M}} as well as the latent factor distributions and hyperparameters.

Theorem 2.2. Algorithm 4 iteratively maximizes FELBOF_{ELBO} over q(U,V)q(U,V), σ\boldsymbol{\sigma}, KK, η\boldsymbol{\eta}, YMY_{\mathcal{M}}

The proof of Theorem 2.2 is given in Section S2 of the Supplementary Materials. Compared to the related imputation algorithm in [4], we do not require pre-normalizing the data by estimating σ\sigma first, but rather incorporate it into the optimization algorithm. This also avoids handling σ\boldsymbol{\sigma} differently under entrywise and blockwise missingness.

Simulations

We conduct two simulation studies to assess both signal recovery and imputation accuracy. We use the regular timepoints model in all simulations to allow for comparisons with competing methods that may not accommodate irregular timepoints.

Simulation study 1: Complete data signal recovery

Here we assess our model’s ability to estimate underlying structure for complete data, and illustrate aspects of the different covariance functions. We generate matrices of size 100×120100 \times120 for 5 regularly spaced timepoints with true rank 3 according to our model description, under each of ECEC, RBFRBF, and FreeFree covariance structures with varying signal-to-noise ratios (SNR) and correlation levels. We also use a block covariance structure that none of the covariance functions correctly capture to assess misspecification, where an ECEC correlation is given independently for different subsets of timepoints. See Section S5 of the Supplementary Materials for complete details on the data generation for each scenario. We implement our model using the three non-trivial covariance functions in § 2.4 (EC, RBF, Free), and without correlation using the EVB implementation in [4]. We also consider the general tensor decomposition method TEMPTED that doesn’t explicitly model covariance between timepoints but rather treats time as a continuous variable [8]. Accuracy is measured using the relative squared error:

RSE=∑t,j,i(S^tji−Stji)2∑t,j,iStji2(19)\mathrm{RSE} = \frac{\sum_{t,j,i}(\widehat{S}_{tji} - S_{tji})^2}{\sum_{t,j,i}S_{tji}^2} \tag*{(19)}

where StjiS_{tji} is the low-rank signal: St=UVT\mathbf{S}_t = \mathbf{U}\mathbf{V}^{T}. Table 1 shows the results for all methods. For each covariance structure, the correctly specified model consistently performs the best, as one would reasonably expect. When SNR is high, the model can perform well without modeling correlation, even when the true underlying structure is highly correlated. However, the margin of improvement in estimation accuracy is more significant under low SNR settings, demonstrating that the borrowing of information is effective when the data itself has low information. The TEMPTED method does not perform in a comparable range in this experiment as it does not assume a correlated low rank matrix structure. In the misspecified scenario, the free covariance model performs consistently the best across SNR ratios and correlation level, since it can be considered as correctly specified but overparametrized, which leads to more robust estimation at the cost of lower efficiency. Lastly, we note that in all simulation repetitions, all models selected the true rank.

MethodLow SNR Low CorrLow SNR High CorrMedium SNR Low CorrMedium SNR High CorrHigh SNR Low CorrHigh SNR High Corr
EC0.239 (0.007)0.184 (0.005)0.042 (0.001)0.043 (0.001)0.012 (0.000)0.011 (0.000)
RBF0.213 (0.006)0.139 (0.005)0.040 (0.001)0.029 (0.001)0.011 (0.000)0.008 (0.000)
Free0.218 (0.006)0.149 (0.005)0.040 (0.001)0.030 (0.001)0.011 (0.000)0.008 (0.000)
EVB0.251 (0.007)0.251 (0.008)0.042 (0.001)0.047 (0.002)0.012 (0.000)0.012 (0.000)
TEMPTED0.509 (0.023)0.515 (0.018)0.110 (0.002)0.080 (0.003)0.073 (0.001)0.016 (0.001)
EC0.244 (0.007)0.181 (0.003)0.045 (0.001)0.040 (0.001)0.012 (0.000)0.011 (0.000)
RBF0.248 (0.007)0.231 (0.003)0.045 (0.001)0.042 (0.001)0.012 (0.000)0.011 (0.000)
Free0.247 (0.007)0.189 (0.003)0.045 (0.001)0.040 (0.001)0.012 (0.000)0.011 (0.000)
EVB0.254 (0.007)0.251 (0.005)0.045 (0.001)0.043 (0.001)0.012 (0.000)0.012 (0.000)
TEMPTED0.647 (0.018)0.639 (0.017)0.247 (0.004)0.109 (0.003)0.208 (0.003)0.062 (0.002)
EC0.243 (0.024)0.046 (0.005)0.015 (0.001)
RBF0.242 (0.024)0.046 (0.005)0.015 (0.001)
Free0.203 (0.022)0.040 (0.005)0.013 (0.001)
EVB0.248 (0.026)0.047 (0.006)0.015 (0.001)
TEMPTED0.536 (0.080)0.089 (0.009)0.050 (0.006)
EC0.250 (0.006)0.232 (0.004)0.045 (0.001)0.044 (0.001)0.012 (0.000)0.011 (0.000)
RBF0.250 (0.006)0.225 (0.004)0.045 (0.001)0.043 (0.001)0.012 (0.000)0.011 (0.000)
Free0.252 (0.006)0.203 (0.004)0.045 (0.001)0.042 (0.001)0.012 (0.000)0.011 (0.000)
EVB0.256 (0.006)0.245 (0.005)0.045 (0.001)0.044 (0.001)0.012 (0.000)0.011 (0.000)
TEMPTED0.669 (0.018)0.551 (0.012)0.260 (0.003)0.111 (0.002)0.227 (0.003)0.068 (0.002)

Table 1. RSE for complete data estimation

Simulation study 2: Missing data imputation

Here we evaluate the imputation performance of our model. We consider both entrywise and columnwise missingness, where the latter refers to entire observations missing at a timepoint. Data are generated as in § 3.1, then 10% of the data are randomly set to be missing. In addition to the methods in experiment 1, we compare our model against two imputation algorithms - BAMITA, which is a Bayesian tensor imputation model [9], as well as MissForest, a non-parametric, random forest-based imputation method that doesn’t assume low rank latent structure [25]. We evaluate imputation accuracy by restricting to the RSERSE for missing entries only.

RSEmiss=∑(t,j,i)∈M(S^tji−Stji)2∑(t,j,i)∈MStji2(20)RSE_{miss} = \frac{\sum_{(t,j,i)\in\mathcal{M}}(\hat{S}_{tji}-S_{tji})^2}{\sum_{(t,j,i)\in\mathcal{M}}S_{tji}^2} \tag*{(20)}

We note that TEMPTED cannot handle entrywise missing data so it is excluded from the setting. The results for entrywise missingness are shown in Table 2. The patterns closely mirror the complete data estimation results from Table 1, where the correctly specified covariance model consistently performs the best, especially in low SNR settings. Alternative imputation methods perform markedly worse across all scenarios, with BAMITA performing substantially worse in high SNR settings, due to the strong signal indicating a non-tensor data structure. MissForest exhibits substantially higher errors potentially due to not assuming low rank structure or an explicit correlation structure across time.

RBF Scenario
MethodLow SNR: Low CorrLow SNR: High CorrMedium SNR: Low CorrMedium SNR: High CorrHigh SNR: Low CorrHigh SNR: High Corr
EC0.280 (0.010)0.212 (0.007)0.051 (0.001)0.049 (0.002)0.013 (0.000)0.013 (0.001)
RBF0.249 (0.009)0.161 (0.006)0.048 (0.001)0.033 (0.001)0.013 (0.000)0.009 (0.000)
Free0.255 (0.009)0.187 (0.009)0.049 (0.001)0.034 (0.001)0.013 (0.000)0.009 (0.000)
EVB0.295 (0.010)0.295 (0.008)0.052 (0.001)0.054 (0.002)0.014 (0.000)0.013 (0.001)
BAMITA0.390 (0.012)0.272 (0.009)0.115 (0.002)0.053 (0.002)0.075 (0.001)0.013 (0.001)
MissForest0.802 (0.018)0.810 (0.015)0.211 (0.004)0.223 (0.004)0.089 (0.002)0.089 (0.003)
EC Scenario
MethodLow SNR: Low CorrLow SNR: High CorrMedium SNR: Low CorrMedium SNR: High CorrHigh SNR: Low CorrHigh SNR: High Corr
EC0.287 (0.012)0.199 (0.006)0.051 (0.002)0.048 (0.001)0.014 (0.000)0.013 (0.000)
RBF0.290 (0.013)0.246 (0.006)0.051 (0.002)0.051 (0.002)0.014 (0.000)0.014 (0.000)
Free0.291 (0.012)0.211 (0.007)0.051 (0.002)0.048 (0.002)0.014 (0.000)0.013 (0.000)
EVB0.296 (0.013)0.291 (0.009)0.052 (0.002)0.053 (0.002)0.014 (0.000)0.014 (0.000)
BAMITA0.528 (0.013)0.314 (0.008)0.240 (0.004)0.103 (0.002)0.206 (0.005)0.065 (0.002)
MissForest0.800 (0.019)0.804 (0.020)0.209 (0.006)0.210 (0.006)0.088 (0.003)0.088 (0.003)
Free Scenario
MethodLow SNRMedium SNRHigh SNR
Single ConditionSingle ConditionSingle Condition
EC0.332 (0.027)0.051 (0.004)0.015 (0.001)
RBF0.331 (0.028)0.051 (0.004)0.015 (0.001)
Free0.283 (0.040)0.043 (0.004)0.014 (0.001)
EVB0.339 (0.029)0.051 (0.004)0.015 (0.001)
BAMITA0.401 (0.040)0.082 (0.007)0.051 (0.006)
MissForest0.883 (0.052)0.215 (0.014)0.098 (0.005)
Block Scenario
MethodLow SNR: Low CorrLow SNR: High CorrMedium SNR: Low CorrMedium SNR: High CorrHigh SNR: Low CorrHigh SNR: High Corr
EC0.286 (0.005)0.277 (0.007)0.051 (0.001)0.054 (0.001)0.013 (0.000)0.014 (0.000)
RBF0.287 (0.005)0.266 (0.006)0.051 (0.001)0.053 (0.001)0.013 (0.000)0.014 (0.000)
Free0.288 (0.005)0.245 (0.006)0.051 (0.002)0.050 (0.001)0.013 (0.000)0.013 (0.000)
EVB0.291 (0.005)0.295 (0.008)0.051 (0.001)0.054 (0.001)0.013 (0.000)0.014 (0.000)
BAMITA0.517 (0.007)0.363 (0.009)0.255 (0.005)0.111 (0.002)0.212 (0.004)0.066 (0.001)
MissForest0.795 (0.010)0.800 (0.014)0.209 (0.005)0.222 (0.004)0.086 (0.003)0.090 (0.002)

Table 2. RSE for entrywise missingness imputation

Table 3 summarizes the results for columnwise missingness imputation. The task becomes much more difficult, as imputation completely relies on shared information between timepoints. This is shown by the results of low correlation scenarios under both the ECEC and RBFRBF correlation structure, where RSERSE vary little over SNR levels. Even for the correct model, RSERSE is significantly worse than in estimation or entrywise imputation. All alternative methods other than BAMITA have RSE>1RSE>1, which is the RSERSE of null imputing with 0. It is also worth noting that misspecification of the correlation structure is more costly, as shown by the relatively worse performance of our model in misspecified scenarios compared to estimation and entrywise imputation. Only the most flexible free covariance could still achieve an RSERSE well below 1. On the other hand, it is remarkable that the proposed method performs relatively well even under the lowest SNR under high correlation, recovering about 70% of true structure in both the ECEC and RBFRBF scenarios. In all repetitions, all models selected the true rank.

MethodLow SNR Low CorrLow SNR High CorrMedium SNR Low CorrMedium SNR High CorrHigh SNR Low CorrHigh SNR High Corr
RBF Scenario
EC0.880 (0.071)0.352 (0.037)0.843 (0.057)0.291 (0.052)0.847 (0.022)0.282 (0.038)
RBF0.627 (0.061)0.234 (0.033)0.462 (0.050)0.083 (0.014)0.449 (0.063)0.042 (0.009)
Free0.670 (0.066)0.264 (0.042)0.476 (0.050)0.089 (0.015)0.462 (0.068)0.044 (0.008)
EVB1.000 (0.000)1.000 (0.000)1.000 (0.000)1.000 (0.000)1.000 (0.000)1.000 (0.000)
BAMITA0.874 (0.082)0.350 (0.061)0.652 (0.105)0.124 (0.021)0.900 (0.233)0.059 (0.013)
MissForest11.205 (1.150)10.368 (1.577)3.430 (0.432)3.392 (0.346)2.443 (0.319)2.279 (0.219)
TEMPTED2.005 (0.401)1.978 (0.132)1.643 (0.204)1.776 (0.197)1.508 (0.139)1.908 (0.300)
EC Scenario
EC0.928 (0.030)0.335 (0.027)0.931 (0.036)0.278 (0.033)0.916 (0.018)0.254 (0.027)
RBF0.976 (0.018)0.380 (0.048)0.973 (0.028)0.557 (0.064)0.970 (0.020)0.506 (0.050)
Free0.964 (0.028)0.373 (0.028)0.975 (0.064)0.294 (0.039)0.932 (0.025)0.263 (0.032)
EVB1.000 (0.000)1.000 (0.000)1.000 (0.000)1.000 (0.000)1.000 (0.000)1.000 (0.000)
BAMITA1.325 (0.163)0.451 (0.033)2.723 (0.255)0.516 (0.069)5.835 (1.340)0.812 (0.220)
MissForest9.877 (1.417)10.730 (1.043)3.556 (0.396)3.282 (0.224)2.459 (0.164)2.249 (0.213)
TEMPTED1.723 (0.216)2.172 (0.317)1.519 (0.140)1.811 (0.188)1.500 (0.162)1.856 (0.273)
Free Scenario
EC0.999 (0.014)0.970 (0.042)0.986 (0.032)
RBF0.981 (0.037)0.971 (0.035)0.976 (0.040)
Free0.662 (0.196)0.364 (0.126)0.318 (0.133)
EVB1.000 (0.000)1.000 (0.000)1.000 (0.000)
BAMITA0.807 (0.222)0.477 (0.147)0.673 (0.415)
MissForest11.321 (2.702)3.653 (0.504)2.353 (0.165)
TEMPTED2.041 (0.358)1.793 (0.183)1.784 (0.234)
Block Scenario
EC0.979 (0.029)0.866 (0.071)0.981 (0.018)0.843 (0.054)0.991 (0.020)0.868 (0.028)
RBF0.983 (0.020)0.697 (0.063)0.972 (0.023)0.651 (0.058)0.988 (0.015)0.690 (0.044)
Free0.999 (0.045)0.549 (0.041)0.990 (0.047)0.405 (0.040)0.969 (0.030)0.418 (0.056)
EVB1.000 (0.000)1.000 (0.000)1.000 (0.000)1.000 (0.000)1.000 (0.000)1.000 (0.000)
BAMITA1.416 (0.123)0.700 (0.053)3.010 (0.381)0.633 (0.071)5.545 (1.837)0.985 (0.125)
MissForest10.182 (0.997)11.127 (1.703)3.275 (0.332)3.389 (0.325)2.384 (0.167)2.209 (0.134)
TEMPTED1.801 (0.163)2.093 (0.332)1.616 (0.155)1.686 (0.241)1.564 (0.112)26.49 (77.14)

Table 3. RSE of blockwise imputation for all methods under varying simulated conditions, with the best performing method highlighted in bold font.

Gene Time Course Data Application

We apply our method to a set of gene time course data described in [5]. This study explored methods to classify genetic response of multiple sclerosis patients to recombinant human interferon beta (rIFNβ\beta) treatment. rIFNβ\beta was given to 53 patients and expression levels of 76 genes were measured at 7 timepoints (baseline and follow-ups at 3, 6, 9, 12, 18, 24 months after treatment). Most patients had some percentage of missingness, ranging from 1 – 6 entire timepoints missing and additional partial missingness at observed timepoints. Overall, 10.5% of all data were missing. We first use this dataset to evaluate the empirical imputation performance of our model. Similar to the experiments in § 3.2, we apply our model with the three non-trivial covariance functions, EVB under independence, BAMITA, and MissForest to the data. TEMPTED is omitted as it cannot handle entrywise missingness. We take the complete indices and perform 10, 3, 2-fold cross validation based on the missing proportion, repeated 10 times over different random seeds. Figure 1 shows the results.

Analysis results for the IFN$\beta$ data application

Figure 1. Analysis results for the IFNβ\beta data application: Imputation accuracy for all methods from cross validation.

In the comparisons, the EC covariance model performed the best across both entrywise and columnwise missingness. The independent model performed comparably in the entrywise missing scenarios, but had RSE=1RSE = 1 in columnwise missingness, consistent with simulation results. All alternative methods performed worse than the independent model, highlighting the advantage of our proposed model in columnwise imputation.

As the ECEC covariance model performed the best, we refer to the results produced by this model for interpretation. We estimated a total of 14 components; the top 7 ordered by variance explained are shown in Figure 2a, along with the estimated correlation in each component’s scores. The first component explains about 35% variance, while the next 6 each explains less than 10% variance each. Both the third and seventh component exhibits high correlation between sample scores at different timepoints. We plot individual score trajectories in Figure 2b, colored by manually labeled good/poor responders from the original study. The first component, which is the major contributor, exhibits similar patterns for most individuals, indicating a mean effect of subjects. The seventh component, on the other hand, shows a clear clustering effect in good and poor responders, with good responders at the bottom and poor responders at the top. The individual scores have little variation between time points, which indicates that the two types of responders can be distinguished without inferring from temporal patterns. This is consistent with the results in [5], where no group*time interaction effects were discovered; as well as [26], where distinction between the two groups were not driven by factors that vary over time. More generally, the EC covariance and accompanying score trajectories suggest some clear subject effects but no other temporal trends.

Figure 2. (a) Variance explained and estimated correlation (ρ^\hat{\rho}) for the first 7 components. (b) Individual trajectories for the first and seventh component scores. Figure 2: Analysis results for the IFNβ\beta data application.

To further investigate the biological relevance of Component 7, we performed a gene-set enrichment analysis of its loadings using GO Biological Processes via WebGestalt [27]. While no processes were significant at the FDR <0.05< 0.05 level, the top enriched processes and their nominal significance is shown in Figure 3. The most significant and top downregulated process, implying higher levels for good responders, was response to transforming growth factor beta; this is consistent with prior literature associating higher TGF-β\beta levels with better treatment response [28].

Gene set enrichment analysis for loadings of Component 7 in the IFN$\beta$ factorization

Figure 3. Gene set enrichment analysis for loadings of Component 7 in the IFNβ\beta factorization

Discussion

In this paper, we introduce a latent matrix factorization method for high dimensional longitudinal data that captures temporal correlation by flexibly modeling the covariance structure in latent factor scores. The additive nature of our model enables decomposition of data into static and temporal factors, enhancing interpretability. The model employs a computationally efficient empirical variational Bayes estimation algorithm that automatically estimates hyperparameters and selects model dimension by optimizing an objective function balancing model fit and complexity. Through extensive simulation studies and a real world data application, the proposed method demonstrates superior estimation and imputation accuracy compared to existing approaches across diverse scenarios.

While we focus on single-source data, multimodal datasets combining data from multiple sites are increasingly common in biomedical studies, such as collaborative neurodegenerative disease research that collects imaging, clinical, and genetic modalities [29, 30, 31]. Extensions of our method to multimodal matrix factorization—where some components are restricted to specific cohorts or data modalities—is an interesting direction for future research, as the full rank RR and greedy estimation algorithms do not straightforwardly extend to such settings.

A known limitation of variational Bayes methods is uncertainty quantification. While point estimates generally perform well and have good properties, variational Bayes methods typically underestimate marginal posterior variances [32, 33]. To address this limitation, our method can serve as initialization for the associated fully Bayesian model, offering several advantages including initialization near the posterior mode and automatic rank selection, which are challenging for the fully Bayesian approach to achieve alone. A related Bayesian model is described in [34], which illustrates the benefits of accounting for uncertainty in imputation and prediction. However, standard MCMC strategies such as Gibbs sampling would likely face substantial computational challenges, necessitating the development of more efficient sampling algorithms. As our model imputes values given the fit to observed data, it implicitly assumes data are missing at random; extensions to scenarios where the missing mechanism is not random but known (i.e., a limit of detection) is another direction of future work.

Code & Data Availability

Documented R code to perform the proposed method, including an R package, and the IFNβ\beta dataset analyzed in § 4 is available at https://github.com/andyljk/Longitudinal_Factorization. Additionally, a supplementary zipped folder includes code to perform all the simulations and data analysis in the manuscript.

Supplementary Materials

A supplementary document available online includes proofs, details for the method with irregular timepoints, several additional simulation studies, additional computational details, and further details for the real data application.

References

Supplementary Materials for “Empirical Variational Bayes Factorization for Longitudinal Matrix Data”

Jiakun Li; Eric F. Lock

Supplementary Materials

Irregular Timepoint Model

In this section, we provide a description of the key differences of the extended model that handles inconsistency in measured timepoints across individuals. For the i=1,…,ni=1,\ldots,n individual, assume each with pp variables measured at time t1(i),…,tdi(i)t_1^{(i)},\ldots,t_{d_i}^{(i)}. The outcome for individual ii, variable jj at time tt is modeled as

Ytji=∑r=1Rurjvtri+Etji(21)Y_{tji}=\sum_{r=1}^{R}u_{rj}v_{tri}+E_{tji} \tag*{(21)}

Note that the matrix product representation in (1) is not available in this setting, since vv does not have consistent measures at each timepoint. Priors are prescribed to each of u,vu,v similar to (3), where

urj∼iidN(0,ηr2),v~ri∼iidmvN(0,Kr(i)),etji∼iidN(0,σtji2)(22)u_{rj}\mathrel{\overset{\mathrm{iid}}{\sim}}N(0,\eta_r^2),\quad\tilde{v}_{ri}\mathrel{\overset{\mathrm{iid}}{\sim}}mvN(0,K_r^{(i)}),\quad e_{tji}\mathrel{\overset{\mathrm{iid}}{\sim}}N(0,\sigma_{tji}^2) \tag*{(22)}

and v~ri=[vt1(i)ri ⋯ vtdi(i)ri]∈Rdi\tilde{v}_{ri}=[v_{t_1^{(i)}ri}\ \cdots\ v_{t_{d_i}^{(i)}ri}]\in\mathbb{R}^{d_i}. The prior covariance matrix Kr(i)∈Rdi×diK_r^{(i)}\in\mathbb{R}^{d_i\times d_i} has dimension matching the number of measurements from individual ii. We assume all prior covariance are parametrized by a common set of parameters as described in § 2.4, and denote estimates of KK as K^\widehat{K} for convenience. When the measurements are highly irregular, modeling the noise variance separately at each timepoints becomes difficult. In this situation, a reasonable modification is to set σt2=σ2\sigma_t^2=\sigma^2 for all tt.

Denoting U={ujr:j=1,…,p;r=1,…,R}U=\{u_{jr}:j=1,\ldots,p;r=1,\ldots,R\} and V={vtir:t∈T;i=1,…,n;r=1,…,R}V=\{v_{tir}:t\in T;i=1,\ldots,n;r=1,\ldots,R\}, we maximize the same objective function FELBO(q,η,K,σ)F_{ELBO}(q,\eta,K,\sigma) using algorithm 2. For the rank 1 model, it can be shown following the proof of Theorem 2.1, step 4, 5 of algorithm 1 are equivalent to solving the following

arg⁡max⁡qu,η FELBO(qu,qv,η,K,σ)=EBRPM⁡(xj=∑i=1n∑t=t1(i)tdi(i)Ytjivˉti/σt2∑i=1n∑t=t1(i)tdi(i)vˉti2/σt2, Sj=1∑i=1n∑t=t1(i)tdi(i)vˉti2/σt2)(23)\underset{q_u,\eta}{\arg\max}\ F_{ELBO}(q_u,q_v,\eta,\mathbf{K},\boldsymbol{\sigma})=\operatorname{EBRPM}\left(x_j=\frac{\sum_{i=1}^{n}\sum_{t=t^{(i)}_1}^{t^{(i)}_{d_i}}Y_{tji}\bar{v}_{ti}/\sigma_t^2}{\sum_{i=1}^{n}\sum_{t=t^{(i)}_1}^{t^{(i)}_{d_i}}\bar{v}_{ti}^{2}/\sigma_t^2},\ S_j=\frac{1}{\sum_{i=1}^{n}\sum_{t=t^{(i)}_1}^{t^{(i)}_{d_i}}\bar{v}_{ti}^{2}/\sigma_t^2}\right) \tag*{(23)}

and

arg⁡max⁡qv,K FELBO(qu,qv,η,K,σ)=EBRPM⁡(xi=∑j=1puˉjyji∑j=1puˉj2, Si=1∑j=1puˉj2Σi)(24)\underset{q_v,\mathbf{K}}{\arg\max}\ F_{ELBO}(q_u,q_v,\eta,\mathbf{K},\boldsymbol{\sigma})=\operatorname{EBRPM}\left(x_i=\frac{\sum_{j=1}^{p}\bar{u}_j y_{ji}}{\sum_{j=1}^{p}\bar{u}_j^2},\ S_i=\frac{1}{\sum_{j=1}^{p}\bar{u}_j^2}\boldsymbol{\Sigma}_i\right) \tag*{(24)}

Note that each yji\mathbf{y}_{ji} is of potentially different dimension, which fits in the framework of the EBRPM model by design. For estimation of the multiple rank model, steps 5, 6 of algorithm 2 and step 3 of algorithm 3 are modified accordingly to accommodate the timepoint structure.

Proofs

Theorem 2.1

We prove Theorem 2.1, where we stated the equivalence between step 4, 5 of algorithm 1 and the EBRPM problem. The proof is similar to Proposition 5 in [3], which can be viewed as a univariate case of our model. We first introduce the following lemma as an intermediate step.

Lemma S1. For i=1,…,ni=1,\ldots,n, let xi∼iidmvNdi(θi,Si)\mathbf{x}_i \overset{\mathrm{iid}}{\sim} mvN_{d_i}(\boldsymbol{\theta}_i,\mathbf{S}_i), θi∼iidGP(0,K)\boldsymbol{\theta}_i \overset{\mathrm{iid}}{\sim} \mathcal{GP}(0,\mathbf{K}), and S={Si:i=1,…,n}\mathbf{S}=\{\mathbf{S}_i:i=1,\ldots,n\} be known. Recall the EBRPM first estimates the prior variance function as

K^=arg⁡max⁡K log⁡p(x∣K),\widehat{\mathbf{K}}=\underset{\mathbf{K}}{\arg\max}\ \log p(\mathbf{x}\mid\mathbf{K}),

then uses this estimate to calculate the posterior p(θ∣x,K^)p(\boldsymbol{\theta}\mid\mathbf{x},\widehat{\mathbf{K}}). The solution to the EBRPM problem (p(θ∣x,K^),K^)(p(\boldsymbol{\theta}\mid\mathbf{x},\widehat{\mathbf{K}}),\widehat{\mathbf{K}}) is equivalent to the solution of the following objective:

arg⁡max⁡q,K F0(K,q(θ))(25)\underset{q,\mathbf{K}}{\arg\max}\ F_0(\mathbf{K},q(\boldsymbol{\theta})) \tag*{(25)}

where

F0(K,q(θ))=Eq[log⁡p(x,θ∣K,S)q(θ)]F_0(\mathbf{K},q(\theta)) = E_q\left[\log\frac{p(\mathbf{x},\theta\mid K,S)}{q(\theta)}\right]
∝Eq[−1/2∑i=1nθiTAiθi−2BiTθi]+Eq[log⁡g(θ∣K)q(θ)](26)\propto E_q\left[-1/2\sum_{i=1}^{n}\theta_i^T A_i\theta_i-2B_i^T\theta_i\right]+E_q\left[\log\frac{g(\theta\mid K)}{q(\theta)}\right] \tag*{(26)}

where Ai=Si−1A_i=S_i^{-1}, Bi=Si−1xiB_i=S_i^{-1}\mathbf{x}_i

Proof. To demonstrate equivalence, we need to show that q^(θ)=p(θ∣x,K^)\hat{q}(\theta)=p(\theta\mid\mathbf{x},\hat{K}) and arg⁡max⁡KF0=K^EB\underset{K}{\arg\max}F_0=\hat{K}_{EB}. Note that

log⁡p(x∣K)=∫q(θ)log⁡p(x,θ∣K,S)q(θ)dθ+∫q(θ)log⁡q(θ)p(θ∣x,K)dθ=F0(K,q(θ))+KL(qθ∥pθ∣x)(27)\begin{aligned} \log p(\mathbf{x}\mid K)=\int q(\theta)\log\frac{p(\mathbf{x},\theta\mid K,S)}{q(\theta)}d\theta+\int q(\theta)\log\frac{q(\theta)}{p(\theta\mid\mathbf{x},K)}d\theta\\ &=F_0(K,q(\theta))+KL(q_\theta\Vert p_{\theta\mid\mathbf{x}}) \tag*{(27)} \end{aligned}

Since KL(qθ∥pθ∣x)≥0KL(q_\theta\Vert p_{\theta\mid\mathbf{x}})\ge0 with equality iff qθ=pθ∣xq_\theta=p_{\theta\mid\mathbf{x}}, we see that

max⁡q F0(q(θ),K)=log⁡p(x∣K)(28)\underset{q}{\max}\ F_0(q(\theta),K)=\log p(\mathbf{x}\mid K) \tag*{(28)}

when q(θ)=p(θ∣x,K)q(\theta)=p(\theta\mid\mathbf{x},K), so then

arg⁡max⁡K{max⁡q F0(q(θ),K)}=arg⁡max⁡Klog⁡p(x∣K)=K^EB(29)\underset{K}{\arg\max}\left\{\underset{q}{\max}\ F_0(q(\theta),K)\right\}=\underset{K}{\arg\max}\log p(\mathbf{x}\mid K)=\hat{K}_{EB} \tag*{(29)}

Lastly we show that F0F_0 has the expanded form, which can be done by straightforwardly factorizing the joint distribution:

F0(K,q(θ))=Eq[log⁡p(x,θ∣K,S)q(θ)]=Eq[log⁡p(x∣θ,K,S)]+Eq[log⁡g(θ∣K)q(θ)]=Eq[∑i=1nlog⁡Ndi(xi∣θi,Si)]+Eq[log⁡g(θ∣K)q(θ)]∝Eq[−1/2∑i=1nθiTAiθi−2BiTθi]+Eq[log⁡g(θ∣K)q(θ)](30)\begin{aligned} F_0(K,q(\theta))&=E_q\left[\log\frac{p(\mathbf{x},\theta\mid K,S)}{q(\theta)}\right]\\ &=E_q[\log p(\mathbf{x}\mid\theta,K,S)]+E_q\left[\log\frac{g(\theta\mid K)}{q(\theta)}\right]\\ &=E_q\left[\sum_{i=1}^{n}\log N_{d_i}(\mathbf{x}_i\mid\theta_i,S_i)\right]+E_q\left[\log\frac{g(\theta\mid K)}{q(\theta)}\right]\\ &\propto E_q\left[-1/2\sum_{i=1}^{n}\theta_i^T A_i\theta_i-2B_i^T\theta_i\right]+E_q\left[\log\frac{g(\theta\mid K)}{q(\theta)}\right] \tag*{(30)} \end{aligned}

We now prove Theorem 2.1.

Proof. In step 4 of Algorithm 1, we treat all quantities other than qu,ηq_u,\eta as fixed. The objective function is then

F(q,η,K,σ):=Eq[log⁡p(Y,u,v;η,K,σ)q(u,v)]F(q,\eta,K,\sigma) := E_q\left[\log\frac{p(Y,u,v;\eta,K,\sigma)}{q(u,v)}\right]
∝Equ[∑t=t1tDlog⁡p(Yt∣u,vt,η,K)]+Equ[log⁡g(u∣η)q(u)](31)\propto E_{q_u}\left[\sum_{t=t_1}^{t_D}\log p(Y_t\mid u,v_t,\eta,K)\right] + E_{q_u}\left[\log\frac{g(u\mid\eta)}{q(u)}\right] \tag*{(31)}
∝Equ[−1/2∑j=1pujTAjuj−2BjTuj]+Equ[log⁡g(u∣η)q(u)]\propto E_{q_u}\left[-1/2\sum_{j=1}^{p}u_j^T A_j u_j-2B_j^T u_j\right] + E_{q_u}\left[\log\frac{g(u\mid\eta)}{q(u)}\right]

Where Aj=∑t=t1tD∑i=1nv‾ti2/σt2A_j=\sum_{t=t_1}^{t_D}\sum_{i=1}^{n}\overline{v}_{ti}^{2}/\sigma_t^2, Bj=∑t=t1tD∑i=1nYtjiv‾ti/σt2B_j=\sum_{t=t_1}^{t_D}\sum_{i=1}^{n}Y_{tji}\overline{v}_{ti}/\sigma_t^2

Then equation (12) follows from Lemma S1. Similarly, with respect to qv,Kq_v,K for step 5, the objective function becomes

F(q,η,K,σ)∝Eqv[−1/2∑i=1nviTAivi−2BiTvi]+Eqv[log⁡g(v∣K)q(v)](32)F(q,\eta,K,\sigma)\propto E_{q_v}\left[-1/2\sum_{i=1}^{n}v_i^T A_i v_i-2B_i^T v_i\right]+E_{q_v}\left[\log\frac{g(v\mid K)}{q(v)}\right] \tag*{(32)}

Where Ai=(∑j=1pu‾j2)Σ−1A_i=\left(\sum_{j=1}^{p}\overline{u}_j^2\right)\Sigma^{-1}, Bi=Σ−1(∑j=1pu‾jyji)B_i=\Sigma^{-1}\left(\sum_{j=1}^{p}\overline{u}_j y_{ji}\right), Σ=diag⁡(σt12,…,σtD2)\Sigma=\operatorname{diag}(\sigma_{t_1}^2,\ldots,\sigma_{t_D}^2)

and equation (13) follows from Lemma S1.

Theorem 2.2

Proof. From algorithm 2, it can be seen that the M steps of algorithm 4 each partially maximize the ELBO over σ,q(U),q(V)\boldsymbol{\sigma},q(U),q(V), holding all other quantities fixed. So it leaves to show that the E step maximizes the ELBO over the missing entries YMY_{\mathcal{M}}. The ELBO can be decomposed into

FELBO=Eql[∑t=t1tDlog⁡p(Yt∣U,Vt,η,K)]+Equ[log⁡g(U∣η)q(U)]+Eqv[log⁡g(V∣K)q(V)](33)F_{ELBO}=E_{q_l}\left[\sum_{t=t_1}^{t_D}\log p(Y_t\mid U,V_t,\eta,\mathbb{K})\right]+E_{q_u}\left[\log\frac{g(U\mid\eta)}{q(U)}\right]+E_{q_v}\left[\log\frac{g(V\mid\mathbb{K})}{q(V)}\right] \tag*{(33)}

holding all other quantities fixed, FELBOF_{ELBO} with respect to YMY_{\mathcal{M}} is proportional to

FELBO(YM)∝Eq[∑(t,j,i)∈Mlog⁡p(Ytji∣uj,vti,η,K)]=∑(t,j,i)∈M−12σt2(Ytji−∑r=1REq[UjrVtir])2+b≤b(34)\begin{aligned} F_{ELBO}(Y_{\mathcal{M}})\propto E_q\left[\sum_{(t,j,i)\in\mathcal{M}}\log p(Y_{tji}\mid u_j,v_{ti},\eta,\mathbb{K})\right] \\ &=\sum_{(t,j,i)\in\mathcal{M}}-\frac{1}{2\sigma_t^2}\left(Y_{tji}-\sum_{r=1}^{R}E_q[U_{jr}V_{tir}]\right)^2+b\leq b \tag*{(34)} \end{aligned}

where bb is a constant

Thus it can be seen that FELBOF_{ELBO} is maximized at Ytji=∑r=1REq[UjrVtir]=S^tjiY_{tji}=\sum_{r=1}^{R}E_q[U_{jr}V_{tir}]=\hat{S}_{tji}, for all (t,j,i)∈M(t,j,i)\in\mathcal{M}. □\square

Solving the EBRPM problem

Here, we provide the details for implementing the two step solution to solve the EBRPM problem described in § 2.2.2. From the model described in equation (8), 9, the first step is to estimate KK by maximizing the marginal likelihood. We restrict to the case where KK is parametrized by some parameters α\boldsymbol{\alpha}, and estimate α\alpha instead, which is the case that all covariance functions discussed fall into. Below, we calculate the analytic form of the gradients and several well-documented gradient-based optimization methods can be used. For convenience, let Ki=Si+KK_i=S_i+K, the marginal log likelihood is given by:

l(α)=−12∑i=1n{dilog⁡2π+∑i=1nlog⁡∣Ki∣+xiTKi−1xi}(35)l(\boldsymbol{\alpha})=-\frac{1}{2}\sum_{i=1}^{n}\left\{d_i\log2\pi+\sum_{i=1}^{n}\log\lvert K_i\rvert+x_i^{T}K_i^{-1}x_i\right\} \tag*{(35)}

It can be shown that the matrix derivative of inverse matrices and log determinant of positive definite matrices are given by

∂∂αKi−1=−Ki−1∂Ki∂αKi−1(36)\frac{\partial}{\partial\alpha}K_i^{-1}=-K_i^{-1}\frac{\partial K_i}{\partial\alpha}K_i^{-1} \tag*{(36)}
∂∂αlog⁡∣Ki∣=tr⁡(Ki−1)∂Ki∂α(37)\frac{\partial}{\partial\alpha}\log\lvert K_i\rvert=\operatorname{tr}\left(K_i^{-1}\right)\frac{\partial K_i}{\partial\alpha} \tag*{(37)}

Where ∂Ki∂α\frac{\partial K_i}{\partial\alpha} denotes the elementwise derivative matrix. Thus the gradient is given by

∂l∂αj=12∑i=1nxiTKi−1∂Ki∂αjKi−1xi−tr⁡(Ki−1∂Ki∂αj)=12∑i=1ntr⁡((γiγiT−Ki−1)∂Ki∂αj)(38)\begin{aligned} \frac{\partial l}{\partial\alpha_j} &=\frac{1}{2}\sum_{i=1}^{n}x_i^{T}K_i^{-1}\frac{\partial K_i}{\partial\alpha_j}K_i^{-1}x_i-\operatorname{tr}\left(K_i^{-1}\frac{\partial K_i}{\partial\alpha_j}\right)\\ &=\frac{1}{2}\sum_{i=1}^{n}\operatorname{tr}\left(\left(\gamma_i\gamma_i^{T}-K_i^{-1}\right)\frac{\partial K_i}{\partial\alpha_j}\right) \tag*{(38)} \end{aligned}

Where γi=Ki−1xi\gamma_i=K_i^{-1}x_i. Once α^\hat{\alpha} (and K^\hat{K}) is obtained, the posterior distribution for θi\theta_i is given by

θi∣xi,K^∼Ndi((Si−1+K^−1)−1Si−1xi,(Si−1+K^−1)−1)(39)\theta_i\mid x_i,\hat{K}\sim\mathcal{N}_{d_i}\left(\left(S_i^{-1}+\hat{K}^{-1}\right)^{-1}S_i^{-1}x_i,\left(S_i^{-1}+\hat{K}^{-1}\right)^{-1}\right) \tag*{(39)}

ELBO Calculation

In the CAVI algorithm described in section 2.2.2, each update step to the parameters does not explicitly compute the objective function itself. Being able to calculate the objective function FELBOF_{ELBO} could be beneficial for monitoring the convergence of the optimization algorithm, especially since FELBOF_{ELBO} can be highly nonconcave. Here, we provide the procedure for computing FELBOF_{ELBO} using the same decomposition as in equation (S13). The first term ∑t=t1tDE[log⁡p(Yt∣U,Vt,η,K)]\sum_{t=t_1}^{t_D}E[\log p(Y_t\mid U,V_t,\eta,K)] is straightforward to compute, given the factor distributions q(U,V)q(U,V). The other two terms can be identified as ∑r=1R−KL(q(Ur)∥g(Ur))\sum_{r=1}^{R}-KL(q(U_r)\|g(U_r)) and ∑r=1R−KL(q(Vr)∥g(Vr))\sum_{r=1}^{R}-KL(q(V_r)\|g(V_r)), each rr’th term from either sums can be found as follows from the EBRPM subproblem:

log⁡p(x∣K^)=F0(K^,q^(θ))=Eq[log⁡p(x,θ∣K^,S)q^(θ)]=Eq[log⁡p(x∣θ)]+Eq[log⁡g^(θ)q^(θ)]Eq[log⁡g^(θ)q^(θ)]=l(g^)+12∑i=1n{dilog⁡(2π)+log⁡∣Si∣+(xiTSi−1xi−2xiTSi−1Eq[θi]+tr⁡(Si−1Var⁡q(θi))+Eq[θi]TSi−1Eq[θi])}(40)\begin{aligned} \log p(x \mid\widehat{K}) &= F_0(\widehat{K},\widehat{q}(\theta)) = E_q\left[\log\frac{p(x,\theta\mid\widehat{K},S)}{\widehat{q}(\theta)}\right] \\ &= E_q[\log p(x \mid\theta)] + E_q\left[\log\frac{\widehat{g}(\theta)}{\widehat{q}(\theta)}\right] \\ E_q\left[\log\frac{\widehat{g}(\theta)}{\widehat{q}(\theta)}\right] &= l(\widehat{g}) + \frac{1}{2}\sum_{i=1}^{n}\left\{d_i\log(2\pi)+\log|S_i| \right.\\ &\qquad\left.{}+\left(x_i^{T}S_i^{-1}x_i-2x_i^{T}S_i^{-1}E_q[\theta_i]+\operatorname{tr}(S_i^{-1}\operatorname{Var}_q(\theta_i))+E_q[\theta_i]^{T}S_i^{-1}E_q[\theta_i]\right)\right\} \tag*{(40)} \end{aligned}

This calculation is straightforward to implement, as the only additional quantity required is the optimized log likelihood value from the EBRPM problem, which is usually not difficult to obtain.

Simulations: Data Generation

In the simulation section, we set the sample size n=120n=120, variable dimension p=100p=100, and number of timepoints to be D=5D=5 complete data are generated according to the prior described in (3): For r=1,…,Rr=1,\ldots,R, draw the factor matrices ur∼N(0,η2I)\mathbf{u}_r \sim N(0,\eta^2I), v~ri∼N(0,K)\widetilde{\mathbf{v}}_{ri} \sim N(0,\mathbf{K}), compute the underlying structure St=UVtTS_t=U\mathbf{V}_t^{T} and Xt=St+EtX_t=S_t+E_t where entries of EtE_t are standard normal draws. For the covariance structures (15), (16), we set σv=0.2,0.5,1\sigma_v=0.2,0.5,1 for the three SNR we consider and fix η\eta at 1. This gives the low, medium, high SNR settings the SNR of 0.12, 0.75, 3 respectively, as the total SNR is the sum of each single rank component’s variance. In low and high correlation levels, we set ρ=0.2,0.8\rho=0.2,0.8 for the ECEC structure and α=1,3\alpha=1,3 for the RBFRBF structure respectively. The free covariance structure (17) is generated as ATA/dA^{T}A/d, where AA has standard normal entries. The block covariance is defined as a block diagonal covariance matrix diag⁡(Σ1,Σ2)\operatorname{diag}(\Sigma_1,\Sigma_2) where Σ1,Σ2\Sigma_1,\Sigma_2 are 2-by-2 and 3-by-3 ECEC covariances respectively.

Additional Simulations

Irregular timepoints

Settings

In this experiment we investigate our model’s performance under irregular timepoints. The data dimensions are set to n=120n = 120, p=100p = 100, R=3R = 3, consistent with § 3. The loadings and scores are generated as described in section S5. For subject ii, each score vector is drawn from the rbf covariance kernel. Parameters of the kernel are set to scale η∈{1,0.5,0.2}\eta\in\{1, 0.5, 0.2\} and length-scale α∈{3,1}\alpha\in\{3, 1\} for all of the RR components. The noise variance is set to σ=1\sigma= 1. Each setting is repeated 100 times.

We used two designs for the visit times (Figure S1).

Figure S1. Visit times for 120 subjects in one draw from each design.

  • Jittered. Each subject has five planned fixed timepoints at times 1,…,51, \ldots, 5. Each visit is then offset by a small amount of noise N(0,0.12)N(0, 0.1^2). We then randomly remove 7% of the visits. After each visit, the subject drops out with probability 0.03.

  • Completely irregular. The number of timepoints is generated from a negative binomial distribution with size 2.8 and mean 4 then added 1, upper bounded at 15 timepoints. The time of the first visit is randomly generated using a LogNormal(0.4,0.83)(0.4, 0.83) distribution, followed by the time between visits, randomly drawn from a Gamma(2,2)(2, 2) distribution.

For each dataset, 10% of all visits are held out, one visit for each selected subject. Each subject is guaranteed to have at least one timepoint, so we only randomly sample from subjects with two or more timepoints. We fitted two types of model:

  • Irregular time model. The irregular model uses the true visit times of each subject. We used the EC and RBF priors.

  • Regular time model with binned times. Observed timepoints are rounded to the

nearest whole time unit. If multiple visits are in one time bin, we take the average.

We then fitted the aligned model with the EC, Free and RBF priors.

Figure S1 provides a visualization of the generated timepoints from a single dataset. To assess performance, we report the relative squared error ∥S^−S∥F2/∥S∥F2\lVert\hat{S} - S \rVert_{F}^{2}/\lVert S \rVert_{F}^{2} at the observed visits and at the held-out visits. We also report the selected rank and the fit time.

Results

Table S1 and Table S2 provides a summary of the RSE performance on observed visits and heldout visits. In the jittered time design, both the irregular time model and the regular time model fitted with binned timepoints perform comparably across all SNR and correlation strength settings. However, in the irregular time setting, fitting the regular time model fails severely, especially when SNR is low. The irregular time models selected the true rank in all simulation replicates across all settings. On the other hand, while the binned model selected the true rank in all jittered time settings, Table S3 shows that it was unable to accurately estimate the rank in all completely irregular time settings with η=0.2\eta= 0.2. Table S3 gives the settings where the binned models failed. These results demonstrate that correctly accounting for the irregular timepoints is essential and that fitting the regular time model with approximated timepoints results in poor model fit and incorrect model structure.

Designη\etaα\alphaEC irregularRBF irregularEC binnedFree binnedRBF binned
Jittered130.0120.0090.0120.0100.009
Jittered110.0120.0120.0120.0120.012
Jittered0.530.0440.0320.0450.0340.033
Jittered0.510.0470.0450.0470.0480.046
Jittered0.230.2000.1590.2120.1660.161
Jittered0.210.2510.2320.2550.2480.238
Irregular130.0120.0080.0240.0140.016
Irregular110.0120.0110.0390.0320.040
Irregular0.530.0440.0280.1430.0460.053
Irregular0.510.0460.0400.1100.0760.105
Irregular0.230.2120.1470.8380.2760.675
Irregular0.210.2450.2030.9950.4600.925

Table S1. Mean RSE at the observed visits.

Designη\etaα\alphaEC irregularRBF irregularEC binnedFree binnedRBF binned
Jittered130.2760.0400.2800.0440.043
Jittered110.8440.4320.8420.4650.440
Jittered0.530.2970.0820.2970.0880.084
Jittered0.510.8410.4740.8400.5300.486
Jittered0.230.3650.2380.3650.2520.242
Jittered0.210.8610.6160.8610.6690.623
Irregular130.3380.0650.2550.1360.083
Irregular110.8120.3590.6040.5650.457
Irregular0.530.3320.0950.2740.1580.132
Irregular0.510.7840.3800.6300.6250.488
Irregular0.230.4100.2380.8560.3710.702
Irregular0.210.8260.5240.9980.7860.959

Table S2. RSE at the held-out visits.

α\alphaEC binnedFree binnedRBF binned
30 (0–1)76 (2–3)0 (0–2)
10 (0–1)39 (1–3)0 (0–1)

Table S3. Number of replicates (out of 100) with the correct rank for the binned models, completely irregular design, η=0.2\eta= 0.2. The range of the selected rank is included in parentheses.

Misspecified Settings

In this section, we investigate the robustness of our model to several ways in which data generation can misalign with the model. Unless stated otherwise, we use n=120n = 120, p=100p = 100, 5 timepoints, η∈{1,0.5,0.2}\eta\in\{1, 0.5, 0.2\}, σ=1\sigma= 1, 10% entrywise or columnwise missingness and 100 replicates per setting, as in the main simulations.

Tensor data structure

Let ∘\circ denote the outer product. Data are generated from a rank-3 CP tensor model,

X=∑r=13ur∘vr∘wr+E,(41)\mathcal{X} = \sum_{r=1}^{3} \mathbf{u}_r \circ\mathbf{v}_r \circ\mathbf{w}_r + \mathcal{E}, \tag*{(41)}

where ur∈Rn\mathbf{u}_r \in\mathbb{R}^{n} and vr∈Rp\mathbf{v}_r \in\mathbb{R}^{p} have N(0,1)N(0,1) entries, wr∈R5\mathbf{w}_r \in\mathbb{R}^{5} has N(0,η2)N(0,\eta^2) entries to control the SNR, and the random errors terms E\mathcal{E} are iid distributed N(0,1)N(0,1). Figure S2 shows the results under completely observed data, 10% random entrywise missingness, and 10% random columnwise missingness. In all three settings, we observe that while the EC and RBF models perform worse than the correctly specified BAMITA model, the Free covariance model performs comparably across all SNR and observed/missing data settings. Indeed, the tensor structure is very similar to the Free covariance model, as each frontal slice of the tensor can be viewed as a matrix observation at a single timepoint. Under this angle, the flexible Free covariance structure model is able to capture covariance that is not structured smoothly over time, demonstrating robustness to such misspecifications.

Boxplots of RSE for complete data, entrywise missingness, and columnwise missingness

Figure S2. RSE for data generated from the CP tensor model.

Mixture of temporal & static factors

Data are generated from our model with rank R=4R = 4: two static factors, whose scores are the same at every timepoint for each subject, and two temporal factors, whose scores follow the RBF covariance with α=1\alpha= 1. Each factor has score variance 3η2/43\eta^2/4, so the total signal matches the rank-3 settings. From Figure S3, we observe that our model performs similar to the correctly specified setting, and captures the true signal of the model.

Boxplots of RSE for complete data, entrywise missingness, and columnwise missingness

Figure S3. RSE for data with two static and two temporal factors.

Non-Gaussian loadings

Data are generated as in the main simulations from § 3, with the loading-generating distribution replaced by the centered Gamma(1, 1) distribution. Each setting is compared with the same setting under loadings generated from a standard normal distribution. Figure S4 shows that the models under non-Gaussian loadings perform similarly to the correctly specified Gaussian loading setting, and thus misspecification of the loading distribution does not result in poor model fit of the EVBLM model.

RSE with centered gamma loadings (faded) and Gaussian loadings (solid)

Figure S4. RSE with centered gamma loadings (faded) and Gaussian loadings (solid). Rows give the true covariance of the scores.

Scalability

In this experiment, we investigate the scalability of our model. We benchmark the runtime to fit our model by varying each of the number of subjects, variables, timepoints, and missingness. Data are generated from our model with the rbf covariance with parameters η=0.5\eta= 0.5, α=3\alpha= 3, and σ2=1\sigma^{2} = 1. We establish a reference setting of p=1000p = 1000 variables, n=200n = 200 subjects, T=5T = 5 timepoints and rank 3. We scale one of the dimensions each at a time: from 100 to 20000 variables; 50 to 5000 subjects; 3 to 50 timepoints; rank 1 to 20; and the total entries at a timepoint (p×np \times n) from 250×50250 \times50 to (10000×2000)(10000 \times2000), which gives 6.25×1046.25 \times10^{4} to 10810^{8} entries; and 10%, 30% and 50% of the entries or of the visits missing. Each setting is repeated 100 times. The model is fitted with the ec, free and rbf covariances, with automatic rank selection. All timings were obtained on a single node with two 64-core AMD EPYC 7763 processors and 512 GB of RAM, using R 4.3.0 with single-threaded OpenBLAS, with eight fits running concurrently.

Figure S5 shows the median runtime per model fit. The log computing time is linear in log growth of each of pp, nn, and DD. The consistent behavior is expected as these parameters all scale the total number of matrix entries linearly. This is also observed in the first panel on the second row of Figure S5, where we scale the parameters by the product of pp, nn, DD. All three covariance models perform comparably when scaling pp and nn. The free covariance model takes longer to fit with more timepoints, as there are D(D+1)/2D(D + 1)/2 parameters to estimate, whereas the parameters of EC and rbf are constant with respect to timepoints. In the missingness scenarios, the computing time depends on the mode of missingness. Entrywise missingness scales the log computing time linearly similar to the previous settings, because more missing entries require more iterations of the imputation step before the ELBO converges. On the other hand, computing time stays relatively stable in the blockwise missingness scenario, and even faster when 50% of the visits are missing.

Figure S5. Median computing time per fit (100 replicates) as each dimension increases, with the other dimensions fixed at 1000 variables, 200 subjects, 5 timepoints and rank 3. Both axes are on a log scale.

IFNβ\beta Data Application

Preprocessing

As described in the original study [5], expression of pre-selected genes was measured in PBMC RNA by one-step kinetic RT-PCR (ABI 7900), with triplicate reactions on 384-well plates that included positive and negative controls and a calibration curve. Absolute expression values were obtained by interpolation from a calibration curve spanning five orders of magnitude (built from an in vitro transcribed GAPDH clone), and the triplicate means were divided by the value of a positive-control RNA (thymus) on the same plate to correct for plate-to-plate variability. Housekeeping-gene normalization was deliberately not used; instead, equal RNA input was ensured through fluorescence-based RNA quantitation, dilution of all samples to a common concentration, and automated liquid handling. The original authors did not apply a formal batch correction beyond this plate-level normalization. We used the publicly released, normalized expression values (Dataset S1), which were not log-scaled, applied a natural log transformation, and then centered each gene to have mean zero across its observed values. No further processing was performed.

Number of components

The greedy algorithm adds one component at a time and stops when a new component does not increase the ELBO; backfitting then updates all components jointly. With the EC covariance this selects 13 components on the full data. Table S4 shows the number selected across bootstrap resamples (§ 10.7.4).

Number of components121314151617
Count10191521253241
Matched position23456789101113
Count33646104173407162401694

Table S4. Bootstrap results for the IFNβ\beta data (B=1000B = 1000): number of components selected, and position of the component matched to full-data component 7.

Responder Difference Significance Test

Let SijS_{ij} be the posterior mean score of component 7 for subject ii at visit jj, with tj∈{0,0.25,0.5,0.75,1,1.5,2}t_j \in\{0, 0.25, 0.5, 0.75, 1, 1.5, 2\} the visit time in years from baseline, and RiR_i the indicator that subject ii is a good responder. We fit the linear mixed model

Sij=β0+β1Ri+β2tj+β3Ritj+ei+εij,ei∼N(0,τ2),εij∼N(0,σ2),(42)S_{ij} = \beta_0 + \beta_1 R_i + \beta_2 t_j + \beta_3 R_i t_j + e_i + \varepsilon_{ij}, \qquad e_i \sim N(0,\tau^2), \qquad\varepsilon_{ij} \sim N(0,\sigma^2), \tag*{(42)}

where eie_i is a random subject intercept. Here β1\beta_1 is the difference between good and poor responders at baseline, and β3\beta_3 is the change in that difference per year; if β3=0\beta_3 = 0, the baseline difference holds at every visit. The model uses all 53×753 \times7 estimated scores, including those at missing visits, and is fit by REML with the nlme package.

The estimates are β^1=0.364\hat{\beta}_1 = 0.364 (SE 0.054, p=1.8×10−8p = 1.8 \times10^{-8}) and β^3=0.0006\hat{\beta}_3 = 0.0006 per year (SE 0.0041, p=0.89p = 0.89). Good responders have higher component 7 scores, and we find no evidence that the difference changes over time. These pp-values treat the estimated scores as observed and do not account for component 7 having been chosen after inspecting the score trajectories; a Bonferroni correction over the 13 components gives p=2.3×10−7p = 2.3 \times10^{-7} for β1\beta_1.

Stability across resamples

The EVBLM initialization is deterministic, so refitting the full data returns the same components; we therefore assess stability over resamples of subjects. We drew B=1000B = 1000 bootstrap samples of the 53 subjects with replacement and refit EVBLM with the same settings (EC covariance, greedy selection followed by backfitting). Because components with similar variance explained can change order between fits, the component matched to full-data component 7 is the one whose loading vector has the largest absolute correlation with the full-data loading of component 7. Its scores are multiplied by the sign of that correlation, and model (S22) is refit to them.

The median absolute loading correlation of the matched component is 0.84 (interquartile range 0.72 to 0.90), and it exceeds 0.8 in 61% of resamples. The matched component is at position 5 to 8 in 85% of resamples. Across resamples, β^1\hat{\beta}_{1} has median 0.352 (2.5% and 97.5% quantiles 0.122 and 0.473) and p<0.05p < 0.05 in 96.8% of resamples; β^3\hat{\beta}_{3} has median 0.000 (quantiles −0.015-0.015 and 0.019) and p<0.05p < 0.05 in 10.6%.

References

  1. [1]Charles W Fox and Stephen J Roberts. A tutorial on variational Bayesian inference. Artificial intelligence review, 38(2):85–95, 2012.
  2. [2]S. Nakajima, M. Sugiyama, S. D. Babacan, and R. T omioka. Global analytic solution of fully-observed variational Bayesian matrix factorization. Journal of Machine Learning Research, 14(1):1–37, 2013.
  3. [3]W. Wang and M. Stephens. Empirical Bayes matrix factorization. Journal of Machine Learning Research, 22(120):1–40, 2021.
  4. [4]E. F. Lock. Empirical Bayes linked matrix decomposition. Machine Learning, 113:7451–7477, 2024.DOI
  5. [5]S. E. Baranzini, P. Mousavi, J. Rio, S. J. Caillier, A. Stillman, P. Villoslada, M. M. Wyatt, M. Comabella, L. D. Greller, R. Somogyi, X. Montalban, and J. O. Oksenberg. Transcription-based prediction of response to IFNβ using supervised computational methods. PloS Biology, 3(1):e2, 2004.pubmed.ncbi.nlm.nih.gov/15630474
  6. [6]Tamara G Kolda and Brett W Bader. Tensor decompositions and applications. SIAM review, 51(3):455–500, 2009.DOI
  7. [7]Leying Guan. Smooth and probabilistic PARAFAC model with auxiliary covariates. Journal of Computational and Graphical Statistics, 33(2):538–550, 2024.arxiv.org/abs/2104.05184
  8. [8]P. Shi, C. Martino, R. Han, and et al. TEMPTED: time-informed dimensionality reduction for longitudinal microbiome studies. Genome Biology, 25(317):891–921, 2024. doi: https://doi.org/10.1186/s13059-024-03453-x.
  9. [9]Z. Jiang, G. Li, and E. F. Lock. BAMITA: Bayesian multiple imputation for tensor arrays. Biostatistics, 26(1), 2025.arxiv.org/abs/2410.23412
  10. [10]Ajit P Singh and Geoffrey J Gordon. Relational learning via collective matrix factorization. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 650–658, 2008.
  11. [11]Mehmet Gonen and Samuel Kaski. Kernelized Bayesian matrix factorization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(10):2047–2060, 2014.DOI
  12. [12]Z. Wang, F. Zhang, C. Zheng, X. Hu, M. Cai, and C. Yang. MFAI : A scalable Bayesian matrix factorization approach to leveraging auxiliary information. Jour- nal of Computational and Graphical Statistics, 33(4):1339–1349, 2024. doi: https: //doi.org/10.1080/10618600.2024.2319160.DOI
  13. [13]X. An, Q. Yang, and P.M. Bentler. A latent factor linear mixed model for high-dimensional longitudinal data analysis. Statistics in Medicine, 32(24):4229–4239, 2013.DOI
  14. [14]Z. Lu and N.K. Chandra. A sparse factor model for clustering high-dimensional longitudinal data. Statistics in Medicine, 43(19):3633–3648, 2024.DOI
  15. [15]Qinxia Wang and Yuanjia Wang. Multilayer exponential family factor models for integrative analysis and learning disease progression. Biostatistics, 25(1):203–219, 2024.
  16. [16]C. M. Bishop. Bayesian PCA. In Advances in Neural Information Processing Systems 11, pages 382–388. MIT Press, 1999.
  17. [17]C. M. Bishop. Variational principal components. In Ninth International Conference on Artificial Neural Networks, 1(470):509–514, 1999.DOI
  18. [18]Q. Zhao, L. Zhang, and A. Cichocki. Bayesian CP factorization of incomplete tensors with automatic rank determination. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(9):1751–1763, 2015. doi: 10.1109/TPAMI.2015.2392756.DOI
  19. [19]S. Nakajima, R. Tomioka, M. Sugiyama, and S. D. Babacan. Condition for perfect dimensionality recovery by variational Bayesian PCA. Journal of Machine Learning Research, 16(114):3757–3811, 2015.
  20. [20]R. M. Neal. Probabilistic inference using Markov chain Monte Carlo methods. Technical Reports, Department of Computer Science:University of Toronto, 1993.
  21. [21]D. M. Blei, A. Kucukelbir, and J. D. McAuliffe. Variational inference: a review for statisticians. Journal of the American Statistical Association, 112(518):859–877, 2017.
  22. [22]Y. Yang, P. Carbonetto, D. Gerard, and M. Stephens. Improved Methods for Empirical Bayes Multivariate Multiple Testing and Effect Size Estimation. arXiv preprint arXiv: 2406.08784, 2024.arxiv.org/abs/2406.08784
  23. [23]Patrik O. Hoyer. Non-negative matrix factorization with sparseness constraints. Journal of Machine Learning Research, 5:1457–1469, 2004. URL https://www.jmlr.org/papers/v5/hoyer04a.html.DOI
  24. [24]R. Mazumder, T. Hastie, and R. Tibshirani. Spectral regularization algorithms for learning large incomplete matrices. Journal of Machine Learning Research, 11(80):2287–2322, 2010.pubmed.ncbi.nlm.nih.gov/21552465
  25. [25]D. J. Stekhoven and P Bühlmann. Missforest: non-parametric missing value imputation for mixed-type data. Bioinformatics, 28(1):112–128, 2012.
  26. [26]B. Guo, L. E. Eberly, P. G. Henry, C. Lenglet, and E. F. Lock. Multiway sparse distance weighted discrimination. Journal of Computational and Graphical Statistics, 32(2):730–743, 2022. doi: https://doi.org/10.1080/10618600.2022.2099404.
  27. [27]John M Elizarraras, Yuxing Liao, Zhiaο Shi, Qian Zhu, Alexander R Pico, and Bing Zhang. Webgestalt 2024: faster gene set analysis and new support for metabolomics and multi-omics. Nucleic acids research, 52(W1):W415–W421, 2024.DOI
  28. [28]Angele Cucci, Pierangelo Barbero, Marinella Clerico, Bruno Ferrero, Elisabetta Versino, Giulia Contessa, Stefania Demercanti, Emanuela Viglietta, Alessandra Di Liberto, Alessandra Giai Vai, et al. Pro-inflammatory cytokine and chemokine mrna blood level in multiple sclerosis is related to treatment response and interferon-beta dose. Journal of Neuroimmunology, 226(1-2):150–157, 2010.
  29. [29]Nellie Georgiou-Karistianis, Louise A. Corben, Eric F. Lock, Helena Bujalka, Isaac Adanyeguh, Manuela Corti, Dinesh K. Deelchand, Martin B. Delatycki, Imis Dogan, Jennifer Farmer, Marcondes C. França Jr., Anthony S. Gabay, William Gaetz, Ian H. Harding, James Joers, Michelle A. Lax, Jiakun Li, David R. Lynch, Thomas H. Mareci, Alberto R. M. Martinez, Massimo Pandolfo, Marina Papoutsi, Richard G. Parker, Kathrin Reetz, Thiago J.R. Rezende, Timothy P. Roberts, Sandro Romanzetti, David A. Rudko, Susmita Saha, Jörg B. Schulz, Sub H. Subramony, Veena G. Supramaniam, Christophe Lenglet, and Pierre-Gilles Henry. Neuroimaging biomarkers for friedreich ataxia: A cross-sectional analysis of the track-fa study. Annals of Neurology, 98(2): 386–397, 2025.DOI
  30. [30]H.S.H. Oh, D.Y. Urey, and L. et al. Karlsson. A cerebrospinal fluid synaptic protein biomarker for prediction of cognitive resilience versus decline in alzheimer’s disease. Nature Medicine, 31:1592–1603, 2025.
  31. [31]Sarah J Tabrizi, Rachael I Scahill, Alexandra Durr, Raymund AC Roos, Blair R Leavitt, Rebecca Jones, G Bernhard Landwehrmeyer, Nick C Fox, Hans Johnson, Stephen L Hicks, Christopher Kennard, David Craufurd, Chris Frost, Douglas R Langbehn, Ralf Reilmann, and Julie C Stout. Biological and clinical changes in premanifest and early stage huntington’s disease in the track-hd study: the 12-month longitudinal analysis. The Lancet Neurology, 10(1):31–42, 2011.DOI
  32. [32]B Wang and M Titterington. Inadequacy of interval estimates corresponding to variational bayesian approximations. Workshop on Artificial Intelligence and Statistics, pages 373–380, 2005.
  33. [33]Richard Turner, M. Sahani, D. Barber, Ali Cemgil, and S. Chiappa. Two problems with variational expectation maximisation for time-series models. Bayesian Time Series Models, pages 109–130, 2011.DOI
  34. [34]Sarah Samorodnitsky, Chris H. Wendt, and Eric F. Lock. Bayesian simultaneous factor- ization and prediction using multi-omic data. Computational Statistics & Data Analysis, 197, 2024.

Paper details

vlnc:W_b8b6yrqj
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents