betaFeedback

WorksMathematics

Unifying Data-Adaptive Longitudinal Treatment-Effect Subgroup Discovery and Pretreatment Membership Prediction

Abstract

Precision medicine seeks to match patients with treatments from which they are most likely to benefit. A key step toward this goal is to learn treatment-effect heterogeneity from longitudinal study data. This requires discovering subgroups with distinct treatment-effect trajectories and predicting subgroup membership for new patients from pretreatment information. Existing methods do not fully unify these tasks. Covariate-informed membership models typically prespecify the subgroup number or covariance structure, whereas data-adaptive discovery methods often require prediction in a separate stage. We propose a data-adaptive semiparametric mixture-of-experts framework that unifies longitudinal treatment-effect subgroup discovery and pretreatment membership prediction. Subgroup-specific varying-coefficient functions characterize treatment-effect trajectories, while flexible covariance functions account for within-subject dependence. A covariate-dependent gating model links membership to pretreatment characteristics. A penalty on global component weights removes redundant subgroups, while separate gating coefficients allow patient-specific membership probabilities among retained subgroups. A blockwise generalized expectation–maximization algorithm estimates all model components under a single penalized observed-data objective. We establish consistent subgroup-number selection, convergence rates for the functional estimators, and asymptotic normality for the finite-dimensional parameters. Simulations show reliable subgroup-number selection and more accurate conditional membership prediction than a two-step procedure. Applications to the SANAD epilepsy study and the COMPETE migraine trial identify distinct temporal treatment-effect patterns.

Introduction

A central goal of precision medicine is to use pretreatment information to guide treatment selection for new patients. Conventional analyses often summarize treatment benefit through a population-average effect, which may obscure important differences across patients [11, 15]. Subgroup analysis of longitudinal outcomes can characterize this heterogeneity by identifying groups with distinct treatment-effect trajectories. To translate outcome-defined subgroups to new patients, membership must be predictable from information available before treatment. This calls for a unified model that learns the subgroup number and treatment-effect trajectories from study outcomes while predicting membership from pretreatment covariates.

Longitudinal follow-up provides a richer basis for studying treatment-effect heterogeneity than a single endpoint. Repeated measurements reveal when treatment responses emerge, how they evolve, and whether they persist, and may also improve the precision of treatment comparisons ([7, 22] (Collins 2006)). These advantages have motivated a substantial literature on statistical methods for longitudinal treatment-response analysis. At the same time, longitudinal data introduce within-subject dependence and irregular observation times, while treatment-response trajectories may be nonlinear. In heterogeneous populations, both treatment-effect trajectories and within-subject covariance structures may vary across latent subgroups, requiring flexible subgroup-specific modeling of both features.

Existing methods fall into two broad, partly overlapping strands. Covariate-defined approaches construct treatment-sensitive subgroups from pretreatment characteristics. Threshold and change-plane models define subgroup boundaries through one or more covariates, with extensions for dropout, time-varying treatment effects, and floor or ceiling effects ([5, 26, 27, 6]). Tree-based methods recursively partition the covariate space and accommodate longitudinal or multivariate outcomes ([16, 28, 31]). Outcome-driven approaches instead infer latent heterogeneity from repeated responses. Random-effects mixtures and structured mixed-effects models identify subgroups with distinct treatment-response trajectories; some also model membership probabilities using pretreatment covariates ([30, 25, 29]).

More recent work has pursued data-adaptive subgroup discovery and membership prediction, although often separately. Zhang et al. [32] proposed a semiparametric longitudinal treatment-effect model that selects the subgroup number and estimates subgroup-specific covariance functions. Related methods include sparse mixture regression for multivariate functional responses ([33]) and fusion-based clustering of longitudinal profiles ([34]). For nonlongitudinal outcomes, Shen and He [24] jointly identify a treatment-sensitive subgroup and construct predictive scores for membership under a structured mixture model. Chen et al. [3] considered a scalar outcome with a multivariate grouping vector, allowing coefficient-specific subgroup structures; their procedure first detects groups by regularization and then uses a support vector machine to recover covariate-space partitions for predicting future membership.

Taken together, these methods do not jointly address data-adaptive subgroup discovery and pretreatment membership prediction. Methods that incorporate pretreatment covariates often prespecify the subgroup number or within-subject covariance structure. The threshold and change-plane methods cited above, as well as Shen and Qu [25], assume two subgroups. Wen et al. [29] select the number of classes by comparing prespecified candidate models and retain a parametric model for within-subject dependence. In contrast, the data-adaptive longitudinal methods discussed above do not estimate a pretreatment membership model jointly with subgroup discovery. Zhang et al. [32], for example, fit a membership model only after discovering the subgroups; in a nonlongitudinal setting, Chen et al. [3] similarly recover covariate-space partitions with a support vector machine after detecting coefficient groups. Because pretreatment covariates do not contribute to the discovery stage, these procedures treat estimated membership labels or responses as fixed during prediction, so discovery errors can propagate to the prediction stage.

We address this gap with a data-adaptive semiparametric mixture-of-experts model that jointly discovers longitudinal treatment-effect subgroups and predicts membership from pretreatment covariates. Within each subgroup, varying-coefficient functions characterize the treatment-effect trajectory, while a flexible covariance function captures within-subject dependence. A covariate-dependent gating model relates subgroup membership to pretreatment characteristics. Beginning with a prespecified upper bound on the number of subgroups, we penalize global component weights to eliminate redundant groups, while separate gating coefficients allow membership probabilities among the retained groups to vary across patients. A blockwise generalized expectation–maximization algorithm jointly estimates all model components under a single penalized observed-data objective.

For study participants, posterior membership probabilities combine information from pretreatment covariates and longitudinal outcomes; for future patients, the fitted gating model produces membership probabilities from pretreatment covariates alone.

The framework differs from existing methods in three respects. First, because the gating model enters the observed-data likelihood, pretreatment covariates contribute directly to subgroup discovery rather than being used only to predict subgroup labels estimated in a separate stage. Second, separating global component weights from covariate-dependent gating coefficients enables subgroup-number selection and patient-specific membership modeling within the same objective. A redundant component can therefore be eliminated globally without forcing membership probabilities among the retained groups to be constant across patients. Third, the longitudinal model allows both treatment-effect trajectories and covariance functions to vary across subgroups without prespecifying a parametric correlation structure. We establish selection consistency for the subgroup number, convergence rates for the functional estimators, and asymptotic normality of the finite-dimensional estimators.

The remainder of the paper is organized as follows. Section 2 develops the proposed model and estimation procedure, including tuning-parameter selection and prospective membership prediction. Section 3 establishes subgroup-selection consistency and the asymptotic properties of the estimators. Section 4 evaluates finite-sample performance through simulations. Section 5 illustrates the method using data from the COMPETE migraine trial and the SANAD epilepsy study. The final section discusses the implications, limitations, and directions for future research.

Model and estimation

Consider nn independent subjects, with subject ii observed at times tij∈[0,1]t_{ij} \in[0,1], j=1,…,nij=1,\ldots,n_i. Let Yi(tij)Y_i(t_{ij}) and Xij∈RdX_{ij} \in\mathbb{R}^d denote the corresponding longitudinal response and covariate vector. Define Yi=(Yi(ti1),…,Yi(tini))T\mathbf{Y}_i=(Y_i(t_{i1}),\ldots,Y_i(t_{in_i}))^{\mathrm{T}}, Xi=(Xi1,…,Xini)T\mathbf{X}_i=(X_{i1},\ldots,X_{in_i})^{\mathrm{T}}, ti=(ti1,…,tini)T\mathbf{t}_i=(t_{i1},\ldots,t_{in_i})^{\mathrm{T}}. Let Di∈{0,1}D_i \in\{0,1\} denote treatment assignment, with Di=1D_i=1 indicating the treatment of interest and Di=0D_i=0 the comparator. Let Zi∈RqZ_i \in\mathbb{R}^q contain pretreatment covariates used to predict subgroup membership for new subjects. The vectors XijX_{ij} and ZiZ_i may share baseline covariates. The observed data for subject ii are Oi=(Yi,Xi,ti,Di,Zi)O_i=(\mathbf{Y}_i,\mathbf{X}_i,\mathbf{t}_i,D_i,Z_i).

Let Gi∈{1,…,G0}G_i \in\{1,\ldots,G_0\} denote the latent subgroup membership of subject ii, where G0G_0 is the true but unknown number of subgroups. Conditional on Gi=gG_i=g, we assume the subgroup-specific varying-coefficient model

Yi(tij)=XijT{βg1(tij)+Diβg2(tij)}+eig(tij)+ϵigj,(1)Y_i(t_{ij})=\mathbf{X}_{ij}^{\mathrm{T}}\{\boldsymbol{\beta}_{g1}(t_{ij})+D_i\boldsymbol{\beta}_{g2}(t_{ij})\}+e_{ig}(t_{ij})+\epsilon_{igj}, \tag*{(1)}

where βg1(t)=(βg1,1(t),…,βg1,d(t))T\boldsymbol{\beta}_{g1}(t)=(\beta_{g1,1}(t),\ldots,\beta_{g1,d}(t))^{\mathrm{T}} represents the time-varying covariate effects, and βg2(t)=(βg2,1(t),…,βg2,d(t))T\boldsymbol{\beta}_{g2}(t)=(\beta_{g2,1}(t),\ldots,\beta_{g2,d}(t))^{\mathrm{T}} represents the time-varying treatment effects in subgroup gg. For a fixed covariate profile x\boldsymbol{x}, the corresponding treatment contrast is τg(t∣x)=xTβg2(t)\tau_g(t\mid\boldsymbol{x})=\boldsymbol{x}^{\mathrm{T}}\boldsymbol{\beta}_{g2}(t).

Within-subject dependence in subgroup gg is represented by a mean-zero Gaussian process eig(t)e_{ig}(t) with covariance function Σg(s,t)=Cov⁡{eig(s),eig(t)}\Sigma_g(s,t)=\operatorname{Cov}\{e_{ig}(s),e_{ig}(t)\}. Its Karhunen–Loève expansion is eig(t)=∑k=1∞Vigkϕgk(t)e_{ig}(t)=\sum_{k=1}^{\infty}V_{igk}\phi_{gk}(t) [1, 10], where ϕgk(t)\phi_{gk}(t) is the kkth eigenfunction of Σg\Sigma_g. The eigenfunctions are orthonormal, satisfying ∫01ϕgk(t)ϕgℓ(t) dt=I(k=ℓ)\int_0^1\phi_{gk}(t)\phi_{g\ell}(t)\,dt=I(k=\ell), and the scores are independent across kk, with Vigk∼N(0,λgk)V_{igk}\sim N(0,\lambda_{gk}). We order the eigenvalues so that λg1≥λg2≥⋯>0\lambda_{g1}\geq\lambda_{g2}\geq\cdots>0 and assume ∑k=1∞λgk<∞\sum_{k=1}^{\infty}\lambda_{gk}<\infty. In practice, we retain the first KK terms and define Vig=(Vig1,…,VigK)T\mathbf{V}_{ig}=(V_{ig1},\ldots,V_{igK})^{\mathrm{T}} and ϕg(t)=(ϕg1(t),…,ϕgK(t))T\boldsymbol{\phi}_g(t)=(\phi_{g1}(t),\ldots,\phi_{gK}(t))^{\mathrm{T}}. Let Λg=diag⁡(λg1,…,λgK)\boldsymbol{\Lambda}_g=\operatorname{diag}(\lambda_{g1},\ldots,\lambda_{gK}); then eig(t)≈VigTϕg(t)e_{ig}(t)\approx\mathbf{V}_{ig}^{\mathrm{T}}\boldsymbol{\phi}_g(t). The measurement errors satisfy ϵigj∼indN(0,σg2)\epsilon_{igj}\mathrel{\mathop{\sim}^{\mathrm{ind}}}N(0,\sigma_g^2) and are independent of eige_{ig}. Let Φg(ti)∈Rni×K\boldsymbol{\Phi}_g(t_i) \in\mathbb{R}^{n_i \times K} denote the matrix with jjth row ϕg(tij)T\boldsymbol{\phi}_g(t_{ij})^{\mathsf{T}}. Under the rank-KK approximation, the covariance matrix of Yi\mathbf{Y}_i conditional on Gi=gG_i = g is Σig=Φg(ti)ΛgΦg(ti)T+σg2Ini\boldsymbol{\Sigma}_{ig} = \boldsymbol{\Phi}_g(t_i)\boldsymbol{\Lambda}_g\boldsymbol{\Phi}_g(t_i)^{\mathsf{T}} + \sigma_g^2\mathbf{I}_{n_i}.

Because G0G_0 is unknown, we embed the true model in an over-specified working model with Gmax⁡≥G0G_{\max} \ge G_0 candidate subgroups. Under this working model, Gi∈{1,…,Gmax⁡}G_i \in\{1,\ldots,G_{\max}\}, and we relate subgroup membership to pretreatment covariates through

πig:=πg(Zi)=P(Gi=g∣Zi)=ωgexp⁡(ZiTηg)∑h=1Gmax⁡ωhexp⁡(ZiTηh),g=1,…,Gmax⁡,(2)\pi_{ig} := \pi_g(\mathbf{Z}_i) = \mathbb{P}(G_i = g \mid\mathbf{Z}_i) = \frac{\omega_g \exp(\mathbf{Z}_i^{\mathsf{T}}\boldsymbol{\eta}_g)}{\sum_{h=1}^{G_{\max}}\omega_h \exp(\mathbf{Z}_i^{\mathsf{T}}\boldsymbol{\eta}_h)}, \qquad g = 1,\ldots,G_{\max}, \tag*{(2)}

where ωg≥0\omega_g \ge0, ∑g=1Gmax⁡ωg=1\sum_{g=1}^{G_{\max}}\omega_g = 1, and ηg∈Rq\boldsymbol{\eta}_g \in\mathbb{R}^q. The global weights determine which components remain active, whereas the gating coefficients allow membership probabilities among active components to vary with pretreatment covariates. In particular, ωg=0\omega_g = 0 removes subgroup gg for every covariate profile, whereas variation in πig\pi_{ig} represents subject-specific membership rather than global subgroup removal. Let A0={g:ωg0>0}\mathcal{A}_0 = \{g : \omega_{g0} > 0\} denote the true active set, so that ∣A0∣=G0|\mathcal{A}_0| = G_0. For identifiability, we exclude an intercept from Zi\mathbf{Z}_i and set η1=0\boldsymbol{\eta}_1 = \mathbf{0}. The global weights satisfy πg(0)=ωg\pi_g(\mathbf{0}) = \omega_g. If ηg=0\boldsymbol{\eta}_g = \mathbf{0} for every gg, the gating model reduces to constant mixing proportions, with πig=ωg\pi_{ig} = \omega_g for all ii; in general, however, ωg\omega_g need not equal the marginal prevalence of subgroup gg.

Penalized estimation

Let μig=(μig1,…,μigni)T\boldsymbol{\mu}_{ig} = (\mu_{ig1},\ldots,\mu_{ign_i})^{\mathsf{T}} denote the subgroup-specific mean vector, where μigj=XijT{βg1(tij)+Diβg2(tij)}\mu_{igj} = \mathbf{X}_{ij}^{\mathsf{T}}\{\boldsymbol{\beta}_{g1}(t_{ij}) + D_i\boldsymbol{\beta}_{g2}(t_{ij})\}. Under the rank-KK covariance approximation, the subgroup-conditional density is fig:=fg(Yi∣Xi,ti,Di)=ϕni(Yi;μig,Σig)f_{ig} := f_g(\mathbf{Y}_i \mid\mathbf{X}_i,t_i,D_i) = \phi_{n_i}(\mathbf{Y}_i;\boldsymbol{\mu}_{ig},\boldsymbol{\Sigma}_{ig}), where ϕni\phi_{n_i} denotes the nin_i-variate Gaussian density. Marginalizing over GiG_i gives the observed-data density fi(Yi∣Xi,ti,Di,Zi)=∑g=1Gmax⁡πigfigf_i(\mathbf{Y}_i \mid\mathbf{X}_i,t_i,D_i,\mathbf{Z}_i) = \sum_{g=1}^{G_{\max}}\pi_{ig}f_{ig}.

We approximate the nonparametric functions {βg1,βg2,ϕg:g=1,…,Gmax⁡}\{\boldsymbol{\beta}_{g1},\boldsymbol{\beta}_{g2},\boldsymbol{\phi}_g : g = 1,\ldots,G_{\max}\} using B-spline basis expansions [23]. Define the Hölder class

F={f:[0,1]→R∣∣f(q1)(s)−f(q1)(t)∣≤c0∣s−t∣q2,s,t∈[0,1]},(3)\mathcal{F} = \left\{f : [0,1] \to\mathbb{R} \mid\left|f^{(q_1)}(s) - f^{(q_1)}(t)\right| \le c_0|s-t|^{q_2},\quad s,t \in[0,1]\right\}, \tag*{(3)}

where q1q_1 is a nonnegative integer, q2∈(0,1]q_2 \in(0,1], r=q1+q2≥2r = q_1 + q_2 \ge2, and c0>0c_0 > 0. Assume that βg1,βg2∈Fd\boldsymbol{\beta}_{g1},\boldsymbol{\beta}_{g2} \in\mathcal{F}^d and ϕg∈FK\boldsymbol{\phi}_g \in\mathcal{F}^K. Let bn(t)=(b1(t),…,bqn(t))T\mathbf{b}_n(t) = (b_1(t),\ldots,b_{q_n}(t))^{\mathsf{T}} be a qnq_n-dimensional B-spline basis on [0,1][0,1]. We use βgm(t)≈θngmTbn(t)\boldsymbol{\beta}_{gm}(t) \approx\boldsymbol{\theta}_{ngm}^{\mathsf{T}}\mathbf{b}_n(t) for m=1,2m = 1,2 and ϕg(t)≈γngTbn(t)\boldsymbol{\phi}_g(t) \approx\boldsymbol{\gamma}_{ng}^{\mathsf{T}}\mathbf{b}_n(t), where θngm∈Rqn×d\boldsymbol{\theta}_{ngm} \in\mathbb{R}^{q_n \times d} and γng∈Rqn×K\boldsymbol{\gamma}_{ng} \in\mathbb{R}^{q_n \times K}.

Let Rn=∫01bn(t)bn(t)Tdt\mathbf{R}_n = \int_0^1 \mathbf{b}_n(t)\mathbf{b}_n(t)^{\mathsf{T}} \mathrm{d}t. The constraints γngTRnγng=IK\boldsymbol{\gamma}_{ng}^{\mathsf{T}}\mathbf{R}_n\boldsymbol{\gamma}_{ng} = \mathbf{I}_K ensure orthonormality. For an orthonormalized spline basis, Rn=Iqn\mathbf{R}_n = \mathbf{I}_{q_n}.

Let Bi\mathbf{B}_i be the ni×qnn_i \times q_n matrix whose jjth row is bn(tij)T\mathbf{b}_n(t_{ij})^{\mathsf{T}}. Set uij=Xij⊗bn(tij)\mathbf{u}_{ij} = \mathbf{X}_{ij} \otimes\mathbf{b}_n(t_{ij}), Ui=(ui1,…,uini)T\mathbf{U}_i = (\mathbf{u}_{i1},\ldots,\mathbf{u}_{in_i})^{\mathsf{T}}, and Wi=(Ui,DiUi)\mathbf{W}_i = (\mathbf{U}_i,D_i\mathbf{U}_i). Let θng\boldsymbol{\theta}_{ng} denote the vector formed by stacking vec⁡(θng1)\operatorname{vec}(\boldsymbol{\theta}_{ng1}) and vec⁡(θng2)\operatorname{vec}(\boldsymbol{\theta}_{ng2}). The approximated mean and covariance are then μig=Wiθng\boldsymbol{\mu}_{ig} = \mathbf{W}_i\boldsymbol{\theta}_{ng}, and Σig=BiγngΛgγngTBiT+σg2Ini\boldsymbol{\Sigma}_{ig} = \mathbf{B}_i\boldsymbol{\gamma}_{ng}\boldsymbol{\Lambda}_g\boldsymbol{\gamma}_{ng}^{\mathsf{T}}\mathbf{B}_i^{\mathsf{T}} + \sigma_g^2\mathbf{I}_{n_i}. Substituting these approximations into the mixture density, the observed-data log-likelihood is

ℓn(Θ)=∑i=1nlog⁡{∑g=1Gmax⁡πigfig},\ell_n(\boldsymbol{\Theta}) = \sum_{i=1}^{n}\log\left\{\sum_{g=1}^{G_{\max}}\pi_{ig}f_{ig}\right\},

where Θ\boldsymbol{\Theta} collects the spline coefficients, eigenvalues, measurement-error variances, global weights, and gating coefficients.

To remove redundant candidate subgroups, we apply an adapted logarithmic penalty to the global weights [9],

Pλn,ϵn(ω)=nλn∑g=1Gmax⁡log⁡{(ϵn+ωg)/ϵn},\mathcal{P}_{\lambda_n,\epsilon_n}(\boldsymbol{\omega}) = n\lambda_n\sum_{g=1}^{G_{\max}}\log\{(\epsilon_n+\omega_g)/\epsilon_n\},

where λn>0\lambda_n > 0 controls the degree of subgroup selection and ϵn>0\epsilon_n > 0 ensures that the penalty remains finite at zero. The penalized log-likelihood is

Qn(Θ)=ℓn(Θ)−Pλn,ϵn(ω).(4)Q_n(\boldsymbol{\Theta}) = \ell_n(\boldsymbol{\Theta}) - P_{\lambda_n,\epsilon_n}(\boldsymbol{\omega}). \tag*{(4)}

The estimator maximizes (4) subject to the weight-simplex constraint, the reference restriction η1=0\boldsymbol{\eta}_1 = 0, and the covariance and orthonormality constraints. The estimated number of subgroups is G^=#{g:ω^g>0}\widehat{G} = \#\{g : \widehat{\omega}_g > 0\}.

Estimation algorithm

We estimate Θ\boldsymbol{\Theta} using a blockwise ascent procedure that combines EM-type posterior weighting with direct optimization of the penalized observed-data objective QnQ_n. Each iteration first computes the posterior subgroup probabilities and then updates the longitudinal and gating parameters in separate blocks. The longitudinal block updates the mean and covariance parameters, whereas the gating block jointly updates the global weights and covariate effects. Candidate updates are accepted only when they do not decrease QnQ_n beyond a numerical tolerance. This structure accommodates the covariance and weight-simplex constraints while retaining a common objective for all model components.

Let δig=I(Gi=g)\delta_{ig} = \mathbb{I}(G_i = g) denote the latent subgroup indicator. Treating {δig:i=1,…,n,g=1,…,Gmax⁡}\{\delta_{ig} : i = 1,\ldots,n, g = 1,\ldots,G_{\max}\} as missing data, the complete-data log-likelihood is ℓnc(Θ)=∑i=1n∑g=1Gmax⁡δig{log⁡fig+log⁡πig}\ell_n^c(\boldsymbol{\Theta}) = \sum_{i=1}^n \sum_{g=1}^{G_{\max}} \delta_{ig}\{\log f_{ig} + \log\pi_{ig}\}. Given the current estimate Θ(t)\boldsymbol{\Theta}^{(t)}, the E-step computes the posterior responsibilities

hig(t)=P(Gi=g∣Oi;Θ(t))=πig(t)fig(t)∑ℓ=1Gmax⁡πiℓ(t)fiℓ(t).h_{ig}^{(t)} = \mathbb{P}(G_i = g \mid O_i; \boldsymbol{\Theta}^{(t)}) = \frac{\pi_{ig}^{(t)} f_{ig}^{(t)}}{\sum_{\ell=1}^{G_{\max}} \pi_{i\ell}^{(t)} f_{i\ell}^{(t)}}.

These probabilities combine pretreatment information through πig(t)\pi_{ig}^{(t)} with longitudinal evidence through fig(t)f_{ig}^{(t)}. They serve as subject- and subgroup-specific weights in the subsequent longitudinal updates. For notational simplicity, iteration superscripts are suppressed below, and unspecified quantities on the right-hand sides are evaluated at the current iterate.

For fixed responsibilities and covariance matrices, the mean-function coefficients are updated by posterior-weighted generalized least squares,

θ~ng={∑i=1nhigWiTΣig−1Wi}−1∑i=1nhigWiTΣig−1Yi.(5)\widetilde{\theta}_{ng} = \left\{\sum_{i=1}^{n} h_{ig} W_i^{\mathsf{T}} \Sigma_{ig}^{-1} W_i\right\}^{-1} \sum_{i=1}^{n} h_{ig} W_i^{\mathsf{T}} \Sigma_{ig}^{-1} Y_i. \tag*{(5)}

We next update the covariance structure using the GLS residuals r~ig=Yi−Wiθ~ng\widetilde{r}_{ig} = Y_i - W_i\widetilde{\theta}_{ng}. Let γngk\gamma_{ngk} denote the kkth column of γng\gamma_{ng}. With the remaining covariance parameters fixed, the covariance score equation gives the unconstrained proposal

γ~ngk={∑i=1nhigBiTΣig−1Bi}−1∑i=1nhigBiTΣig−1r~igr~igTΣig−1Biγngk,k=1,…,K.(6)\widetilde{\gamma}_{ngk} = \left\{\sum_{i=1}^{n} h_{ig} B_i^{\mathsf{T}} \Sigma_{ig}^{-1} B_i\right\}^{-1} \sum_{i=1}^{n} h_{ig} B_i^{\mathsf{T}} \Sigma_{ig}^{-1} \widetilde{r}_{ig}\widetilde{r}_{ig}^{\mathsf{T}}\Sigma_{ig}^{-1} B_i\gamma_{ngk}, \qquad k = 1,\ldots,K. \tag*{(6)}

For the eigenvalue update, let ϕigk=Biγngk\phi_{igk} = B_i\gamma_{ngk}. With the remaining covariance parameters fixed, a Fisher-scoring proposal for λgk\lambda_{gk} is

λ~gk=λgk+∑i=1nhig{ϕigkTΣig−1r~igr~igTΣig−1ϕigk−ϕigkTΣig−1ϕigk}∑i=1nhig{ϕigkTΣig−1ϕigk}2,k=1,…,K.(7)\widetilde{\lambda}_{gk}=\lambda_{gk}+\frac{\sum_{i=1}^{n}h_{ig}\left\{\phi_{igk}^{\mathsf{T}}\boldsymbol{\Sigma}_{ig}^{-1}\widetilde{\boldsymbol{r}}_{ig}\widetilde{\boldsymbol{r}}_{ig}^{\mathsf{T}}\boldsymbol{\Sigma}_{ig}^{-1}\phi_{igk}-\phi_{igk}^{\mathsf{T}}\boldsymbol{\Sigma}_{ig}^{-1}\phi_{igk}\right\}}{\sum_{i=1}^{n}h_{ig}\left\{\phi_{igk}^{\mathsf{T}}\boldsymbol{\Sigma}_{ig}^{-1}\phi_{igk}\right\}^{2}},\qquad k=1,\ldots,K. \tag*{(7)}

Similarly, with the functional covariance parameters fixed, a Fisher-scoring proposal for the measurement-error variance is

σ~g2=σg2+∑i=1nhig{r~igTΣig−2r~ig−tr⁡(Σig−1)}∑i=1nhigtr⁡(Σig−2),g=1,…,Gmax⁡.(8)\widetilde{\sigma}_{g}^{2}=\sigma_{g}^{2}+\frac{\sum_{i=1}^{n}h_{ig}\left\{\widetilde{\boldsymbol{r}}_{ig}^{\mathsf{T}}\boldsymbol{\Sigma}_{ig}^{-2}\widetilde{\boldsymbol{r}}_{ig}-\operatorname{tr}\left(\boldsymbol{\Sigma}_{ig}^{-1}\right)\right\}}{\sum_{i=1}^{n}h_{ig}\operatorname{tr}\left(\boldsymbol{\Sigma}_{ig}^{-2}\right)},\qquad g=1,\ldots,G_{\max}. \tag*{(8)}

The proposed eigenfunction coefficients are orthonormalized by QR decomposition in the orthonormal spline basis. The resulting eigenvalue–eigenfunction pairs are ordered by decreasing eigenvalue, and each eigenfunction is given a consistent sign across iterations. We then apply backtracking between the current and proposed covariance parameters, reimposing these constraints at each trial step. A trial step is accepted only if it does not decrease QnQ_n beyond the numerical tolerance; otherwise, the current covariance parameters are retained.

Holding the subgroup-conditional densities figf_{ig} fixed, we update the global weights and gating coefficients by increasing

QG(ω,η)=∑i=1nlog⁡{∑g=1Gmax⁡πigfig}−Pλn,ϵn(ω),(9)Q_G(\boldsymbol{\omega},\boldsymbol{\eta})=\sum_{i=1}^{n}\log\left\{\sum_{g=1}^{G_{\max}}\pi_{ig}f_{ig}\right\}-\mathcal{P}_{\lambda_n,\epsilon_n}(\boldsymbol{\omega}), \tag*{(9)}

subject to the weight-simplex constraint and η1=0\boldsymbol{\eta}_1=\mathbf{0}. This is the restriction of QnQ_n to the gating parameters with the longitudinal parameters held fixed.

For the gating updates, define ρig=πigfig/{∑ℓ=1Gmax⁡πiℓfiℓ}\rho_{ig}=\pi_{ig}f_{ig}/\{\sum_{\ell=1}^{G_{\max}}\pi_{i\ell}f_{i\ell}\}. Unlike the fixed E-step responsibility high_{ig}, ρig\rho_{ig} varies with the candidate gating parameters. The relevant derivatives are

∂QG∂ηg=∑i=1n(ρig−πig)Zi,g nonreference,\frac{\partial Q_G}{\partial\eta_g}=\sum_{i=1}^{n}(\rho_{ig}-\pi_{ig})\boldsymbol{Z}_i,\qquad g\ \text{nonreference},
∂QG∂ωg=1ωg∑i=1n(ρig−πig)−nλnϵn+ωg,ωg>0.(10)\frac{\partial Q_G}{\partial\omega_g}=\frac{1}{\omega_g}\sum_{i=1}^{n}(\rho_{ig}-\pi_{ig})-\frac{n\lambda_n}{\epsilon_n+\omega_g},\qquad\omega_g>0. \tag*{(10)}

We update the global weights and nonreference gating coefficients by constrained Newton-type optimization of QGQ_G, subject to the weight-simplex constraint, the reference restriction η1=0\boldsymbol{\eta}_1=\mathbf{0}, and any specified coordinate bounds. A backtracking line search is used to ensure that each accepted step does not decrease QGQ_G. After each update, components with weights at or below the numerical threshold 10−610^{-6} are removed, and the remaining weights are renormalized. The active component with the largest weight is then selected as the reference, and the gating coefficients are recentered accordingly.

Algorithm 1 summarizes the procedure for fixed (qn,K,λn)(q_n,K,\lambda_n).

Algorithm 1 Blockwise estimation for fixed (qn,K,λn)(q_n,K,\lambda_n)

Input: Data {Oi}i=1n\{\mathcal{O}_i\}_{i=1}^n, Gmax⁡G_{\max}, tuning parameters, and initial values.

Output: Θ^\widehat{\Theta} and the active set A^\widehat{\mathcal{A}}.

  1. Construct the orthonormal spline basis and design matrices.

  2. for each initialization do

  3. &nbsp;&nbsp;&nbsp;&nbsp;Initialize model parameters and the active subgroup set.

  4. &nbsp;&nbsp;&nbsp;&nbsp;repeat

  5. &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Compute the posterior responsibilities high_{ig}.

  6. &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Update the mean parameters using (5).

  7. &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Update the covariance parameters using (6)–(8) and backtracking.

  8. &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Update the gating parameters by constrained optimization of (9).

  9. &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Prune inactive components and update the active subgroup set.

  10. &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Compute

ΔQ(t)=∣Qn(t)−Qn(t−1)∣1+∣Qn(t−1)∣\Delta_Q^{(t)}=\frac{\left|Q_n^{(t)}-Q_n^{(t-1)}\right|}{1+\left|Q_n^{(t-1)}\right|}

and the blockwise first-order residual R(t)R^{(t)}.

  1. &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;until ΔQ(t)≤ϵQ\Delta_Q^{(t)}\leq\epsilon_Q and R(t)≤ϵRR^{(t)}\leq\epsilon_R, or the iteration limit is reached

  2. &nbsp;&nbsp;&nbsp;&nbsp;Store the fitted parameters and objective value.

  3. Retain the fit with the largest attained value of QnQ_n.

  4. return Θ^\widehat{\Theta} and A^\widehat{\mathcal{A}}.

Tuning-parameter selection

The tuning parameters are the spline dimension qnq_n, covariance rank KK, and penalty parameter λn\lambda_n. For each candidate configuration, we compute BIC=−2ℓn(Θ^)+df(Θ^)log⁡n\mathrm{BIC}=-2\ell_n(\widehat{\Theta})+\mathrm{df}(\widehat{\Theta})\log n, where ℓn\ell_n is the unpenalized observed-data log-likelihood. For a fitted model with G^\widehat{G} active subgroups, the nominal parameter count is

df(Θ^)=G^{2dqn+qnK−K(K+1)2+K+1}+(G^−1)(q+1).\mathrm{df}(\widehat{\Theta})=\widehat{G}\left\{2dq_n+q_nK-\frac{K(K+1)}{2}+K+1\right\}+(\widehat{G}-1)(q+1).

The terms correspond to the varying-coefficient functions, orthonormal eigenfunctions, eigenvalues, measurement-error variances, global weights, and nonreference gating coefficients. We select the configuration with the smallest BIC.

Subgroup membership estimation and prediction

For a subject in the training sample, subgroup membership is inferred from

P(Gi=g∣Oi;Θ^)=π^g(Zi)f^ig∑h=1G^π^h(Zi)f^ih.\mathbb{P}(G_i=g\mid\mathcal{O}_i;\widehat{\Theta})=\frac{\widehat{\pi}_g(Z_i)\widehat{f}_{ig}}{\sum_{h=1}^{\widehat{G}}\widehat{\pi}_h(Z_i)\widehat{f}_{ih}}.

These posterior probabilities incorporate both pretreatment covariates and the observed longitudinal response. For a new subject with pretreatment covariates z∗z_*, only pretreatment information is available. The fitted gating model then gives

π^g(z∗)=ω^gexp⁡(z∗Tη^g)∑h∈A^ω^hexp⁡(z∗Tη^h),g∈A^,(11)\widehat{\pi}_g(z_*)=\frac{\widehat{\omega}_g\exp(z_*^{\mathsf{T}}\widehat{\eta}_g)}{\sum_{h\in\widehat{\mathcal{A}}}\widehat{\omega}_h\exp(z_*^{\mathsf{T}}\widehat{\eta}_h)},\qquad g\in\widehat{\mathcal{A}}, \tag*{(11)}

where A^={g:ω^g>0}\widehat{\mathcal{A}}=\{g:\widehat{\omega}_g>0\}. The subgroup with the largest probability gives a prediction, whereas the full probability vector quantifies assignment uncertainty.

Asymptotic properties

Let Ω^n=(ω^,η^,{β^g1,β^g2,ϕ^g,Λ^g,σ^g2}g=1G^)\widehat{\Omega}_n = \left(\widehat{\omega},\widehat{\eta},\left\{\widehat{\beta}_{g1},\widehat{\beta}_{g2},\widehat{\phi}_g,\widehat{\Lambda}_g,\widehat{\sigma}_g^2\right\}_{g=1}^{\widehat{G}}\right), where β^gm(t)=θ^ngmTbn(t)\widehat{\beta}_{gm}(t) = \widehat{\theta}_{ngm}^{\mathsf{T}}b_n(t), m=1,2m=1,2, and ϕ^g(t)=η^ngTbn(t)\widehat{\phi}_g(t) = \widehat{\eta}_{ng}^{\mathsf{T}}b_n(t). The estimated covariance function is

Σ^g(s,t)=∑k=1K^λ^gkϕ^gk(s)ϕ^gk(t)+σ^g2I(s=t).\widehat{\Sigma}_g(s,t) = \sum_{k=1}^{\widehat{K}}\widehat{\lambda}_{gk}\widehat{\phi}_{gk}(s)\widehat{\phi}_{gk}(t) + \widehat{\sigma}_g^2\mathbb{I}(s=t).

Let ∥⋅∥\|\cdot\| and ∥⋅∥2\|\cdot\|_2 denote the Euclidean and L2L^2 norms, respectively. The working parameter space is

Ω∗=[0,1]Gmax⁡×Rq(Gmax⁡−1)×F2dGmax⁡×FKGmax⁡×R+KGmax⁡×R+Gmax⁡,\Omega^* = [0,1]^{G_{\max}} \times\mathbb{R}^{q(G_{\max}-1)} \times\mathcal{F}^{2dG_{\max}} \times\mathcal{F}^{KG_{\max}} \times\mathbb{R}_+^{KG_{\max}} \times\mathbb{R}_+^{G_{\max}},

with the weight-simplex, reference, and orthonormality constraints specified in § 2, where R+=(0,∞)\mathbb{R}_+ = (0,\infty).

For Γ=(ω,β,Σ)\Gamma=(\omega,\beta,\Sigma), define the distance between two G0G_0-component parameter values, after aligning their component labels, by

d(Γ1,Γ2)={∥ω1−ω2∥2+∑g=1G0∑m=12∥βgm,1−βgm,2∥22+∑g=1G0∥Σg,1−Σg,2∥22}1/2.d(\Gamma_1,\Gamma_2) = \left\{\|\omega_1-\omega_2\|^2+\sum_{g=1}^{G_0}\sum_{m=1}^{2}\|\beta_{gm,1}-\beta_{gm,2}\|_2^2+\sum_{g=1}^{G_0}\|\Sigma_{g,1}-\Sigma_{g,2}\|_2^2\right\}^{1/2}.

A subscript 0 denotes a true parameter or function. Without loss of generality, order the global weights so that ω1,0≥⋯≥ωG0,0>0\omega_{1,0}\ge\cdots\ge\omega_{G_0,0}>0 and ωG0+1,0=⋯=ωGmax⁡,0=0\omega_{G_0+1,0}=\cdots=\omega_{G_{\max},0}=0. The following conditions are used to establish the asymptotic properties of Ω^n\widehat{\Omega}_n.

(A1) Let 0=t0<t1<⋯<tMn<tMn+1=10=t_0<t_1<\cdots<t_{M_n}<t_{M_n+1}=1 denote the spline-knot sequence. Then max⁡1≤j≤Mn+1(tj−tj−1)=O(n−κ)\max_{1\le j\le M_n+1}(t_j-t_{j-1})=O(n^{-\kappa}) for 0<κ<1/20<\kappa<1/2, and the ratio of the largest to the smallest knot spacing is bounded.

(A2) The parameters associated with the G0G_0 active components lie in the interior of their corresponding parameter spaces.

(A3) K=O(ne)K=O(n^e), where 0≤e<min⁡(1−κ,2rκ)0\le e<\min(1-\kappa,2r\kappa).

(A4) For each active subgroup, ∑k=1∞λgk<∞\sum_{k=1}^{\infty}\lambda_{gk}<\infty.

(A5) The matrix E{S(ζ0)S(ζ0)T}E\{S(\zeta_0)S(\zeta_0)^{\mathsf{T}}\} is finite and positive definite, where ζ=(σ2,ω−1,η2:G0)T\zeta=(\sigma^2,\omega_{-1},\eta_{2:G_0})^{\mathsf{T}}. Here ω−1=(ω2,…,ωG0)T\omega_{-1}=(\omega_2,\ldots,\omega_{G_0})^{\mathsf{T}}, η2:G0\eta_{2:G_0} stacks the nonreference gating coefficients, and η1=0\eta_1=0. The score S(ζ0)S(\zeta_0) is defined in the proof of Theorem 3.

Condition (A1) excludes highly unbalanced knot sequences and is standard in spline analysis [17]. Condition (A2) excludes degenerate active-component parameters [2]. Condition (A3) controls the growth of the covariance rank, which is typically small in practice. Condition (A4) ensures that each covariance operator is trace class [8]. Condition (A5) ensures that the asymptotic covariance matrix is well defined [18].

Theorem 1. Suppose Conditions (A1)–(A5) hold, λnn→0\lambda_n\sqrt{n}\to0, λnnlog⁡n→∞\lambda_n\sqrt{n}\log n\to\infty, and ϵn=o{1/(nlog⁡n)}\epsilon_n=o\{1/(\sqrt{n}\log n)\}. Then P(G^=G0)→1\mathbb{P}(\widehat{G}=G_0)\to1 as n→∞n\to\infty.

Theorem 2. Suppose Conditions (A1)(A1)–(A5)(A5) hold, λnn→0\lambda_n\sqrt{n} \to0, and ϵn=o{1/(nlog⁡n)}\epsilon_n = o\{1/(\sqrt{n}\log n)\}. Then

d(Γ^n,Γ0)=Op[n−min⁡{(1−κ−ϵ)/2, rκ−ϵ/2}].d(\widehat{\Gamma}_n,\Gamma_0) = O_p\left[n^{-\min\{(1-\kappa-\epsilon)/2,\,r\kappa-\epsilon/2\}}\right].

Here rr is the smoothness index defined in (3), and the knot sequence grows at rate nκn^\kappa. Increasing ϵ\epsilon enlarges the covariance parameter space and slows the convergence rate. If ϵ=0\epsilon= 0, so that KK is fixed, then

d(Γ^n,Γ0)=Op[n−min⁡{(1−κ)/2, rκ}].d(\widehat{\Gamma}_n,\Gamma_0) = O_p\left[n^{-\min\{(1-\kappa)/2,\,r\kappa\}}\right].

In particular, choosing κ=1/(2r+1)\kappa= 1/(2r+1) yields d(Γ^n,Γ0)=Op{n−r/(2r+1)}d(\widehat{\Gamma}_n,\Gamma_0) = O_p\{n^{-r/(2r+1)}\}, the optimal rate for univariate nonparametric function estimation.

Theorem 3. Suppose Conditions (A1)(A1)–(A5)(A5) hold with r≥2r \ge2, 1/(4r)<κ<1/21/(4r) < \kappa< 1/2, λnn→0\lambda_n\sqrt{n} \to0, and ϵn=o{1/(nlog⁡n)}\epsilon_n = o\{1/(\sqrt{n}\log n)\}. For ζ=(σ2,ω−1,η2:G0)T\zeta= (\sigma^2,\omega_{-1},\eta_{2:G_0})^\mathsf{T},

n(ζ^−ζ0)→dN{0,I−1(ζ0)},\sqrt{n}(\widehat{\zeta}-\zeta_0) \xrightarrow{d} N\{0,I^{-1}(\zeta_0)\},

where I(ζ0)=E{S(ζ0)S(ζ0)T}I(\zeta_0) = \mathbb{E}\{S(\zeta_0)S(\zeta_0)^\mathsf{T}\}.

In particular,

n(ω^−1−ω−1,0)→dN{0,[I−1(ζ0)]ωω}.\sqrt{n}(\widehat{\omega}_{-1}-\omega_{-1,0}) \xrightarrow{d} N\{0,[I^{-1}(\zeta_0)]_{\omega\omega}\}.

Here [I−1(ζ0)]ωω[I^{-1}(\zeta_0)]_{\omega\omega} is the block corresponding to ω−1\omega_{-1}. Because ω1=1−∑g=2G0ωg\omega_1 = 1-\sum_{g=2}^{G_0}\omega_g, this result also determines the asymptotic distribution of the full vector of active global weights.

Simulation study

We evaluate the finite-sample performance of the proposed joint method through Monte Carlo simulations. The primary comparator is a two-step procedure that first identifies longitudinal subgroups and then relates estimated membership to pretreatment covariates Zhang et al. [32]. We also include an oracle estimator based on the true subgroup labels and a latent class mixed-model benchmark (HLME) Lin et al. [14]. We assess subgroup-number selection, longitudinal parameter estimation, training membership recovery, and pretreatment membership prediction for new subjects.

Simulation design

Each replication includes a training sample with ntrain∈{600,800}n_{\mathrm{train}} \in\{600,800\} and an independent test sample with ntest=400n_{\mathrm{test}} = 400. The true subgroup number is G0=2G_0 = 2, and the candidate upper bound is Gmax⁡∈{5,8,10}G_{\max} \in\{5,8,10\}. We use R=200R = 200 independent replications for each setting. Treatment is generated as Di∼Bernoulli⁡(0.5)D_i \sim\operatorname{Bernoulli}(0.5), and ni∼DiscreteUniform⁡{8,9,10,11,12}n_i \sim\operatorname{DiscreteUniform}\{8,9,10,11,12\}. Conditional on nin_i, observation times are sampled independently from Uniform⁡(0,1)\operatorname{Uniform}(0,1) and then ordered within subject.

The longitudinal covariate vector is Xij={Xi1(tij),Xi2(tij),Xi3(tij)}T\mathbf{X}_{ij} = \{X_{i1}(t_{ij}),X_{i2}(t_{ij}),X_{i3}(t_{ij})\}^{\mathsf{T}}, where Xi1(t)=1X_{i1}(t)=1, Xi2(t)=AiX_{i2}(t)=A_i, and Ai∼N(1,1)A_i \sim N(1,1). Its third component is

Xi3(t)=12+sin⁡(2πt)+∑m=110ξimψm(t),ψm(t)=2cos⁡(mπt),ξim∼N(0,4m−2).X_{i3}(t)=\frac{1}{2}+\sin(2\pi t)+\sum_{m=1}^{10}\xi_{im}\psi_m(t),\qquad\psi_m(t)=\sqrt{2}\cos(m\pi t),\qquad\xi_{im}\sim N(0,4m^{-2}).

The scalar AiA_i is generated once per subject. The coefficients ξim\xi_{im} are independent across subjects and basis indices. The pretreatment predictors are generated independently as Zi1,Zi2∼N(1,4)Z_{i1},Z_{i2}\sim N(1,4). Define Z~i1=Zi1−1\widetilde{Z}_{i1}=Z_{i1}-1, Z~i2=Zi2−1\widetilde{Z}_{i2}=Z_{i2}-1, and Z~i=(Z~i1,Z~i2)T\widetilde{Z}_i=(\widetilde{Z}_{i1},\widetilde{Z}_{i2})^{\mathsf{T}}. The latent subgroup label is generated according to

pi:=Pr⁡(Gi=1∣Zi)=exp⁡{c(Z~i1+Z~i2)}1+exp⁡{c(Z~i1+Z~i2)},Pr⁡(Gi=2∣Zi)=1−pi,(12)p_i := \Pr(G_i=1\mid Z_i)=\frac{\exp\{c(\widetilde{Z}_{i1}+\widetilde{Z}_{i2})\}}{1+\exp\{c(\widetilde{Z}_{i1}+\widetilde{Z}_{i2})\}}, \qquad\Pr(G_i=2\mid Z_i)=1-p_i, \tag*{(12)}

with c=2c=2. Because the linear predictor is symmetric about zero, both subgroups have marginal prevalence 0.5. Under the proposed parameterization, the true gating parameters are ω1=ω2=0.5\omega_1=\omega_2=0.5, η1=0\eta_1=0, and η2=(−2,−2)T\eta_2=(-2,-2)^{\mathsf{T}}.

Conditional on Gi=gG_i=g, the longitudinal response follows

Yi(tij)=XijT{βg1(tij)+Diβg2(tij)}+VigTϕg(tij)+ϵigj,Y_i(t_{ij})=X_{ij}^{\mathsf{T}}\{\beta_{g1}(t_{ij})+D_i\beta_{g2}(t_{ij})\}+V_{ig}^{\mathsf{T}}\phi_g(t_{ij})+\epsilon_{igj},

where Vig∼N2(0,Λg)V_{ig}\sim N_2(0,\Lambda_g) and ϵigj∼N(0,σg2)\epsilon_{igj}\sim N(0,\sigma_g^2) are mutually independent. We set Λ1=diag⁡(1,0.25)\Lambda_1=\operatorname{diag}(1,0.25), Λ2=diag⁡(1.2,0.30)\Lambda_2=\operatorname{diag}(1.2,0.30), and σ12=σ22=0.25\sigma_1^2=\sigma_2^2=0.25. Let 13=(1,1,1)T\mathbf{1}_3=(1,1,1)^{\mathsf{T}}. The subgroup-specific coefficient functions are

β11(t)=(1+0.45t)13,β12(t)=(0.4−0.35exp⁡(t))13,β21(t)=(1+0.15t)13,β22(t)=(0.4−0.10exp⁡(t))13.\begin{aligned} \beta_{11}(t)&=(1+0.45t)\mathbf{1}_3, & \beta_{12}(t)&=(0.4-0.35\exp(t))\mathbf{1}_3,\\ \beta_{21}(t)&=(1+0.15t)\mathbf{1}_3, & \beta_{22}(t)&=(0.4-0.10\exp(t))\mathbf{1}_3. \end{aligned}

and ϕ1(t)={2cos⁡(πt),2sin⁡(πt)}T\phi_1(t)=\{\sqrt{2}\cos(\pi t),\sqrt{2}\sin(\pi t)\}^{\mathsf{T}}, with ϕ2(t)=ϕ1(t)\phi_2(t)=\phi_1(t).

Both penalized procedures use the same orthonormalized cubic B-spline basis, with interior knots at 0.25, 0.50, and 0.75 (qn=7q_n=7), and covariance rank K=2K=2. We use λ=0.02\lambda=0.02 for the joint estimator and λ=0.04\lambda=0.04 for the first stage of the two-step procedure. Joint-model gating coefficients are constrained to [−5,5][-5,5], whereas the second-stage logistic coefficients are unconstrained. Each longitudinal fit is allowed at most 200 iterations.

Competing methods and implementation

The joint estimator fits the longitudinal and membership models simultaneously using Algorithm 1. The two-step comparator first fits a longitudinal mixture with constant component weights. It then treats the posterior membership probabilities as fixed soft responses in a multinomial logistic regression on the centered pretreatment predictors. The second stage does not update the longitudinal parameters. For training classification, the fitted gating model is combined with the fixed component densities. Predictions for new subjects use the gating model alone. This implementation uses soft-label logistic regression rather than the posterior-log-odds regression of Zhang et al. [32].

The oracle estimator fits two groups using the true training labels. HLME fits two latent classes using hlme from the R package lcmm ([21]). It uses the same spline mean design and pretreatment membership predictors. Its random-effects model uses the true cosine and sine functions, a diagonal covariance, a class-specific proportional factor, and a common residual variance. Thus, HLME is supplied with the true subgroup number and covariance shapes; it neither estimates eigenfunctions nor selects the subgroup number.

Evaluation criteria

We report the number of replications that correctly identify G0=2G_0=2 out of 200 (RCI). Training membership recovery is measured by IIC, the proportion correctly classified after label alignment. For new subjects, we report classification accuracy (ACC), Brier score, negative log predictive probability (NLP), and RMSE⁡p={ntest−1∑i(p^i−pi)2}1/2\operatorname{RMSE}_p=\{n_{\mathrm{test}}^{-1}\sum_i(\widehat{p}_i-p_i)^2\}^{1/2}, where pip_i is defined in (12). Higher ACC and lower Brier score, NLP, and RMSEp\mathrm{RMSE}_{p} indicate better prediction. All test predictions use pretreatment covariates alone. Parameter results are reported as RMSE (bias), with signed bias for scalar parameters. For a function evaluated at M=200M = 200 equally spaced grid points, functional bias is {M−1∑m[f^(tm)−f0(tm)]2}1/2\left\{M^{-1}\sum_m[\widehat{f}(t_m)-f_0(t_m)]^2\right\}^{1/2}. Functional RMSE averages squared errors over grid points and replications.

Simulation results

Tables 1 and 2 summarize the results for n=600n = 600 and n=800n = 800. Each table compares the oracle estimator, the fixed-two-group HLME benchmark, and both penalized procedures initialized at Gmax⁡=5,8,10G_{\max} = 5, 8, 10. The comparison involved 3,200 fits across 400 paired training and test datasets, and all fits completed without execution errors. All oracle and HLME fits met their numerical convergence criteria. Returned estimates from the penalized procedures that did not meet the numerical convergence criterion are retained in the summaries.

For n=600n = 600, the joint estimator selected two subgroups in 196, 194, and 192 replications for Gmax⁡=5,8,10G_{\max} = 5, 8, 10, respectively. The corresponding counts for the two-step procedure were 196, 180, and 191. For n=800n = 800, these counts were 200, 200, and 198 for the joint estimator and 193, 194, and 195 for the two-step procedure. Thus, the joint estimator had equal or higher correct-selection counts in every configuration considered, indicating that the global-weight penalty effectively removed redundant candidate subgroups across the three choices of Gmax⁡G_{\max}.

Across the candidate upper bounds, the joint estimator’s errors in the longitudinal parameters were stable. Its varying-coefficient RMSEs were consistently smaller than those of the two-step procedure, and its eigenfunction RMSEs were also generally smaller. This pattern is consistent with more accurate recovery of the longitudinal component when pretreatment covariates inform subgroup learning.

Among fits selecting two subgroups, the joint estimator achieved IIC values of 95.42–95.44% for n=600n = 600 and approximately 95.61% for n=800n = 800. The corresponding ranges for the two-step procedure were 67.17–69.36% and 65.08–65.88%. Test-set classification accuracy ranged from 90.48–90.59% for the joint estimator and 79.60–81.78% for the two-step procedure. The corresponding Brier score ranges were 0.068–0.069 and 0.182–0.194, and test-set RMSEp\mathrm{RMSE}_{p} ranged from 0.034 to 0.042 and from 0.331 to 0.350, respectively. Bias in the gating slopes also showed substantially less attenuation under the joint estimator. Thus, the improvement was evident in training-set membership recovery, test-set classification, and probability estimation. These comparisons remain conditional on selecting two subgroups and use method-specific eligible replications.

The correctly specified HLME benchmark had IIC values of 95.53% and 95.70% for n=600n = 600 and n=800n = 800, respectively. Its prediction performance was close to that of the joint estimator. HLME often had smaller covariance and gating-parameter RMSEs, reflecting the additional subgroup-number and covariance-shape information supplied to this benchmark. Accordingly, the joint estimator did not outperform HLME for every parameter. The oracle estimator provided a known-membership reference for longitudinal estimation. Conditional selection and Monte Carlo variation can produce small departures from a strict empirical RMSE ordering.

For n=600n = 600 and Gmax⁡=5G_{\max} = 5, the average estimated eigenfunctions and varying-coefficient functions followed their true shapes. Figure 1 displays the four subgroup-specific eigenfunctions, and Figure 2 displays the six coefficient functions for each subgroup. Solid curves denote the true functions, and dashed curves denote averages across replications. The pointwise bands summarize empirical variation across the 196 replications selecting two subgroups after label and eigenfunction alignment. They are neither standard errors of the Monte Carlo averages nor simultaneous confidence bands. Variation was greater near the boundaries for several coefficient functions.

Estimated eigenfunctions for $n=600$ and candidate upper bound $G_{\max}=5$

Figure 1. Estimated eigenfunctions for n=600n = 600 and candidate upper bound Gmax⁡=5G_{\max} = 5. The first row displays ϕ11(t)\phi_{11}(t) and ϕ12(t)\phi_{12}(t); the second displays ϕ21(t)\phi_{21}(t) and ϕ22(t)\phi_{22}(t). Solid curves are the true functions, and dashed curves are averages across replications. Gray areas are pointwise 95% Monte Carlo variability bands, constructed as the average estimate ±1.96\pm1.96 times the pointwise empirical standard deviation across the 196 replications (of 200) that selected two subgroups. Curves and band boundaries are evaluated at 200 equally spaced points on [0,1][0,1], including both endpoints. These bands describe sampling variation across replications.

Estimated varying coefficients for both subgroups

Figure 2. Estimated varying coefficients for both subgroups, with n=600n = 600 and candidate upper bound Gmax⁡=5G_{\max} = 5. The top two rows show subgroup 1, and the bottom two rows show subgroup 2. Within each subgroup, the first row displays βg11(t)\beta_{g11}(t), βg12(t)\beta_{g12}(t), and βg13(t)\beta_{g13}(t); the second shows βg21(t)\beta_{g21}(t), βg22(t)\beta_{g22}(t), and βg23(t)\beta_{g23}(t). Solid curves, dashed curves, and gray areas denote the true functions, averages across replications, and pointwise 95% Monte Carlo variability bands, respectively, evaluated at 200 equally spaced points on [0,1][0,1], using the same 196 replications and construction as in Figure 1.

ParameterOracleHLME (G=2G = 2)G=5G = 5 ProposedG=5G = 5 Two-stepG=8G = 8 ProposedG=8G = 8 Two-stepG=10G = 10 ProposedG=10G = 10 Two-step
λ11\lambda_{11}0.080 (−0.012)0.080\ (-0.012)0.072 (−0.015)0.072\ (-0.015)0.083 (−0.010)0.083\ (-0.010)0.136 (−0.043)0.136\ (-0.043)0.082 (−0.011)0.082\ (-0.011)0.129 (−0.034)0.129\ (-0.034)0.082 (−0.009)0.082\ (-0.009)0.137 (−0.041)0.137\ (-0.041)
λ12\lambda_{12}0.023 (−0.008)0.023\ (-0.008)0.021 (−0.008)0.021\ (-0.008)0.025 (−0.009)0.025\ (-0.009)0.031 (−0.002)0.031\ (-0.002)0.025 (−0.008)0.025\ (-0.008)0.031 (0.000)0.031\ (0.000)0.025 (−0.008)0.025\ (-0.008)0.029 (0.000)0.029\ (0.000)
λ21\lambda_{21}0.118 (−0.008)0.118\ (-0.008)0.102 (−0.010)0.102\ (-0.010)0.120 (−0.010)0.120\ (-0.010)0.157 (−0.074)0.157\ (-0.074)0.121 (−0.010)0.121\ (-0.010)0.149 (−0.064)0.149\ (-0.064)0.120 (−0.008)0.120\ (-0.008)0.157 (−0.072)0.157\ (-0.072)
λ22\lambda_{22}0.028 (−0.007)0.028\ (-0.007)0.024 (−0.007)0.024\ (-0.007)0.029 (−0.007)0.029\ (-0.007)0.042 (−0.024)0.042\ (-0.024)0.029 (−0.007)0.029\ (-0.007)0.042 (−0.023)0.042\ (-0.023)0.030 (−0.008)0.030\ (-0.008)0.041 (−0.023)0.041\ (-0.023)
ω1\omega_1/0.057 (0.001)0.057\ (0.001)0.101 (0.004)0.101\ (0.004)0.040 (0.001)0.040\ (0.001)0.096 (0.007)0.096\ (0.007)0.040 (−0.002)0.040\ (-0.002)0.094 (0.005)0.094\ (0.005)0.038 (0.000)0.038\ (0.000)
ω2\omega_2/0.057 (−0.001)0.057\ (-0.001)0.101 (−0.004)0.101\ (-0.004)0.040 (−0.001)0.040\ (-0.001)0.096 (−0.007)0.096\ (-0.007)0.040 (0.002)0.040\ (0.002)0.094 (−0.005)0.094\ (-0.005)0.038 (0.000)0.038\ (0.000)
η21\eta_{21}/0.381 (−0.084)0.381\ (-0.084)0.555 (−0.148)0.555\ (-0.148)1.841 (1.835)1.841\ (1.835)0.516 (−0.131)0.516\ (-0.131)1.819 (1.812)1.819\ (1.812)0.498 (−0.137)0.498\ (-0.137)1.826 (1.819)1.826\ (1.819)
η22\eta_{22}/0.376 (−0.087)0.376\ (-0.087)0.553 (−0.153)0.553\ (-0.153)1.838 (1.832)1.838\ (1.832)0.509 (−0.138)0.509\ (-0.138)1.814 (1.807)1.814\ (1.807)0.498 (−0.145)0.498\ (-0.145)1.820 (−1.813)1.820\ (-1.813)
σ12\sigma_1^20.008 (−0.004)0.008\ (-0.004)0.006 (−0.003)0.006\ (-0.003)0.009 (−0.005)0.009\ (-0.005)0.010 (−0.005)0.010\ (-0.005)0.009 (−0.005)0.009\ (-0.005)0.010 (−0.005)0.010\ (-0.005)0.009 (−0.005)0.009\ (-0.005)0.009 (−0.004)0.009\ (-0.004)
σ22\sigma_2^20.008 (−0.004)0.008\ (-0.004)0.006 (−0.003)0.006\ (-0.003)0.009 (−0.004)0.009\ (-0.004)0.011 (−0.006)0.011\ (-0.006)0.009 (−0.004)0.009\ (-0.004)0.011 (−0.006)0.011\ (-0.006)0.009 (−0.004)0.009\ (-0.004)0.011 (−0.006)0.011\ (-0.006)
ϕ11(t)\phi_{11}(t)0.047 (0.002)0.047\ (0.002)/0.051 (0.003)0.051\ (0.003)0.062 (0.003)0.062\ (0.003)0.050 (0.002)0.050\ (0.002)0.060 (0.003)0.060\ (0.003)0.051 (0.003)0.051\ (0.003)0.062 (0.003)0.062\ (0.003)
ϕ12(t)\phi_{12}(t)0.065 (0.004)0.065\ (0.004)/0.070 (0.004)0.070\ (0.004)0.080 (0.005)0.080\ (0.005)0.069 (0.004)0.069\ (0.004)0.079 (0.004)0.079\ (0.004)0.069 (0.005)0.069\ (0.005)0.079 (0.004)0.079\ (0.004)
ϕ21(t)\phi_{21}(t)0.047 (0.005)0.047\ (0.005)/0.049 (0.005)0.049\ (0.005)0.060 (0.004)0.060\ (0.004)0.054 (0.004)0.054\ (0.004)0.058 (0.004)0.058\ (0.004)0.049 (0.004)0.049\ (0.004)0.060 (0.004)0.060\ (0.004)
ϕ22(t)\phi_{22}(t)0.062 (0.006)0.062\ (0.006)/0.065 (0.007)0.065\ (0.007)0.081 (0.006)0.081\ (0.006)0.069 (0.005)0.069\ (0.005)0.078 (0.005)0.078\ (0.005)0.065 (0.006)0.065\ (0.006)0.080 (0.006)0.080\ (0.006)
β111(t)\beta_{111}(t)0.137 (0.008)0.137\ (0.008)0.148 (0.011)0.148\ (0.011)0.148 (0.012)0.148\ (0.012)0.254 (0.123)0.254\ (0.123)0.148 (0.012)0.148\ (0.012)0.257 (0.126)0.257\ (0.126)0.146 (0.010)0.146\ (0.010)0.253 (0.115)0.253\ (0.115)
β112(t)\beta_{112}(t)0.101 (0.005)0.101\ (0.005)0.106 (0.005)0.106\ (0.005)0.107 (0.005)0.107\ (0.005)0.202 (0.131)0.202\ (0.131)0.107 (0.006)0.107\ (0.006)0.194 (0.102)0.194\ (0.102)0.107 (0.005)0.107\ (0.005)0.194 (0.119)0.194\ (0.119)
β113(t)\beta_{113}(t)0.020 (0.001)0.020\ (0.001)0.021 (0.001)0.021\ (0.001)0.021 (0.001)0.021\ (0.001)0.148 (0.125)0.148\ (0.125)0.021 (0.001)0.021\ (0.001)0.141 (0.114)0.141\ (0.114)0.021 (0.001)0.021\ (0.001)0.143 (0.116)0.143\ (0.116)
β121(t)\beta_{121}(t)0.215 (0.015)0.215\ (0.015)0.222 (0.016)0.222\ (0.016)0.221 (0.020)0.221\ (0.020)0.308 (0.120)0.308\ (0.120)0.221 (0.017)0.221\ (0.017)0.320 (0.119)0.320\ (0.119)0.222 (0.019)0.222\ (0.019)0.311 (0.111)0.311\ (0.111)
β122(t)\beta_{122}(t)0.152 (0.008)0.152\ (0.008)0.156 (0.007)0.156\ (0.007)0.156 (0.008)0.156\ (0.008)0.238 (0.134)0.238\ (0.134)0.156 (0.008)0.156\ (0.008)0.240 (0.107)0.240\ (0.107)0.156 (0.009)0.156\ (0.009)0.236 (0.125)0.236\ (0.125)
β123(t)\beta_{123}(t)0.028 (0.001)0.028\ (0.001)0.030 (0.001)0.030\ (0.001)0.030 (0.002)0.030\ (0.002)0.149 (0.125)0.149\ (0.125)0.030 (0.001)0.030\ (0.001)0.142 (0.114)0.142\ (0.114)0.030 (0.001)0.030\ (0.001)0.146 (0.118)0.146\ (0.118)
β211(t)\beta_{211}(t)0.154 (0.012)0.154\ (0.012)0.162 (0.015)0.162\ (0.015)0.163 (0.016)0.163\ (0.016)0.268 (0.122)0.268\ (0.122)0.162 (0.017)0.162\ (0.017)0.256 (0.118)0.256\ (0.118)0.163 (0.013)0.163\ (0.013)0.255 (0.112)0.255\ (0.112)
β212(t)\beta_{212}(t)0.106 (0.007)0.106\ (0.007)0.110 (0.008)0.110\ (0.008)0.111 (0.008)0.111\ (0.008)0.212 (0.132)0.212\ (0.132)0.110 (0.008)0.110\ (0.008)0.196 (0.107)0.196\ (0.107)0.111 (0.006)0.111\ (0.006)0.208 (0.121)0.208\ (0.121)
β213(t)\beta_{213}(t)0.020 (0.001)0.020\ (0.001)0.020 (0.001)0.020\ (0.001)0.021 (0.001)0.021\ (0.001)0.148 (0.125)0.148\ (0.125)0.021 (0.001)0.021\ (0.001)0.141 (0.113)0.141\ (0.113)0.021 (0.001)0.021\ (0.001)0.142 (0.116)0.142\ (0.116)
β221(t)\beta_{221}(t)0.216 (0.017)0.216\ (0.017)0.226 (0.020)0.226\ (0.020)0.225 (0.021)0.225\ (0.021)0.319 (0.110)0.319\ (0.110)0.228 (0.021)0.228\ (0.021)0.308 (0.103)0.308\ (0.103)0.230 (0.020)0.230\ (0.020)0.315 (0.101)0.315\ (0.101)
β222(t)\beta_{222}(t)0.148 (0.012)0.148\ (0.012)0.154 (0.013)0.154\ (0.013)0.154 (0.013)0.154\ (0.013)0.242 (0.137)0.242\ (0.137)0.154 (0.012)0.154\ (0.012)0.237 (0.112)0.237\ (0.112)0.156 (0.013)0.156\ (0.013)0.239 (0.127)0.239\ (0.127)
β223(t)\beta_{223}(t)0.027 (0.002)0.027\ (0.002)0.029 (0.002)0.029\ (0.002)0.029 (0.002)0.029\ (0.002)0.150 (0.125)0.150\ (0.125)0.029 (0.002)0.029\ (0.002)0.143 (0.114)0.143\ (0.114)0.029 (0.002)0.029\ (0.002)0.147 (0.118)0.147\ (0.118)
RCI//196/200196/200194/200180/200192/200191/200
IIC100.00%95.53%95.42%67.17%95.44%69.36%95.43%68.63%
Test ACC/90.51%90.48%79.60%90.51%80.31%90.53%80.17%
Brier/0.0680.0690.1880.0680.1820.0680.184
NLP/0.2200.2240.5610.2230.5460.2220.550
RMSEp_p/0.0340.0420.3410.0410.3310.0410.334

Table 1. Simulation results for n=600n = 600 (200 replications per method).

Note. Parameter entries are RMSE (bias). Scalar bias is signed; functional bias is the root integrated squared bias. RCI uses all 200 replications. Parameter errors, IIC, and test metrics use method-specific fits selecting two subgroups, irrespective of numerical convergence.

The oracle uses known training labels. HLME fixes two groups and the cosine and sine random-effect functions; unavailable quantities are marked “/”. Test prediction uses pretreatment covariates only. Proposed: λ=0.02\lambda= 0.02, ∣η∣≤5|\eta| \le5; two-step: λ=0.04\lambda= 0.04.

ParameterOracleHLME (G=2G = 2)Proposed (G=5G = 5)Two-step (G=5G = 5)Proposed (G=8G = 8)Two-step (G=8G = 8)Proposed (G=10G = 10)Two-step (G=10G = 10)
λ11\lambda_{11}0.067 (−0.003)0.067\ (-0.003)0.063 (−0.008)0.063\ (-0.008)0.068 (−0.003)0.068\ (-0.003)0.102 (0.046)0.102\ (0.046)0.068 (−0.003)0.068\ (-0.003)0.102 (0.052)0.102\ (0.052)0.068 (−0.003)0.068\ (-0.003)0.106 (0.049)0.106\ (0.049)
λ12\lambda_{12}0.020 (−0.003)0.020\ (-0.003)0.017 (−0.001)0.017\ (-0.001)0.021 (−0.003)0.021\ (-0.003)0.027 (0.011)0.027\ (0.011)0.021 (−0.003)0.021\ (-0.003)0.027 (0.011)0.027\ (0.011)0.021 (−0.003)0.021\ (-0.003)0.028 (0.012)0.028\ (0.012)
λ21\lambda_{21}0.088 (−0.020)0.088\ (-0.020)0.084 (−0.019)0.084\ (-0.019)0.091 (−0.021)0.091\ (-0.021)0.130 (−0.083)0.130\ (-0.083)0.091 (−0.021)0.091\ (-0.021)0.133 (−0.086)0.133\ (-0.086)0.090 (−0.021)0.090\ (-0.021)0.133 (−0.086)0.133\ (-0.086)
λ22\lambda_{22}0.024 (−0.003)0.024\ (-0.003)0.020 (−0.004)0.020\ (-0.004)0.025 (−0.004)0.025\ (-0.004)0.035 (−0.020)0.035\ (-0.020)0.025 (−0.004)0.025\ (-0.004)0.034 (−0.020)0.034\ (-0.020)0.025 (−0.004)0.025\ (-0.004)0.036 (−0.022)0.036\ (-0.022)
ω1\omega_1/0.046 (−0.001)0.046\ (-0.001)0.069 (−0.001)0.069\ (-0.001)0.030 (−0.002)0.030\ (-0.002)0.071 (0.000)0.071\ (0.000)0.029 (−0.002)0.029\ (-0.002)0.070 (−0.001)0.070\ (-0.001)0.030 (0.000)0.030\ (0.000)
ω2\omega_2/0.046 (0.001)0.046\ (0.001)0.069 (0.001)0.069\ (0.001)0.030 (−0.002)0.030\ (-0.002)0.071 (0.000)0.071\ (0.000)0.029 (−0.002)0.029\ (-0.002)0.070 (0.001)0.070\ (0.001)0.030 (0.000)0.030\ (0.000)
η21\eta_{21}/0.285 (−0.067)0.285\ (-0.067)0.304 (−0.071)0.304\ (-0.071)1.852 (1.846)1.852\ (1.846)0.318 (−0.082)0.318\ (-0.082)1.855 (1.849)1.855\ (1.849)0.308 (−0.085)0.308\ (-0.085)1.860 (1.854)1.860\ (1.854)
η22\eta_{22}/0.282 (−0.079)0.282\ (-0.079)0.301 (−0.083)0.301\ (-0.083)1.849 (1.843)1.849\ (1.843)0.316 (−0.095)0.316\ (-0.095)1.850 (1.845)1.850\ (1.845)0.306 (−0.096)0.306\ (-0.096)1.856 (1.850)1.856\ (1.850)
σ12\sigma_1^20.007 (−0.003)0.007\ (-0.003)0.005 (−0.003)0.005\ (-0.003)0.007 (−0.003)0.007\ (-0.003)0.008 (−0.004)0.008\ (-0.004)0.007 (−0.003)0.007\ (-0.003)0.008 (−0.004)0.008\ (-0.004)0.007 (−0.003)0.007\ (-0.003)0.008 (−0.004)0.008\ (-0.004)
σ22\sigma_2^20.007 (−0.003)0.007\ (-0.003)0.005 (−0.003)0.005\ (-0.003)0.007 (−0.003)0.007\ (-0.003)0.009 (−0.005)0.009\ (-0.005)0.007 (−0.003)0.007\ (-0.003)0.009 (−0.005)0.009\ (-0.005)0.007 (−0.003)0.007\ (-0.003)0.009 (−0.004)0.009\ (-0.004)
ϕ11(t)\phi_{11}(t)0.043 (0.003)0.043\ (0.003)/0.045 (0.002)0.045\ (0.002)0.050 (0.002)0.050\ (0.002)0.045 (0.002)0.045\ (0.002)0.050 (0.003)0.050\ (0.003)0.046 (0.002)0.046\ (0.002)0.050 (0.003)0.050\ (0.003)
ϕ12(t)\phi_{12}(t)0.057 (0.003)0.057\ (0.003)/0.060 (0.003)0.060\ (0.003)0.065 (0.004)0.065\ (0.004)0.059 (0.003)0.059\ (0.003)0.065 (0.004)0.065\ (0.004)0.060 (0.003)0.060\ (0.003)0.065 (0.003)0.065\ (0.003)
ϕ21(t)\phi_{21}(t)0.040 (0.002)0.040\ (0.002)/0.042 (0.003)0.042\ (0.003)0.050 (0.007)0.050\ (0.007)0.042 (0.003)0.042\ (0.003)0.049 (0.006)0.049\ (0.006)0.042 (0.003)0.042\ (0.003)0.050 (0.008)0.050\ (0.008)
ϕ22(t)\phi_{22}(t)0.052 (0.004)0.052\ (0.004)/0.056 (0.004)0.056\ (0.004)0.065 (0.008)0.065\ (0.008)0.055 (0.004)0.055\ (0.004)0.065 (0.007)0.065\ (0.007)0.055 (0.004)0.055\ (0.004)0.066 (0.009)0.066\ (0.009)
β111(t)\beta_{111}(t)0.122 (0.011)0.122\ (0.011)0.133 (0.010)0.133\ (0.010)0.133 (0.011)0.133\ (0.011)0.233 (0.124)0.233\ (0.124)0.133 (0.011)0.133\ (0.011)0.236 (0.118)0.236\ (0.118)0.133 (0.011)0.133\ (0.011)0.236 (0.128)0.236\ (0.128)
β112(t)\beta_{112}(t)0.084 (0.005)0.084\ (0.005)0.089 (0.005)0.089\ (0.005)0.090 (0.006)0.090\ (0.006)0.199 (0.137)0.199\ (0.137)0.090 (0.006)0.090\ (0.006)0.202 (0.144)0.202\ (0.144)0.090 (0.006)0.090\ (0.006)0.200 (0.139)0.200\ (0.139)
β113(t)\beta_{113}(t)0.017 (0.001)0.017\ (0.001)0.017 (0.001)0.017\ (0.001)0.018 (0.001)0.018\ (0.001)0.154 (0.133)0.154\ (0.133)0.018 (0.001)0.018\ (0.001)0.154 (0.133)0.154\ (0.133)0.018 (0.001)0.018\ (0.001)0.156 (0.136)0.156\ (0.136)
β121(t)\beta_{121}(t)0.171 (0.014)0.171\ (0.014)0.181 (0.013)0.181\ (0.013)0.182 (0.015)0.182\ (0.015)0.267 (0.119)0.267\ (0.119)0.182 (0.015)0.182\ (0.015)0.278 (0.114)0.278\ (0.114)0.183 (0.015)0.183\ (0.015)0.267 (0.122)0.267\ (0.122)
β122(t)\beta_{122}(t)0.116 (0.008)0.116\ (0.008)0.122 (0.008)0.122\ (0.008)0.123 (0.009)0.123\ (0.009)0.221 (0.141)0.221\ (0.141)0.123 (0.009)0.123\ (0.009)0.228 (0.149)0.228\ (0.149)0.123 (0.009)0.123\ (0.009)0.226 (0.143)0.226\ (0.143)
β123(t)\beta_{123}(t)0.024 (0.001)0.024\ (0.001)0.025 (0.001)0.025\ (0.001)0.025 (0.001)0.025\ (0.001)0.155 (0.133)0.155\ (0.133)0.025 (0.001)0.025\ (0.001)0.155 (0.134)0.155\ (0.134)0.025 (0.001)0.025\ (0.001)0.157 (0.137)0.157\ (0.137)
β211(t)\beta_{211}(t)0.137 (0.015)0.137\ (0.015)0.146 (0.018)0.146\ (0.018)0.147 (0.016)0.147\ (0.016)0.240 (0.139)0.240\ (0.139)0.147 (0.016)0.147\ (0.016)0.232 (0.128)0.232\ (0.128)0.146 (0.016)0.146\ (0.016)0.236 (0.139)0.236\ (0.139)
β212(t)\beta_{212}(t)0.096 (0.010)0.096\ (0.010)0.102 (0.010)0.102\ (0.010)0.103 (0.010)0.103\ (0.010)0.190 (0.128)0.190\ (0.128)0.102 (0.010)0.102\ (0.010)0.196 (0.137)0.196\ (0.137)0.102 (0.010)0.102\ (0.010)0.193 (0.129)0.193\ (0.129)
β213(t)\beta_{213}(t)0.017 (0.002)0.017\ (0.002)0.018 (0.001)0.018\ (0.001)0.018 (0.001)0.018\ (0.001)0.152 (0.131)0.152\ (0.131)0.018 (0.001)0.018\ (0.001)0.152 (0.132)0.152\ (0.132)0.018 (0.001)0.018\ (0.001)0.153 (0.135)0.153\ (0.135)
β221(t)\beta_{221}(t)0.177 (0.016)0.177\ (0.016)0.187 (0.017)0.187\ (0.017)0.188 (0.015)0.188\ (0.015)0.278 (0.133)0.278\ (0.133)0.188 (0.015)0.188\ (0.015)0.268 (0.122)0.268\ (0.122)0.188 (0.014)0.188\ (0.014)0.267 (0.136)0.267\ (0.136)
β222(t)\beta_{222}(t)0.132 (0.006)0.132\ (0.006)0.137 (0.006)0.137\ (0.006)0.139 (0.006)0.139\ (0.006)0.218 (0.132)0.218\ (0.132)0.138 (0.006)0.138\ (0.006)0.221 (0.142)0.221\ (0.142)0.138 (0.005)0.138\ (0.005)0.213 (0.131)0.213\ (0.131)
β223(t)\beta_{223}(t)0.024 (0.002)0.024\ (0.002)0.024 (0.002)0.024\ (0.002)0.025 (0.002)0.025\ (0.002)0.153 (0.131)0.153\ (0.131)0.025 (0.002)0.025\ (0.002)0.153 (0.132)0.153\ (0.132)0.025 (0.002)0.025\ (0.002)0.154 (0.134)0.154\ (0.134)
RCI//200/200193/200200/200194/200198/200195/200
IIC100.00%95.70%95.61%65.88%95.61%65.70%95.61%65.08%
Test ACC/90.65%90.56%81.78%90.55%81.78%90.59%81.01%
Brier/0.0670.0680.1910.0680.1920.0680.194
NLP/0.2190.2210.5680.2210.5690.2210.573
RMSEp_p/0.0280.0340.3460.0340.3470.0340.350

Table 2. Simulation results for n=800n = 800 (200 replications per method). Note. Parameter entries are RMSE (bias). Scalar bias is signed; functional bias is the root integrated squared bias. RCI uses all 200 replications. Parameter errors, IIC, and test metrics use method-specific fits selecting two subgroups, irrespective of numerical convergence. The oracle uses known training labels. HLME fixes two groups and the cosine and sine random-effect functions; unavailable quantities are marked “/”. Test prediction uses pretreatment covariates only. Proposed: λ=0.02\lambda= 0.02, ∣η∣≤5\lvert\eta\rvert\le5; two-step: λ=0.04\lambda= 0.04.

Applications to clinical trial data

COMPETE migraine trial

We illustrate the proposed mixture-of-experts model using the COMPETE randomized clinical trial of migraine prevention in patients with patent foramen ovale. COMPETE was a multicenter, randomized, active-controlled study of antithrombotic agents versus metoprolol [12]. We compare rivaroxaban with metoprolol, the active control. The complete-case analysis included 469 participants, of whom 236 received rivaroxaban and 233 received metoprolol. The primary trial report evaluated the 12-week responder outcome [13].

The longitudinal outcome was the monthly number of migraine days, transformed as log⁡(1+Yij)\log(1+Y_{ij}), and measured at baseline and weeks 4, 8, and 12. Follow-up time was rescaled to [0,1][0,1]. We set Di=1D_i=1 for rivaroxaban and Di=0D_i=0 for metoprolol. The standardized baseline migraine-attack frequency was included as a prognostic covariate. The longitudinal model was

log⁡{1+MigraineDaysij}=∑g=1Gδig{βg1+Diβg2(tij)+βg3MigraineAttacksi+Vigϕg(tij)+ϵigj}.\log\{1+\mathrm{MigraineDays}_{ij}\}=\sum_{g=1}^{G}\delta_{ig}\{\beta_{g1}+D_i\beta_{g2}(t_{ij})+\beta_{g3}\mathrm{MigraineAttacks}_i+V_{ig}\phi_g(t_{ij})+\epsilon_{igj}\}.

In this model, βg2(t)\beta_{g2}(t) is the subgroup-specific treatment-effect function, approximated by cubic B-splines. The term Vigϕg(t)V_{ig}\phi_g(t) represents the leading functional principal component of within-subject variation.

We used two standardized pretreatment domains from the Migraine-Specific Quality of Life Questionnaire to model subgroup membership. The Role Restrictive (MSQ-RR) and Emotional Function (MSQ-EF) domains measure limitations in daily activities and emotional functioning, respectively. Let Zi=(Zi,RR,Zi,EF)T\mathbf{Z}_i=(Z_{i,\mathrm{RR}},Z_{i,\mathrm{EF}})^{\mathrm{T}}, without an intercept. The gating model was

πig=ωgexp⁡(ZiTηg)∑h=1Gmax⁡ωhexp⁡(ZiTηh),g=1,…,Gmax⁡,\pi_{ig}=\frac{\omega_g\exp(\mathbf{Z}_i^{\mathrm{T}}\boldsymbol{\eta}_g)}{\sum_{h=1}^{G_{\max}}\omega_h\exp(\mathbf{Z}_i^{\mathrm{T}}\boldsymbol{\eta}_h)},\qquad g=1,\ldots,G_{\max},

where η1=0\boldsymbol{\eta}_1=0. We initialized Gmax⁡=5G_{\max}=5 candidate components and set λ=0.01\lambda=0.01. The fitted model retained two subgroups. Table 3 reports the parameter estimates and bootstrap summaries.

ParameterSubgroup 1: EstimateSubgroup 1: SDSubgroup 1: 95% CISubgroup 2: EstimateSubgroup 2: SDSubgroup 2: 95% CI
Longitudinal and covariance parameters
ωg\omega_g0.3650.108(0.191, 0.616)0.6350.108(0.384, 0.809)
βg1\beta_{g1}1.9880.071(1.890, 2.168)2.1760.042(2.073, 2.217)
βg3\beta_{g3}0.3220.088(0.188, 0.533)0.7130.058(0.606, 0.832)
σg2\sigma_g^20.2330.034(0.147, 0.279)0.0290.007(0.012, 0.040)
λg\lambda_g0.5480.072(0.434, 0.736)0.5040.056(0.413, 0.630)
Gating coefficients relative to subgroup 1
CoefficientEstimateSD95% CI
η21\eta_{21} (MSQ-RR)0.4460.232(0.024, 0.936)
η22\eta_{22} (MSQ-EF)-0.2680.225(−0.723, 0.157)

Table 3. Parameter estimates for the two identified subgroups in the COMPETE migraine trial. Note. Point estimates use the full sample. Longitudinal and covariance SDs and CIs use 200 nonstratified patient-level warm-start bootstrap replications. Gating SDs and CIs use 200 patient-level bootstrap replications stratified by the original posterior-modal subgroups (162 and 307 patients); all 200 fits converged. The gating intervals are bootstrap means ±1.96 bootstrap SDs. Gating coefficients are log-odds coefficients for subgroup 2 relative to subgroup 1 per one-SD increase in the standardized MSQ score.

The fitted global weights were ω^1=0.365\widehat{\omega}_1=0.365 and ω^2=0.635\widehat{\omega}_2=0.635. Relative to subgroup 1, the estimated subgroup 2 gating coefficients were 0.446 for MSQ-RR and −0.268-0.268 for MSQ-EF. Their bootstrap SDs were 0.232 and 0.225, respectively. The corresponding bootstrap-mean-centered normal 95% intervals were (0.024, 0.936) and (−0.723-0.723, 0.157). Thus, higher MSQ-RR was associated with greater odds of subgroup 2 membership, whereas the association with MSQ-EF was less precisely estimated.

Figure 3(a) shows two distinct temporal treatment-effect patterns. Subgroup 2 exhibited an early-plateau pattern, with estimated effects of −0.68-0.68, −0.79-0.79, and −0.88-0.88 at weeks 4, 8, and 12. Subgroup 1 exhibited a delayed-strengthening pattern, with corresponding estimates of −0.27-0.27, −0.99-0.99, and −1.37-1.37. The estimated curves crossed during follow-up, showing that their ordering changed over time.

Estimated treatment-effect trajectories and leading eigenfunctions for the COMPETE migraine trial

Figure 3. Estimated treatment-effect trajectories and leading eigenfunctions for the COMPETE migraine trial. Panel (a) shows βg2(t)\beta_{g2}(t) for subgroup 1 (red, delayed-strengthening pattern) and subgroup 2 (blue, early-plateau pattern). Panels (b) and (c) show ϕ1(t)\phi_{1}(t) and ϕ2(t)\phi_{2}(t), respectively. Curves are fitted estimates; shaded regions are pointwise 95% confidence intervals based on 200 patient-level warm-start bootstrap resamples. Follow-up is expressed in weeks. Negative treatment-effect values indicate a lower transformed migraine-day outcome under rivaroxaban than under metoprolol.

Because the outcome was measured on the log⁡(1+migraine days)\log(1+\text{migraine days}) scale, negative estimates indicate a lower conditional mean under rivaroxaban than under metoprolol. The estimated heterogeneity therefore concerned both the magnitude and timing of the treatment contrast. A population-average curve would not distinguish an early response approaching a plateau from a contrast that strengthened later.

Figure 3(b,Figure 3c) shows the estimated leading eigenfunctions. Their distinct shapes describe subgroup-specific patterns of within-subject temporal variation beyond the estimated mean treatment effects.

For a new patient, the fitted gating model gives the subgroup 2 probability

π^2(Z∗)=0.635exp⁡{0.446Z∗,RR−0.268Z∗,EF}0.365+0.635exp⁡{0.446Z∗,RR−0.268Z∗,EF}.\widehat{\pi}_2(Z_*) = \frac{0.635\exp\{0.446Z_{*,\mathrm{RR}} - 0.268Z_{*,\mathrm{EF}}\}}{0.365 + 0.635\exp\{0.446Z_{*,\mathrm{RR}} - 0.268Z_{*,\mathrm{EF}}\}}.

The scores Z∗,RRZ_{*,\mathrm{RR}} and Z∗,EFZ_{*,\mathrm{EF}} are standardized using the analysis-sample means and standard deviations, and π^1(Z∗)=1−π^2(Z∗)\widehat{\pi}_1(Z_*) = 1 - \widehat{\pi}_2(Z_*). The subgroup with the larger probability gives a hard prediction, whereas retaining both probabilities quantifies assignment uncertainty.

SANAD epilepsy study

The Standard and New Antiepileptic Drugs (SANAD) study was a randomized trial of standard and newer antiepileptic drugs [19 b]. Following Wei et al. [27], we analyzed patients receiving lamotrigine (LTG) or carbamazepine (CBZ). The outcome was the log-transformed Liverpool Adverse Events Profile (AEP), for which higher scores indicate a greater burden of adverse effects. The complete-case dataset contained 1,720 observations from 540 patients, including 262 assigned to LTG and 278 assigned to CBZ. Observation times were divided by the maximum observed follow-up time, giving tij∈[0,1]t_{ij} \in[0,1].

For each subject, the first recorded anxiety (ANX) and depression (DEP) scores were standardized and used as subject-level predictors. The model included constant prognostic coefficients and a time-varying treatment effect. Conditional on Gi=gG_i=g, the fitted longitudinal model was

log⁡(AEPij)=βg1+Diβg2(tij)+βg3ANXi+βg4DEPi+Vigϕg(tij)+ϵigj,\log(\mathrm{AEP}_{ij})=\beta_{g1}+D_i\beta_{g2}(t_{ij})+\beta_{g3}\mathrm{ANX}_i+\beta_{g4}\mathrm{DEP}_i+V_{ig}\phi_g(t_{ij})+\epsilon_{igj},

where Di=1D_i=1 for LTG and Di=0D_i=0 for CBZ. The same ANX and DEP scores entered the gating model in (2). We used cubic B-splines with interior knots at 0.153 and 0.312 and set K=1K=1. Starting from Gmax⁡=5G_{\max}=5 with λ=0.03\lambda=0.03, the algorithm converged to two subgroups after 25 iterations.

Table 4 summarizes the fitted parameters. The global weights were ω1=0.292\omega_1=0.292 and ω2=0.708\omega_2=0.708. These are membership probabilities at the centered predictor profile, not marginal subgroup proportions. ANX and DEP had positive associations with log-AEP in both subgroups, with larger estimates in subgroup 2. The estimated latent-process and measurement-error variances were also larger in subgroup 2.

ParameterEstimateSD95% CIEstimateSD95% CI
ωg\omega_g0.2920.105(0.087, 0.497)0.7080.105(0.503, 0.913)
βg1\beta_{g1}3.8490.043(3.766, 3.933)3.6410.041(3.561, 3.721)
βg3\beta_{g3} (ANX)0.0490.020(0.010, 0.088)0.1270.021(0.086, 0.169)
βg4\beta_{g4} (DEP)0.0450.014(0.018, 0.073)0.0970.022(0.055, 0.139)
σg2\sigma_g^20.0060.003(0.000, 0.012)0.0250.003(0.018, 0.031)
λg\lambda_g0.0120.006(0.001, 0.023)0.0430.007(0.029, 0.056)
Gating coefficients: subgroup 2 relative to subgroup 1
CoefficientEstimateSD95% CI
η21\eta_{21} (ANX)-0.9930.473(−1.920-1.920, −0.066-0.066)
η22\eta_{22} (DEP)-0.4330.330(−1.079-1.079, 0.213)

Table 4. Parameter estimates for the two identified subgroups in the SANAD epilepsy study.

Of 200 subject-level bootstrap replications, 186 selected two subgroups and were used for interval estimation; seven of these did not meet the numerical convergence criterion. Scalar intervals used a normal approximation, whereas function intervals used pointwise percentiles. Before summarization, subgroup labels were aligned with the fitted model and gating coefficients were recentered to the same reference subgroup.

Note. ANX and DEP are standardized first-recorded subject-level scores. SDs and normal-approximation 95% CIs use the 186 bootstrap replications (of 200) that selected two subgroups; 179 met the numerical convergence criterion; the other seven fits are retained. Gating coefficients are recentered to subgroup 1 after label alignment.

Figure 4(a) shows the subgroup-specific treatment contrasts. The estimated β12(t)\beta_{12}(t) remained close to zero, and its pointwise interval included zero throughout follow-up. In subgroup 2, the estimated contrast was negative during the middle of follow-up, returned toward zero, and decreased again near the end. Its pointwise interval lay below zero over approximately [0.25, 0.43] and [0.87, 1.00] on the normalized time scale. Negative contrasts indicate a lower conditional mean log-AEP under LTG than under CBZ. The late estimates were less precise, as indicated by the widening intervals. Panels (b) and (c) show the corresponding eigenfunctions. Subgroup 1 had a late upswing, whereas subgroup 2 was comparatively stable during most of the later follow-up.

Estimated treatment-effect trajectories and leading eigenfunctions

Figure 4. Estimated treatment-effect trajectories and leading eigenfunctions for the SANAD epilepsy study. Panel (a) displays β12(t)\beta_{12}(t) (upper, red) and β22(t)\beta_{22}(t) (lower, blue) with separate vertical scales. Panels (b) and (c) display ϕ1(t)\phi_1(t) and ϕ2(t)\phi_2(t). Curves are fitted estimates, and shaded regions are pointwise 95% bootstrap percentile intervals based on the 186 replications (of 200) that selected two subgroups. Eigenfunction signs are aligned to the fitted functions. Time is normalized to [0,1][0,1].

The gating estimates for subgroup 2 relative to subgroup 1 were η^21=−0.993\hat{\eta}_{21}=-0.993 for ANX and η^22=−0.433\hat{\eta}_{22}=-0.433 for DEP. The reference-aligned 95% intervals were (−1.920,−0.066)(-1.920,-0.066) and (−1.079,0.213)(-1.079,0.213), respectively. Higher ANX was therefore associated with lower odds of subgroup 2 membership. The association with DEP was less precisely estimated. Although the fitted gating model uses these two scores alone, their pretreatment availability has not been established. Using the fitted model at treatment assignment would require confirming that both measurements precede treatment.

Discussion

In this article, we proposed a data-adaptive semiparametric mixture-of-experts model that unifies longitudinal treatment-effect subgroup discovery and pretreatment membership prediction. Unlike the two-step strategy, the pretreatment membership model enters the observed-data likelihood and directly contributes to subgroup discovery. Separate global component weights and covariate-dependent gating coefficients allow redundant components to be removed while retaining patient-specific membership probabilities. The model also accommodates subgroup-specific treatment-effect trajectories and covariance functions. Under regularity conditions, it consistently selects the subgroup number, with established convergence rates and asymptotic normality. Across candidate upper bounds, simulations showed reliable subgroup-number selection and generally more accurate recovery of the longitudinal components and membership probabilities than the two-step procedure. These comparisons were conditional on selecting two subgroups; the correctly specified HLME benchmark, supplied with the true subgroup number and covariance shapes, performed better for some parameters.

The applications illustrate the value of studying temporal treatment-effect heterogeneity together with pretreatment membership prediction. In the SANAD study, the comparison between lamotrigine and carbamazepine was close to zero in one subgroup but favored lamotrigine during specific periods in the other. In the COMPETE trial, the analysis distinguished an early-plateau pattern from a delayed-strengthening pattern in the comparison of rivaroxaban with metoprolol. Such temporal differences would be obscured by a population-average trajectory. The COMPETE analysis also translated pretreatment characteristics into membership probabilities for new patients. Retaining these probabilities instead of a hard assignment preserves uncertainty when subgroup separation is incomplete. These clinical findings remain exploratory and require independent validation.

Several limitations remain. Because the latent subgroups are defined by the full longitudinal distribution, they may differ in prognosis or within-subject covariance as well as in treatment response and need not represent distinct treatment-sensitivity phenotypes. The working model assumes Gaussian component outcomes, a finite-rank covariance representation, smooth coefficient functions, and a multinomial-logit gating model. The penalized mixture objective is nonconvex, and both applications used complete-case data. Future work could accommodate sparse or nonlinear gating effects, non-Gaussian or multivariate longitudinal outcomes, informative dropout, and subgroup-selection uncertainty. External validation is essential because prediction requires the learned subgroup structure to transport to the target population. Finally, accurate membership prediction does not establish an optimal treatment rule; decision-oriented methods must also account for membership uncertainty and the consequences of incorrect treatment assignment.

References

  1. [1]Robert B. Ash and Melvyn F. Gardner. Topics in Stochastic Processes: Probability and Mathematical Statistics: A Series of Monographs and Textbooks, volume 27. Academic press, 2014.
  2. [2]Kani Chen and Xingwei Tong. Varying coefficient transformation models with censored data. Biometrika, 97(4):969–976, 2010.DOI
  3. [3]Lu Chen, Xuerong Chen, Xinzhou Guo, and Yi Li. Subgroup identification and membership prediction. Biometrics, 82(3):ujag122, 2026. doi: 10.1093/biomtc/ujag122.DOI
  4. [5]Xinyi Ge, Yingwei Peng, and Dongsheng Tu. A threshold linear mixed model for identification of treatment-sensitive subsets in a clinical trial based on longitudinal outcomes and a continuous covariate. Statistical Methods in Medical Research, 29(10):2919–2931, 2020. doi: 10.1177/0962280220912772.DOI
  5. [6]Xinyi Ge, Yingwei Peng, and Dongsheng Tu. A threshold mixed-effects Tobit model for treatment-sensitive subgroup identification based on longitudinal measures with floor and ceiling effects and a continuous covariate. Journal of Statistical Computation and Simulation, 94(11):2544–2563, 2024. doi: 10.1080/00949655.2024.2344126.DOI
  6. [7]Robert D. Gibbons, Donald Hedeker, and Stephen DuToit. Advances in analysis of longitudinal data. Annual Review of Clinical Psychology, 6:79–107, 2010. doi: 10.1146/annurev.clinpsy.032408.153550.DOI
  7. [8]Peter Hall and Joel L Horowitz. Methodology and convergence rates for functional linear regression. The Annals of Statistics, 2007.
  8. [9]Tao Huang, Heng Peng, and Kun Zhang. Model selection for Gaussian mixture models. Statistica Sinica, 27(1):147–169, 2017. doi: 10.5705/ss.2014.105.DOI
  9. [10]Gareth M James, Trevor J Hastie, and Catherine A Sugar. Principal component models for sparse functional data. Biometrika, 87(3):587–602, 2000.DOI
  10. [11]David M. Kent, Ewout Steyerberg, and David van Klaveren. Personalized evidence based medicine: predictive approaches to heterogeneous treatment effects. BMJ, 363:k4245, 2018. doi: 10.1136/bmj.k4245.DOI
  11. [12]Ziping Li and colleagues. Study design and rationale of compete: Comparison of the effect of medication therapy in alleviating migraine with patent foramen ovale. American Heart Journal, 269:1–7, 2024. doi: 10.1016/j.ahj.2023.12.011.DOI
  12. [13]Ziping Li and colleagues. Antithrombotic treatment for migraine in patients with patent foramen ovale: multicentre, randomised, active controlled, open label trial. The BMJ, 394:e100103, 2026. doi: 10.1136/bmj-2026-100103.DOI
  13. [14]Haiqun Lin, Charles E. McCulloch, Bruce W. Turnbull, Elizabeth H. Slate, and Larry C. Clark. A latent class mixed model for analysing biomarker trajectories with irregularly scheduled observations. Statistics in Medicine, 19(10):1303–1318, 2000. doi: 10.1002/(SICI)1097-0258(20000530)19:10<1303::AID-SIM424>3.0.CO;2-E.doi.org/10.1002/(sici)1097-0258(20000530)19:10<1303::aid-sim424>3.0.co;2-e
  14. [15]Ilya Lipkovich, Alex Dmitrienko, and Ralph B. D’Agostino, Sr. Tutorial in biostatistics: data-driven subgroup identification and analysis in clinical trials. Statistics in Medicine, 36(1):136–196, 2017. doi: 10.1002/sim.7064.DOI
  15. [16]Wei-Yin Loh, Haoda Fu, Michael Man, Victoria Champion, and Menggang Yu. Identification of subgroups with differential treatment effects for longitudinal and multiresponse variables. Statistics in Medicine, 35(26):4837–4855, 2016. doi: 10.1002/sim.7020.DOI
  16. [17]Minggen Lu, Ying Zhang, and Jian Huang. Semiparametric estimation methods for panel count data using monotone b-splines. Journal of the American Statistical Association, 104(487):1060–1070, 2009.
  17. [18]Ling Ma, Tao Hu, and Jianguo Sun. Sieve maximum likelihood regression analysis of dependent current status data. Biometrika, 102(3):731–738, 2015.DOI
  18. [19]AG Marson, R Appleton, GA Baker, DW Chadwick, J Doughty, B Eaton, C Gamble, A Jacoby, P Shackley, DF Smith, et al. A randomised controlled trial examining the longer-term out- comes of standard versus new antiepileptic drugs. the sanad trial. HEALTH TECHNOLOGY ASSESSMENT-SOUTHAMPTON-, 11(37), 2007a.pubmed.ncbi.nlm.nih.gov/17903391
  19. [21]Cécile Proust-Lima, Viviane Philipps, and Benoit Liquet. Estimation of extended mixed models using latent classes and latent processes: The R package lcmm. Journal of Statistical Software, 78(2):1–56, 2017. doi: 10.18637/jss.v078.i02.
  20. [22]Patrick Schober and Thomas R. Vetter. Repeated measures designs and analysis of longitudinal data: If at first you do not succeed—try, try again. Anesthesia & Analgesia, 127(2):569–575, 2018. doi: 10.1213/ANE.00000000000003511.DOI
  21. [23]Larry Schumaker. Spline functions: basic theory. Cambridge university press, 2007.
  22. [24]Juan Shen and Xuming He. Inference for subgroup analysis with a structured logistic-normal mixture model. Journal of the American Statistical Association, 110(509):303–312, 2015. doi: 10.1080/01621459.2014.894763.DOI
  23. [25]Juan Shen and Annie Qu. Subgroup analysis based on structured mixed-effects models for longitudinal data. Journal of Biopharmaceutical Statistics, 30(4):607–622, 2020. doi: 10.1080/10543406.2020. 1730867.DOI
  24. [26]Kecheng Wei, Huichen Zhu, Guoyou Qin, Zhongyi Zhu, and Dongsheng Tu. Multiply robust subgroup analysis based on a single-index threshold linear marginal model for longitudinal data with dropouts. Statistics in Medicine, 41(15):2822–2839, 2022. doi: 10.1002/sim.9386.DOI
  25. [27]Kecheng Wei, Guoyou Qin, and Zhongyi Zhu. Subgroup analysis for longitudinal data based on a partial linear varying coefficient model with a change plane. Statistics in Medicine, 42(20): 3716–3731, 2023. doi: 10.1002/sim.9827.DOI
  26. [28]Yishu Wei, Lei Liu, Xiaogang Su, Lihui Zhao, and Hongmei Jiang. Precision medicine: Subgroup identification in longitudinal trajectories. Statistical Methods in Medical Research, 29(9):2603–2616, 2020. doi: 10.1177/0962280220904114.DOI
  27. [29]Chun-Che Wen, Nathaniel Baker, Rajib Paul, Elizabeth Hill, Kelly Hunt, Hong Li, Kevin Gray, and Brian Neelon. A Bayesian beta-binomial piecewise growth mixture model for longitudinal overdispersed nominal data. Statistical Methods in Medical Research, 33(10):1859–1876, 2024. doi: 10.1177/09622802241279109.DOI
  28. [30]Weichun Xu and Donald Hedeker. A random-effects mixture model for classifying treatment response in longitudinal clinical trials. Journal of Biopharmaceutical Statistics, 11(4):253–273, 2001. doi: 10.1081/BIP-120008848.DOI
  29. [31]Zhichen Xu, Jimin Ding, Xiaogang Su, Guoqiao Wang, Ke Xie, Yining Zhao, Chengjie Xiong, Eric McDade, Lon S. Schneider, and Lei Liu. Subgroup identification via interaction tree and mixed model for repeated measures with application to Alzheimer’s disease. Biometrics, 82(2):ujag104, 2026. doi: 10.1093/biomtc/ujag104.DOI
  30. [32]Pu Zhang, Xinsheng Zhang, Jiao Jin, and Jiakun Jiang. Varying treatment effects in subgroups: A unified framework for longitudinal data analysis. Statistics in Medicine, 45(10–12):e70600, 2026. doi: 10.1002/sim.70600.DOI
  31. [33]Qingzhi Zhong and Xinyuan Song. Sparse functional varying-coefficient mixture regression. Journal of Multivariate Analysis, 206:105383, 2025. doi: 10.1016/j.jmva.2024.105383.DOI
  32. [34]Xiaolu Zhu and Annie Qu. Cluster analysis of longitudinal profiles with subgroups. Electronic Journal of Statistics, 12(1):171–193, 2018. doi: 10.1214/17-EJS1389.DOI

Paper details

vlnc:W_8tyfzxhv
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC0-1.0
Usage

views · 8

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents