betaFeedback

WorksComputer Science

InertiaMPDD: Multimodal Personalized Depression Detection via Emotional Inertia-Inspired Dynamics Modeling

Abstract

Multimodal depression screening estimates risk from speech, visual behavior and language. Current methods mainly predict labels, with limited evidence of departures from a non-depressed reference. Recording-level supervision also leaves fine-grained emotional dynamics unannotated. Motivated by emotional inertia, the persistence of affect, InertiaMPDD learns healthy next-state patterns using a world-model-inspired latent predictor. Using pretrained multimodal representations, it predicts from observed history and context. Reference deviation, residual decay rate and a Jacobian spectral descriptor form reference-derived emotional dynamics descriptors, evaluated as candidate proxy biomarkers for standalone screening and personalized decision correction. Across five cohorts from four datasets, fusion reaches 98.41% F1 on D-Vlog and 84.98% on LMVD. Standalone descriptors outperform fixed additive fusion on MPDD but discriminate weakly on AVEC2014. These findings distinguish reference informativeness from fusion benefit and reveal context-dependent utility.

Introduction

Multimodal depression detection uses language, speech and facial behavior to estimate risk from long recordings. Recent sequence models and multimodal large models learn increasingly effective relationships between these observations and depression labels [1, 2, 3, 4]. Some also generate symptom-based rationales [5, 6]. Affective science offers a complementary temporal perspective. Emotional inertia describes the tendency of affective states to persist over time, commonly studied through their lagged autocorrelation and associated with depression [7, 8, 9]. This motivates looking for measurable dynamical evidence alongside a recording-level risk prediction.

Two limitations make this evidence difficult to obtain. Coarse supervision leaves emotional dynamics unobserved: depression datasets commonly assign one label to an entire recording or participant [10, 11, 12], without identifying the informative emotional segments. Emotion benchmarks such as IEMOCAP instead annotate individual utterances [13]. Comparable supervision for directly tracking emotional persistence is usually unavailable in depression recordings. A population prediction also lacks an explicit behavioral reference. Slow speech or limited facial movement may be habitual for one speaker and unusual for another. A useful comparison therefore needs to accommodate context while remaining learnable without segment-level emotion or personality labels (Figure 1).

Image not extracted

Figure 1. Motivation. (a) Existing Multimodal Approaches. (b) Our Emotion Dynamics Trait Analysis.

Motivated by these limitations, we ask **whether a pretrained multimodal model can provide measurable evidence of behavioral departures from a non-depressed reference, and whether this evidence can support depression

screening. World models and latent predictive architectures learn representations and predict how observations evolve [14, 15, 16, 17]. We use this principle to learn a healthy next-state predictor from frozen pretrained representations. It predicts from observed history and context. Comparison with the next observation gives a time-resolved residual. Inspired by emotional inertia, we examine reference deviation, residual persistence and the predictor’s local spectral structure. We call these reference-derived emotional dynamics descriptors: proxies computed from behavioral representations, rather than direct measurements of annotated emotions. Their association with depression labels tests their value as candidate proxy biomarkers. Experiments across recording settings assess their screening utility and applicability.

We address this question with InertiaMPDD. Our contributions are:

  • We formulate reference-based behavioral evidence under recording-level depression supervision, using emotional inertia to motivate measurable descriptions of predictive departure and persistence.

  • We introduce the Idiographic Inertia Pathway (IIP), a world-model-inspired healthy predictor that extracts reference deviation, residual decay rate and a Jacobian spectral descriptor. These descriptors support standalone screening and an individual correction to the multimodal Nomothetic Screening Pathway (NSP).

  • We evaluate the pathways on five cohorts from four datasets, finding complementary evidence on spontaneous vlogs, stronger standalone IIP than fixed fusion on MPDD and limited reference discrimination on AVEC2014 [10, 11, 12, 18]. Feature controls and visualizations distinguish reference informativeness from fusion benefit.

Related Work

A predictive reference complements population screening

Depression screening has progressed from hand-crafted paralinguistic features to multimodal sequence models. DAIC-WOZ and AVEC established interview-based benchmarks [18, 19]. Recent methods model temporal dependencies, cross-modal interactions and individual context. Examples include MMFormer [1], CAF-Mamba [20], DynMultiDep [2], ASYM [21], AttMamba [22] and personality-aware PAVNet [23]. DepressionAgent provides symptom-level explanations through multimodal reasoning [5]. IIP adds a predictive reference against which the observed continuation can be compared. This supplies a time-resolved departure signal alongside the population decision.

Personality-aware benchmark design includes the MPDD challenge [12] and MPDD-AVG, which combines audio-visual interviews with gait analysis [24]. Physiological measurements offer a complementary route: M3DEPM^{3}\mathrm{DEP} organizes multimodal, multitask and multievent depression assessment [25], while M3DEP studies individual differences in EEG-EMG signals [26]. These studies broaden the modalities and individual factors considered in screening. IIP examines recording-dependent variation through a healthy predictive reference in pretrained behavioral representations.

Prediction-based anomaly detection also learns regular behavior and measures future-prediction error [27]. Our question is whether a healthy-trained reference, conditioned on context, yields useful descriptors for depression screening. Counterfactual approaches address related questions about spurious associations, such as gender in acoustic prediction [28]. Here the comparison is operationally between a predicted next target state and the next observation in the same recording.

World-model-inspired prediction supplies a learnable reference

World models learn representations and predict how observations evolve [14]. Joint-embedding predictive architectures apply prediction in representation space [15, 16]. V-JEPA 2 combines action-free predictive pretraining with action-conditioned learning for planning [17]. IIP adapts latent prediction to a healthy behavioral reference for screening, with variance and covariance regularization [29]. The evaluation concerns reference-derived descriptors rather than free-running simulation or planning.

Emotional inertia motivates residual-persistence analysis

Emotional inertia concerns the persistence of affect from one moment to the next, commonly described by lagged autocorrelation. It has been associated with depression [7, 8], and meta-analytic work connects affective dynamics with psychological well-being [9]. Critical slowing down provides a related motivation for examining persistence and recovery around depressive transitions [30]. These findings motivate our residual AR analysis. Its measured sequence is prediction disagreement in learned audiovisual coordinates. We therefore distinguish the affective construct from its computational proxy: the AR analysis describes persistence in prediction disagreement, without assigning that disagreement an emotional valence.

Recovery and reactivity additionally require an identified baseline or eliciting event, neither of which is directly measured here. Context conditioning also does not by itself identify stable psychological traits separately from momentary states [31, 32].

Method

Problem statement

We observe recordings Xi=(Ai,Vi,Li)\mathcal{X}_i = (A_i, V_i, L_i) containing audio, video and transcript, with recording-level depression labels yi∈{0,1}y_i \in\{0, 1\}. Let D={(Xi,yi)}i=1N\mathcal{D} = \{(\mathcal{X}_i, y_i)\}_{i=1}^{N} denote labeled recordings and Dtrain\mathcal{D}_{\mathrm{train}} their training subset. Segment-level emotion labels are unavailable. Our goal is to predict risk while identifying measurable departures from a non-depressed behavioral reference. NSP maps the multimodal recording to a population margin mi=Mϕ(Xi)m_i = M_{\phi}(\mathcal{X}_i). IIP maps its frozen feature sequence x1:Tix_{1:T_i} to a reference-derived descriptor vector ui=U(x1:Ti)u_i = U(x_{1:T_i}). A supervised head converts uiu_i into an individual correction bib_i, giving the screening score mi+bim_i + b_i.

To construct UU, we learn next-state prediction from non-depressed training recordings. The online state st=E(xt)s_t = E(x_t) and context zz determine the prediction s^t+1\hat{s}_{t+1}, which is compared with the independently encoded observation s~t+1\tilde{s}_{t+1}. This comparison yields recording-level descriptors without assigning emotions to individual segments. Here zz conditions the reference, whereas the training-standardized descriptor vector uˉ\bar{u} adjusts the decision. Figure 2 shows this process.

Image not extracted

Figure 2. InertiaMPDD framework. (a) Input encoding. (b) NSP population margin and IIP healthy-reference descriptors. (c) Individual correction and screening. Snowflakes mark frozen parameters. Curves and representations are schematic.

Subject-conditioned healthy-reference modeling

The frozen backbone provides 2048-dimensional audio and visual features, aligned at 1 Hz and concatenated into xt∈R4096x_t \in\mathbb{R}^{4096}. The online encoder produces st=E(xt)∈R256s_t = E(x_t) \in\mathbb{R}^{256}, a learned audiovisual representation rather than an emotion annotation. Audio tokens are pooled over one-second intervals. 64 sampled visual frames are expanded to the same temporal grid by nearest-neighbor assignment. This gives an audiovisual state sequence for each recording.

Context is encoded from the temporal mean μx\mu_x and elementwise variance vxv_x of the backbone features. The concatenated statistics give

z=gϕ([μx;vx])∈R16,(1)z = g_{\phi}([\mu_x; v_x]) \in\mathbb{R}^{16}, \tag*{(1)}

where gϕg_{\phi} is a learnable MLP with widths 8192 →\rightarrow 512 →\rightarrow 512 →\rightarrow 16 and SiLU activations. The MLP learns jointly with the healthy predictor and is then frozen. No personality labels are required.

For a predictor activation hh, FiLM applies γ(z)∘h+β(z)\gamma(z) \circ h + \beta(z), initialized with gain one and bias zero. Here s<ts_{<t} supplies the changing state history, while zz supplies context. Both condition the next-state prediction through shared model parameters. For offline screening, causal attention limits the state-history input, while zz summarizes the full recording.

To preserve the feature space used for healthy pretraining, IIP extracts states and conditioning statistics from the unadapted frozen backbone Φ\Phi. NSP reads its decision state from the LoRA-adapted configuration Φϕ\Phi_{\phi}. Thus the healthy reference retains its pretraining feature source during supervised NSP adaptation. The adapted NSP decision state is not reused as an IIP trajectory.

We form the reference pool H={i:(Xi,yi)∈Dtrain,yi=0}\mathcal{H} = \{i : (\mathcal{X}_i,y_i) \in\mathcal{D}_{\mathrm{train}}, y_i = 0\} from the training set’s non-depressed records. Here “healthy” refers to that dataset label. A shared predictor learns the next-state patterns within this pool. There is no per-subject control matching. Our world-model-inspired latent predictor [14, 15, 17] produces s^t+1=fθ(s≤t;z)\hat{s}_{t+1} = f_{\theta}(s_{\leq t};z) from the observed history and context. Independently, the next observation is encoded as s~t+1=E′(xt+1)\tilde{s}_{t+1} = E'(x_{t+1}). We fit the predictor by

min⁡θ,E,ψ EH∥fθ(s≤t;z)−sg⁡(s~t+1)∥22+λRVICReg(E),(2)\min_{\theta,E,\psi}\ \mathbb{E}_{\mathcal{H}}\left\lVert f_{\theta}(s_{\leq t};z)-\operatorname{sg}(\tilde{s}_{t+1})\right\rVert_{2}^{2} +\lambda\mathcal{R}_{\mathrm{VICReg}}(E), \tag*{(2)}

where sg⁡\operatorname{sg} blocks gradients through the target. The semi-orthogonal encoder E′E' stays fixed, so prediction errors are measured against a consistent target space. The squared-error term learns to predict the independently encoded next observation. The variance term ∑jmax⁡(0,1−std⁡(Ej))\sum_j \max(0,1-\operatorname{std}(E_j)) of RVICReg\mathcal{R}_{\mathrm{VICReg}} discourages low feature variance, while its covariance term reduces redundant coordinates [29].

For fixed representations and conditioning, squared-error learning targets the average next state of healthy training records with that history and context. The learned reference estimates this conditional pattern subject to data coverage and model capacity (§ 3.6). It is a prediction along the same observed recording, not an observed healthy trajectory for that person.

A shared predictor accommodates different contexts without fitting a separate model to each person. It provides a common construction for comparing long recordings without segment-level emotion annotations.

The predictor uses three causal Transformer layers, expanded in Figure 2. Within the state-history input, causal attention restricts each position to its available prefix, while zz provides context conditioning. Each prediction is evaluated against the independently encoded next observation. At the following step, the predictor again receives the observed history, rather than recursively feeding back its previous prediction. Thus residual peaks localize one-step disagreement along the recording, without accumulating errors from a free-running rollout.

Reference-derived emotional dynamics descriptors

We describe each recording through three complementary properties: departure from a healthy reference, persistence of residual fluctuations and the predictor’s local spectral structure. These reference-derived emotional dynamics descriptors are recording-level proxies computed from latent behavioral states. Emotional dynamics supplies the theoretical motivation, while the measured quantities remain prediction errors and model properties.

Reference Deviation (RR) measures average disagreement with the healthy predictor. For the observed and predicted next states,

rt=∥s~t+1−s^t+1∥2,r_t = \lVert\tilde{s}_{t+1} - \hat{s}_{t+1} \rVert_2,
R=1T−1∑t=1T−1∥s~t+1−s^t+1∥2.(3)R = \frac{1}{T-1}\sum_{t=1}^{T-1}\lVert\tilde{s}_{t+1} - \hat{s}_{t+1} \rVert_2. \tag*{(3)}

A larger RR means that the observed continuation is less consistent with the reference. Averaging over transitions keeps the score from increasing simply because a recording is longer. Unlike within-person variability or successive-change measures, it compares observations with predictions, rather than with their own mean or adjacent observations. Predictive disagreement can also contain observation uncertainty, as formalized by the bias–noise decomposition in § 3.6. Its association with depression is evaluated by the supervised readout rather than assumed from the residual alone.

Residual Decay Rate (forceAR_{\mathrm{AR}}) characterizes the persistence of fluctuations in prediction disagreement. We center the residuals at their recording median, et=rt−median⁡τrτe_t = r_t - \operatorname{median}_{\tau} r_{\tau}, and fit a no-intercept AR(1) coefficient over adjacent valid pairs:

ϕ^=∑tetet+1∑tet2,a=−ln⁡ϕclip.(4)\hat{\phi} = \frac{\sum_t e_t e_{t+1}}{\sum_t e_t^2}, \qquad a = -\ln\phi_{\mathrm{clip}}. \tag*{(4)}

We set ϕclip=clip⁡(ϕ^,0.01,0.999)\phi_{\mathrm{clip}} = \operatorname{clip}(\hat{\phi}, 0.01, 0.999). The resulting decay rate is regularized toward the healthy-training prior a0a_0:

forceAR=a+2a03.(5)\mathrm{force}_{\mathrm{AR}} = \frac{a + 2a_0}{3}. \tag*{(5)}

An unavailable fit receives a0a_0. Under a stationary AR(1) interpretation, ϕk=exp⁡(−ka)\phi^k = \exp(-ka) describes decay over kk one-second steps. Thus a smaller decay rate indicates stronger residual persistence, or greater stickiness of predictive departure [7, 8]. Unlike RR, it concerns fluctuations around the recording’s residual level. A large but rapidly varying residual need not be persistent. Because the fitted sequence is a residual magnitude, this is an inertia-inspired proxy for predictive-departure persistence, rather than a direct measure of the persistence of positive or negative affect.

Jacobian Spectral Descriptor (ρ(J)\rho(J)) summarizes the healthy predictor’s local mapping in its learned coordinates. With mean online state sˉ=T−1∑t=1Tst\bar{s} = T^{-1}\sum_{t=1}^{T}s_t and fixed context zz, define the single-token map Fz(s)=fθ([s];z)F_z(s) = f_\theta([s];z). Its Jacobian and spectral radius are

J=∂Fz(s)∂s∣s=sˉ,ρ(J)=max⁡j∣λj(J)∣.(6)J = \left.\frac{\partial F_z(s)}{\partial s}\right|_{s=\bar{s}}, \qquad\rho(J) = \max_j \lvert\lambda_j(J) \rvert. \tag*{(6)}

We estimate this property using 30 Jacobian–vector power iterations, avoiding an explicit 256 ×\times 256 matrix. The estimate describes the reference predictor in its fixed online-to-target coordinates. § 3.6 explains the distinction from a common-coordinate relaxation model.

Personalized screening with two pathways

NSP adapts the frozen omni-modal backbone with LoRA [33] and casts the risk judgment as the generation of a label token (Figure 3a). Let hposh_{\mathrm{pos}} be the decision state, the hidden state one step before the label is written, and let W∈Rd×2W \in\mathbb{R}^{d \times2} give the scores of the two risk tokens, represented by the label tokens 0 and 1. The label y∈{0,1}y \in\{0,1\} is supervised by cross-entropy,

Image not extracted

Figure 3. Detailed pathway computations. (a) NSP label-score readout. (b) IIP healthy prediction, residual analysis, Jacobian spectral estimation and descriptor assembly. Tokens, curves and matrix cells are schematic.

LCE=−log⁡p(⟨risky⟩∣hpos),(7)\mathcal{L}_{\mathrm{CE}} = -\log p(\langle\mathrm{risk}_y\rangle\mid h_{\mathrm{pos}}), \tag*{(7)}

the adapter’s only label-supervised path. Let ℓj=hposTWj\ell_j = h_{\mathrm{pos}}^{\mathsf{T}}W_j be the logit of label token jj. The population margin is m=ℓ1−ℓ0=hposT(W1−W0)m = \ell_1 - \ell_0 = h_{\mathrm{pos}}^{\mathsf{T}}(W_1 - W_0). Reading both scores at the same decision position also gives m=log⁡(p1/p0)m = \log(p_1/p_0), since their softmax normalizer cancels. Thus m>0m > 0 favors the positive label and m=0m = 0 marks equal label scores. “Population” refers to the shared readout parameters, not an average across subjects: each recording still produces its own margin. This continuous score retains the relative evidence needed for the additive correction. A thresholded label would discard it.

Algorithm 1 InertiaMPDD: training and inference

Require: Training data DD, healthy pool H\mathcal{H}, frozen backbone Φ\Phi and target encoder E′E'

Ensure: IIP modules (E,gψ,fθ)(E,g_{\psi},f_{\theta}), LoRA ϕ\phi, head (w,c)(w,c)

  1. 1. Learn the healthy reference

  2. for each healthy minibatch (600 updates) do

  3. Extract and align frozen features x1:Tx_{1:T} at 1 Hz

  4. st←E(xt)s_t \leftarrow E(x_t), s~t←E′(xt)\tilde{s}_t \leftarrow E'(x_t)

  5. z←gψ([μx;vx])z \leftarrow g_{\psi}([\mu_x;v_x]), s^t+1←fθ(s≤t;z)\hat{s}_{t+1} \leftarrow f_{\theta}(s_{\leq t};z)

  6. Update (E,gψ,fθ)(E,g_{\psi},f_{\theta}) by Equation (2)

  7. end for

  8. 2. Extract descriptors with frozen IIP modules

  9. Estimate decay prior a0a_0 from healthy training records

  10. for each training record do

  11. Recompute states, context and next-state predictions

  12. Compute rtr_t and mean RR by Equation (3)

  13. Fit AR(1) to rt−median⁡τrτr_t-\operatorname{median}_{\tau}r_{\tau}

  14. Clip φ\varphi to [0.01,0.999][0.01,0.999], set a←−ln⁡φa\leftarrow-\ln\varphi

  15. forceAR⁡←(a+2a0)/3\operatorname{forceAR}\leftarrow(a+2a_0)/3 (or a0a_0 if unavailable)

  16. Estimate ρ(J)\rho(J) at sˉ\bar{s} with fixed zz, 30 JVP steps

  17. u←[R,forceAR⁡,ρ(J)]u\leftarrow[R,\operatorname{forceAR},\rho(J)]

  18. end for

  19. Fit training statistics (μu,σu)(\mu_u,\sigma_u) for u~\tilde{u}

  20. 3. Learn screening and correction

  21. for each labeled minibatch do

  22. Read m=ℓ1−ℓ0m=\ell_1-\ell_0 from adapted backbone Φϕ\Phi_{\phi}

  23. b←wTu~+cb\leftarrow w^{\mathsf{T}}\tilde{u}+c

  24. Minimize Equation (9): CE updates ϕ\phi, BCE (w,c)(w,c)

  25. end for

  26. Inference: extract uu and mm using the learned modules

  27. y^←1[m+wT((u−μu)⊘σu)+c≥0]\hat{y}\leftarrow\mathbb{1}[m+w^{\mathsf{T}}((u-\mu_u)\oslash\sigma_u)+c\geq0]

The population margin becomes personalized by merging it with the three IIP features (Figure 2, right). Writing u=[R,forceAR⁡,ρ(J)]u=[R,\operatorname{forceAR},\rho(J)], we standardize it elementwise as u~=(u−μu)⊘σu\tilde{u}=(u-\mu_u)\oslash\sigma_u using training-set statistics. The head converts the three standardized summaries into one correction. Adding this correction to the population score gives

b=wTu~+c,ℓ=m+b,y^=1[ℓ≥0].(8)b=w^{\mathsf{T}}\tilde{u}+c,\qquad\ell=m+b,\qquad\hat{y}=\mathbb{1}[\ell\geq0]. \tag*{(8)}

To learn the population score and its correction with separate gradient paths, we use

L=LCE+BCE(σ(sg⁡(m)+b),y).(9)\mathcal{L}=\mathcal{L}_{\mathrm{CE}}+\mathrm{BCE}(\sigma(\operatorname{sg}(m)+b),y). \tag*{(9)}

The stop-gradient on mm leaves its value available to BCE while blocking its gradient to NSP. The decision can equivalently be written as m≥−bm\geq-b: positive bb lowers the required screening margin, whereas negative bb raises it. Since σ(m)=p1/(p0+p1)\sigma(m)=p_1/(p_0+p_1) is the relative two-label probability, adding bb multiplies the corresponding odds by exp⁡(b)\exp(b). Here cc is a shared intercept, while wTu~w^{\mathsf{T}}\tilde{u} supplies the recording-dependent adjustment. Two recordings with the same NSP margin can therefore receive different decisions when their reference features differ. The coefficient of mm is fixed at one: the head learns an additive adjustment, without separately learning how much to weight the population margin. The standardized vector u~∈R3\tilde{u}\in\mathbb{R}^{3} is distinct from the predictor condition z∈R16z\in\mathbb{R}^{16}. IIP alone predicts from uu without mm, using its own supervised readout. Table 6 examines the contribution of its feature subsets.

FeaturesD-Vlog AccuracyD-Vlog F1 scoreD-Vlog pp-valueLMVD AccuracyLMVD F1 scoreLMVD pp-valueAVEC2014 AccuracyAVEC2014 F1 scoreAVEC2014 pp-valueMPDD-elder AccuracyMPDD-elder F1 scoreMPDD-elder pp-valueMPDD-young AccuracyMPDD-young F1 scoreMPDD-young pp-value
Reference deviation RR60.3656.770.001557.8059.690.50725.0033.330.950086.3682.350.000595.4595.650.0001
Residual decay rate forceAR_{\mathrm{AR}}54.4449.670.09254.4057.930.02362.5050.000.284659.0947.060.388140.9143.480.8969
Jacobian spectral descriptor ρ(J)\rho(J)46.7550.550.87768.3069.47<0.000550.0042.860.511068.1858.820.114259.0957.140.3352
Combined64.0068.970.000573.7075.88<0.000562.5050.000.284695.4592.310.000290.9190.91<0.0001
Combined + duration62.7272.000.00174.7075.18<0.000562.5050.000.284695.4593.330.0001100.00100.00<0.0001
Duration56.2168.640.14853.5052.320.55950.0033.330.715468.1846.150.264663.6469.230.1835

Table 6. Feature ablations and duration controls (%). Combined: RR + forceAR_{\mathrm{AR}} + ρ(J)\rho(J). Bold pp-values: p<0.05p < 0.05.

Standardization lets ww learn the contribution signs on a common training-defined scale. CE updates LoRA, while BCE learns (w,c)(w,c) from the current margin without changing the adapter’s CE gradient. The modules EE, E′E', gψg_{\psi} and fθf_{\theta} remain frozen.

The procedure therefore has three stages: healthy labels select the pool for self-supervised prediction, the frozen predictor supplies training-standardized features, and supervised learning fits the screening margin and its correction. Freezing predictor parameters still permits input differentiation for JJ. Algorithm 1 gives the full training and inference procedure.

Detailed NSP and IIP Computations

Figure 3 expands the two computational routes. In NSP, the highlighted weights W0W_0 and W1W_1 read label scores from the multimodal state immediately before the output digit. In IIP, history and context feed the healthy predictor, while the next observation follows the independent fixed-target route. The blue dashed and red solid curves show a projection of the prediction and observation. Their residual uses every target-space coordinate.

The residual-analysis route supplies RR and force⁡AR\operatorname{force}_{\mathrm{AR}}. The Jacobian route instead differentiates the frozen predictor at sˉ\bar{s} with zz fixed, supplying ρ^30\hat{\rho}_{30}. These routes meet at the descriptor vector uu, whose training-standardized values enter the correction head. Thus zz conditions the reference, while uu summarizes its predictive comparison and local mapping. The spectral route remains separate from the observed residual sequence.

Analytical properties of the reference

Let st=E(xt)∈R256s_t = E(x_t) \in\mathbb{R}^{256} be the online state, s~t=E′(xt)\tilde{s}_t = E'(x_t) the fixed-target state, and z∈R16z \in\mathbb{R}^{16} the context vector. The following results explain which mathematical properties motivate the descriptors and how the decision gradients are separated.

Residuals combine predictive departure and observation noise

Proposition 1 (Bias–noise decomposition of the residual). Let s~t+1=g(s≤t;z)+εt\tilde{s}_{t+1} = g(s_{\leq t}; z) + \varepsilon_t, with E[εt∣s≤t,z]=0\mathbb{E}[\varepsilon_t \mid s_{\leq t}, z] = 0 and conditional covariance Σε(s≤t,z)\Sigma_{\varepsilon}(s_{\leq t}, z). Then

E[rt2∣s≤t,z]=∥g(s≤t;z)−fθ(s≤t;z)∥22⏟departure term Δt2+tr⁡(Σε(s≤t,z)).(10)\mathbb{E}[r_t^2 \mid s_{\leq t}, z] = \underbrace{\left\lVert g(s_{\leq t}; z) - f_{\theta}(s_{\leq t}; z) \right\rVert_2^2}_{\text{departure term }\Delta_t^2} + \operatorname{tr}\left(\Sigma_{\varepsilon}(s_{\leq t}, z)\right). \tag*{(10)}

When g≈fθg \approx f_{\theta}, the expected squared residual approaches the conditional noise trace. A larger departure term increases the expected squared residual at a fixed noise level.

Proof of Proposition 1 (bias–noise decomposition). Substituting the observation model and suppressing the shared function arguments gives

E[rt2∣s≤t,z]=E[∥g+ϵt−fθ∥2∣s≤t,z]=∥g−fθ∥2+2E[ϵt∣s≤t,z]T(g−fθ)+E[∥ϵt∥2∣s≤t,z]=∥g−fθ∥2+tr⁡(Σϵ(s≤t,z)),(11)\begin{aligned} \mathbb{E}[r_t^2 \mid s_{\leq t}, z] &= \mathbb{E}[\lVert g+\epsilon_t-f_\theta\rVert^2 \mid s_{\leq t}, z] \\ &= \lVert g-f_\theta\rVert^2 + 2\mathbb{E}[\epsilon_t \mid s_{\leq t},z]^{\mathsf{T}}(g-f_\theta) \\ &\quad+ \mathbb{E}[\lVert\epsilon_t\rVert^2 \mid s_{\leq t},z] \\ &= \lVert g-f_\theta\rVert^2 + \operatorname{tr}(\Sigma_\epsilon(s_{\leq t},z)), \tag*{(11)} \end{aligned}

where E[ϵt∣s≤t,z]=0\mathbb{E}[\epsilon_t \mid s_{\leq t},z]=0 eliminates the cross term. The recording-level feature RR averages the corresponding unsquared norms.

The decomposition separates mismatch with the learned healthy reference from observation uncertainty. The downstream screening task evaluates the relevance of their combined residual.

AR persistence motivates the regularized decay descriptor

Proposition 2 (Residual AR(1) decay). Let the stationary, mean-centered sequence et=rt−E[rt]e_t=r_t-\mathbb{E}[r_t] follow et+1=φet+ηte_{t+1}=\varphi e_t+\eta_t with innovations satisfying E[ηt∣et,et−1,…]=0\mathbb{E}[\eta_t\mid e_t,e_{t-1},\ldots]=0 and 0<φ<10<\varphi<1. Its lag-one autocorrelation is φ\varphi, and E[et+k∣et]=φket=exp⁡(−ka)et\mathbb{E}[e_{t+k}\mid e_t]=\varphi^k e_t=\exp(-ka)e_t, where a=−ln⁡φa=-\ln\varphi is the unregularized residual decay rate.

This stationary, mean-centered model motivates the decay interpretation. The implemented median-centering, clipping and shrinkage regularize the empirical descriptor without establishing stationarity.

Proof of Proposition 2. Under the stationary AR(1) and conditional-mean assumptions, Cov⁡(et+1,et)=φVar⁡(et)\operatorname{Cov}(e_{t+1},e_t)=\varphi\operatorname{Var}(e_t), so its lag-one autocorrelation is φ\varphi. Repeated conditional expectation gives E[et+k∣et]=φket\mathbb{E}[e_{t+k}\mid e_t]=\varphi^k e_t. Substituting a=−ln⁡φa=-\ln\varphi yields the exponential form. Equation (5) then regularizes this estimate toward the healthy-training prior before it enters the screening features.

The Jacobian spectral descriptor depends on the learned coordinates

Proposition 3 (Common-coordinate transition: conditional relaxation result). For a first-order transition in common state coordinates, let sˉ\bar{s} be a fixed point and δt=st−sˉ\delta_t=s_t-\bar{s}. To first order δt+k≈Jkδt\delta_{t+k}\approx J^k\delta_t. For diagonalizable JJ with 0<ρ(J)<10<\rho(J)<1, its slowest linearized decay has characteristic time τ=−1/ln⁡ρ(J)\tau=-1/\ln\rho(J), which diverges as ρ(J)→1−\rho(J)\to1^{-}.

Proof of Proposition 3. Under the common-coordinate first-order transition assumptions of Proposition 3, a Taylor expansion about the fixed point sˉ\bar{s} gives δt+1≈Jδt\delta_{t+1}\approx J\delta_t, hence δt+k≈Jkδt\delta_{t+k}\approx J^k\delta_t. For diagonalizable J=VΛV−1J=V\Lambda V^{-1} we have ∥Jk∥≤κ(V)ρ(J)k\lVert J^k\rVert\leq\kappa(V)\rho(J)^k with κ(V)\kappa(V) the condition number of VV, which bounds the linearized decay when 0<ρ(J)<10<\rho(J)<1. The slowest mode attains this rate. Solving ρ(J)τ=e−1\rho(J)^\tau=e^{-1} gives τ=−1/ln⁡ρ(J)\tau=-1/\ln\rho(J).

The transition result assumes a common state basis and a fixed point. Our local Jacobian analysis differentiates the online-to-target map FzF_z at the mean online state. Under independent coordinate changes AA and BB, its Jacobian becomes BJA−1BJA^{-1}, so eigenvalues can change. We therefore interpret the spectral estimate in the fixed learned coordinates. The mean state need not be an equilibrium, and non-normal transitions can show transient amplification even when their spectral radius is below one.

For IIP, ρ(J)\rho(J) is therefore reported in the fixed learned coordinates rather than converted to a recovery time. Table 15 evaluates its numerical approximation.

CohortEstimatorMean relative error (%)Spearman correlationRuntime (s)Peak memory (GiB)
D-VlogExact eigendecomposition010.3220.059
D-Vlog30 JVP steps4.250.60860.2040.058
D-Vlog60 JVP steps5.320.55470.4440.058
D-Vlog120 JVP steps4.280.55420.7610.058
LMVDExact eigendecomposition010.3700.059
LMVD30 JVP steps4.530.94230.2240.058
LMVD60 JVP steps5.560.92210.4930.058
LMVD120 JVP steps4.080.95570.9290.058

Table 15. Jacobian spectral estimation: numerical accuracy and extraction cost.

Stop-gradient separates adapter and correction learning

Proposition 4 (Role of the stop-gradient on the margin). Assume bb is computed from the fixed unadapted IIP and is independent of the LoRA parameters. Without a stop-gradient on mm, BCE contributes an adapter gradient. With sg⁡(m)\operatorname{sg}(m), the adapter gradient equals the CE component at the current parameters.

Proof of Proposition 4. Writing ℓBCE=−ylog⁡σ(m+b)−(1−y)log⁡(1−σ(m+b))\ell_{\mathrm{BCE}}=-y\log\sigma(m+b)-(1-y)\log(1-\sigma(m+b)), we have ∂ℓBCE/∂m=σ(m+b)−y\partial\ell_{\mathrm{BCE}}/\partial m=\sigma(m+b)-y. Without a stop-gradient this term reaches the LoRA parameters ϕ\phi through mm. With sg⁡(m)\operatorname{sg}(m), BCE reaches only the bias-head parameters, so ∂L/∂ϕ=∂LCE/∂ϕ\partial\mathcal{L}/\partial\phi=\partial\mathcal{L}_{\mathrm{CE}}/\partial\phi exactly. This identity isolates the two gradients while retaining the dependence of the bias-head update on the screening margin.

Experiments

Experimental setup

D-Vlog [10] and LMVD [11] contain spontaneous, self-recorded vlogs, evaluated here with binary depression labels.

D-Vlog uses English YouTube recordings with the original test assignment restricted to currently available videos. LMVD uses Chinese vlogs and a 7:1:2 split. The healthy predictor is trained only on healthy training recordings, with no uploader shared between training and evaluation. AVEC2014 [18] and MPDD [12] provide interview/task-elicited recordings. Our AVEC2014 evaluation uses a subject-isolated 80:20 split of 83 participants (67 training, 16 test), keeping all recordings and segments from each participant together. MPDD quantitative experiments use cohort-specific retraining with subject-isolated 80:20 splits. Speech is transcribed with Whisper large-v3 [34]. Checkpoints are selected on validation data.

Following the baseline evaluation protocol, we report accuracy and positive-class F1, with depression as the positive class. The pathway and feature results in Table 2 and Table 6 are multi-seed summaries over five seeds. Figures show single-run exports. Group differences and feature-performance significance are assessed by permutation tests across all datasets (§ 4.3). Published baselines on the two vlog corpora include Mamba-based sequence models, transformer-based fusion and DepressionAgent, with values from the original papers in the comparison format of Zhu et al. [5]. For AVEC2014/MPDD, baseline entries are selected by best results for each method and cohort.

MethodD-Vlog AccuracyD-Vlog F1 scoreLMVD AccuracyLMVD F1 scoreAVEC2014 AccuracyAVEC2014 F1 scoreMPDD-elder AccuracyMPDD-elder F1 scoreMPDD-young AccuracyMPDD-young F1 score
Depression Detector [10]–63.50––72.0074.0764.9056.9059.0960.87
TAMFN [36]–65.82––64.0070.9765.2220.0059.0960.87
CAIINET [37]–66.55––60.0053.8045.5050.0081.1082.40
Depressformer [38]65.0059.00––84.0033.3355.2045.7068.1874.07
DepMamba [39]68.8776.4472.1373.2080.0016.6778.2661.5477.2780.00
CMF-Mamba [40]––74.8675.5346.7027.3052.3044.8074.1076.00
STCM-Mamba [41]72.6477.6975.4176.4962.0068.8560.8730.7763.6463.64
MSI-Net [42]70.2573.9174.3477.0669.0538.1047.8353.8572.7376.92
MDDFormer [11]––76.8876.8574.0033.3073.9166.6777.2773.68
BHDD [43]–75.9478.1178.0662.0067.8066.3617.7852.2751.16
DepressionAgent [5]95.2695.6982.5783.0456.2558.8278.2644.4459.0930.77
NSP (alone)97.1497.3874.3074.5775.0050.0086.3676.9250.0066.67
IIP (alone)64.0068.9773.7075.8862.5050.0095.4592.3190.9190.91
InertiaMPDD98.2998.4184.0084.9881.2566.6790.9185.7181.8280.00

Table 2. Depression-screening Accuracy and F1 score (%). Bold: largest displayed value. –: unavailable.

Implementation settings for Qwen2.5-Omni-3B [35], the predictor and the correction head are summarized in Table 1.

ItemValue
BackboneQwen2.5-Omni-3B thinker (bf16, frozen)
LoRArank 8, trained from scratch, lr 1×10−41 \times10^{-4}, batch 1×1 \times grad-accum 8, linear schedule
Bias headLinear(3→13 \to1), fp32, AdamW lr 1×10−21 \times10^{-2} (separate parameter group), N(0,0.1)\mathcal{N}(0, 0.1) init
IIPonline encoder 4096→2564096 \to256, 3-layer FiLM causal Transformer, width 256, 4 heads, feed-forward width 512, dropout 0.1, pre-normalization, fixed semi-orthogonal target
Conditioning MLP8192→512→512→168192 \to512 \to512 \to16, SiLU, final-layer N(0,0.01)\mathcal{N}(0, 0.01) initialization
IIP optimizationAdam, lr 3×10−43 \times10^{-4}, batch 4, 600 steps, variance/covariance weights 0.1 each
LossesNSP: CE, bias head: BCE (stop-gradient on the margin), IIP: MSE + VICReg
Input limitsaudio ≤300\le300 s (25 tok/s), IIP visual sampling: 64 frames
Latent state1 Hz pooled concatenation of audio 2048 and visual 2048 (xt∈R4096x_t \in\mathbb{R}^{4096}) →st∈R256\to s_t \in\mathbb{R}^{256}

Table 1. Implementation and hyperparameters.

Implementation and Evaluation Settings

Temporal alignment and context conditioning

Audio is limited to 300 seconds and provides 25 hidden tokens per second before one-second pooling. Visual alignment follows the grid defined in § 3.2.

The D-Vlog dynamics pool uses recordings of at least 120 seconds. This length criterion applies to reference learning, while screening evaluation retains the full test cohort.

Residual decay and Jacobian spectral estimation

The AR coefficient is fitted without an intercept after subtracting the recording median from the scalar residual sequence. Fewer than ten residuals or a lag-energy sum below 10−610^{-6} yields an unavailable estimate. Otherwise, the coefficient is clipped to [0.01,0.999][0.01, 0.999]. Its negative logarithm lies between approximately 0.001 and 4.605 per one-second step. The prior a0a_0 is the median available raw decay in the healthy training group, with weight two in the shrinkage formula. An unavailable estimate receives a0a_0. The training prior is also used on held-out recordings.

For the spectral feature, the predictor is in evaluation mode, with dropout disabled. The input is the single-token state [sˉ][\bar{s}] and the condition zz is fixed. Autograd computes Jacobian–vector products without storing a dense Jacobian. Power iteration starts from a random vector with seed zero. An iterate norm below 10−1210^{-12} returns zero. The updates and final estimate are

vk+1=Jvk∥Jvk∥2,ρ^30=∥Jv29∥2.(12)v_{k+1} = \frac{Jv_k}{\lVert Jv_k\rVert_2}, \qquad\hat{\rho}_{30} = \lVert Jv_{29}\rVert_2. \tag*{(12)}

The forward pass uses bf16. The reported descriptor is a finite-iteration spectral-radius estimate at the mean online state. Standalone IIP fits its own supervised binary readout on uu, without the NSP margin, with validation-based model selection.

Feature Aggregation and Statistical Tests

Group means summarize recording-level descriptors

Each recording ii supplies three scalars: time-averaged residual RiR_i, shrinkage-adjusted residual decay forceAR(i)_{\mathrm{AR}}^{(i)}, and the power-iteration estimate of ρ(Ji)\rho(J_i) at its mean state. Within class gg and split SS, Table 5 reports

DatasetSplitReference deviation RR HealthyReference deviation RR DepressedReference deviation RR pp-valueResidual decay rate forceAR_{\mathrm{AR}} HealthyResidual decay rate forceAR_{\mathrm{AR}} DepressedResidual decay rate forceAR_{\mathrm{AR}} pp-valueJacobian spectral descriptor ρ(J)\rho(J) HealthyJacobian spectral descriptor ρ(J)\rho(J) DepressedJacobian spectral descriptor ρ(J)\rho(J) pp-value
D-VlogTest6.7076.9010.00032.1031.9640.9980.7160.7100.467
D-VlogTraining6.8206.9160.0782.0901.9980.8970.7180.7090.421
LMVDTest12.51012.8870.00401.5591.4090.00020.9380.731<0.0001
LMVDTraining12.48712.966<0.00011.5471.4350.00100.8980.750<0.0001
AVEC2014Test11.80511.6350.81421.6341.7790.34050.9940.8690.0265
AVEC2014Training10.75911.0760.38302.0041.9690.55021.3641.3010.2868
MPDD-elderTest7.0769.0520.00050.4660.4690.85511.3961.4990.3009
MPDD-elderTraining7.3418.912<0.00010.4670.4790.38291.4001.3670.4118
MPDD-youngTest8.23810.7920.00040.6830.6790.90470.9541.0910.1359
MPDD-youngTraining8.31310.582<0.00010.7280.6780.01191.0381.0680.5215

Table 5. Healthy and depressed group means of unstandardized IIP descriptors. Bold: p<0.05p < 0.05.

qˉg,S=1Ng,S∑i∈Ig,Sqi,(13)\bar{q}_{g,S} = \frac{1}{N_{g,S}} \sum_{i\in\mathcal{I}_{g,S}} q_i, \tag*{(13)}

where qiq_i is one of these extracted, unstandardized features and Ng,S=∣Ig,S∣N_{g,S} = |\mathcal{I}_{g,S}|. Each recording contributes one value, irrespective of its duration. Group means use unstandardized descriptors, while decision readouts standardize each recording separately using training statistics.

Group and prediction tests address different hypotheses

The tests in Table 5 assess healthy–depressed differences in descriptor values. The tests in Table 6 assess predictive performance against label permutations. A descriptor can therefore have a group mean contrast but limited individual discrimination. The vlog residual tests use the directional hypothesis of higher values in the depressed group.

Pathway complementarity across recording settings

Vlog gains indicate complementary reference evidence. On D-Vlog, NSP reaches F1 97.38 and IIP alone 68.97, while fusion reaches 98.41 (Table 2). LMVD has more balanced pathways: IIP and NSP reach 75.88 and 74.57, and fusion reaches 84.98, a 9.10-point gain over its stronger pathway. Self-recorded vlogs permit individually chosen topics and spontaneous expression [10, 11]. Direct population cues are already strong on D-Vlog, whereas LMVD leaves more scope for a context-conditioned healthy comparison. Thus standalone reference strength and complementarity with NSP are distinct sources of screening value.

Interview settings can favor reference-derived descriptors. IIP reaches F1 92.31 on MPDD-elder and 90.91 on young, exceeding fusion at 85.71 and 80.00. Elder fusion still improves NSP by 8.79 points. Guided interaction in MPDD [12] can constrain freely expressed topics and behavior, making reference comparison informative alongside population classification. The ordering also motivates examining fusion separately: mm has coefficient one in Equation (8), so the correction head shifts the boundary without learning a separate pathway weight.

AVEC2014 reveals a boundary of reference discrimination. Fusion reaches 66.67 F1 versus 50.00 for either pathway, but remains below the largest displayed baseline value of 74.07. Although AVEC2014 and MPDD both use elicited recordings, their pathway orderings differ. Task coverage, age, language and cohort composition therefore matter alongside acquisition setting. A useful healthy reference must cover the evaluated behavior. The group and control experiments below examine where its predictive departures carry discriminative information.

Elicited recordings require task-matched references

MPDD uses guided interaction, while AVEC2014 includes prompted responses and reading [12, 18]. Reference coverage must therefore account for task regularities as well as expression, with age, language and cohort composition also varying across datasets.

The unstandardized contrasts retain the same direction across splits on MPDD, whereas AVEC2014 reverses direction. This links the screening differences to reference consistency and motivates broader task coverage.

Paired intervals quantify decision-level uncertainty

Paired bootstrap intervals quantify uncertainty in fixed-export gains. AVEC2014 and MPDD-young have wider intervals than D-Vlog, consistent with their smaller evaluation cohorts.

CohortTraining ΔR\Delta RTest ΔR\Delta R
MPDD-elder1.5711.976
MPDD-young2.2692.554
AVEC20140.317−0.170

Table 3. Cross-split reference-deviation contrasts and interpretation. Differences are depressed minus healthy means.

CohortQuantityF1 / Δ\DeltaF195% CI
D-VlogNSP97.38[94.84, 99.47]
Full98.41[96.38, 100.00]
Gain+1.03[0.00, +2.76]
AVEC2014NSP50.00[0.00, 85.71]
Full66.67[0.00, 100.00]
Gain+16.67[−50.00, +80.00]
MPDD-youngNSP66.67[42.86, 81.08]
Full80.00[54.52, 95.65]
Gain+13.33[−11.52, +37.44]

Table 4. Paired-bootstrap F1 intervals (%). Differences are percentage points.

From reference features to screening evidence

Vlog descriptors distinguish group shifts from individual discrimination. D-Vlog and LMVD show higher depressed-group test residuals (p=0.0003p = 0.0003 and 0.00400.0040), yet LMVD’s residual-only screening is nonsignificant (p=0.507p = 0.507). Its Jacobian descriptor is the strongest single feature (F1 69.47), and the combination reaches 75.88. D-Vlog’s combination improves residual-only F1 from 56.77 to 68.97 despite weak marginal temporal and spectral contrasts. Group tests measure class-mean differences, whereas feature-performance tests assess prediction against shuffled labels. Joint descriptor information can therefore improve classification

Reference deviation is the clearest recurring MPDD descriptor. Depressed-group RR is higher in both training and test for both cohorts, with test p=0.0005p = 0.0005 (elder) and 0.0004 (young). Residual-only F1 reaches 82.35 and 95.65 (Table 6). Combining descriptors raises elder F1 to 92.31 but gives 90.91 on young. The recurring residual contrast supports reference departure as useful evidence, while the value of temporal and spectral additions depends on the cohort.

AVEC2014 lacks a stable cross-split signature. The depressed-minus-healthy residual contrast changes from +0.317 in training to −0.170 in test, with neither significant. The test-only spectral contrast (p=0.0265p = 0.0265) does not recur significantly in training, and spectral-only screening remains nonsignificant (p=0.5110p = 0.5110). Combined F1 is 50.00 (p=0.2846p = 0.2846), locating the main difficulty in reference discrimination rather than simply pathway combination.

Duration can interact with the behavioral descriptors. Adding it raises D-Vlog F1 to 72.00 but reduces accuracy, and gives no F1 gain on LMVD. Duration-augmented MPDD readouts reach 93.33 and 100.00 F1 despite weak duration-only performance. We keep duration outside the main IIP definition. Duration-matched and healthy-pool controls further examine these interactions (Table 14).

SettingLMVDMPDD elderMPDD young
IIP reference75.8892.3190.91
Duration matched56.07100.0090.91
AV interval masking80.6587.5085.71
Healthy-pool subsampling70.5473.6895.24

Table 14. IIP controls for duration, interval masking and reference-pool size: F1 score (%).

Candidate proxy markers and their scope

The results support reference-derived emotional dynamics as candidate proxy biomarkers: computable behavioral descriptors associated with depression labels under recording-level supervision. Their relevance is established through screening, rather than direct emotion annotations. Stress, fatigue and acquisition conditions may also affect predictive departure. Comparison with alternative conditions can test specificity, while repeated recordings and longitudinal symptom measurements can assess repeatability and sensitivity to change within a defined context of use [44]. The cohort-specific fits evaluate within-cohort utility, while a fixed reference on an unseen cohort requires a distinct test of transfer.

Feature Distributions and Screening Implications

Each row of Figure 4 corresponds to one cohort. Dots summarize recordings and dashed lines indicate class means. Table 5 and Table 6 provide the group tests and feature readouts.

Image not extracted

Figure 4. Test-set IIP feature distributions. Green: healthy. Red: depressed. Violins show distributions, dots recordings, and dashed lines class means.

Elicited cohorts differ in reference-deviation consistency

Depressed residuals concentrate at higher values in both MPDD cohorts. Healthy observations sometimes reach the same range, particularly on young. Temporal and spectral ranges overlap substantially. Elder spectral estimates can exceed one within the fixed predictor coordinates. Together, these patterns explain why reference deviation recurs as a strong feature while the remaining descriptors offer cohort-dependent additions.

On AVEC2014, the residual groups occupy a broad shared range. Depressed spectral values tend toward the lower part of the healthy range in test, but that mean contrast does not recur significantly in training or yield significant spectral-only screening. The visualization therefore complements the cross-split evidence for weak reference discrimination.

Vlog overlap distinguishes mean shifts from useful combinations

D-Vlog spectral and AR distributions overlap closely. Depressed residuals shift upward with a longer upper tail, while overlap remains. On LMVD, healthy samples extend to larger spectral-radius estimates and decay values and have a lower residual tail. These shapes explain how a residual mean difference can coexist with weak residual-only screening, and why a joint readout can benefit from the other features.

Component ablation

Table 7 summarizes decision and reference-construction ablations. The comparisons below separate the effects of context conditioning, temporal order, modality and pathway combination.

ConfigurationD-VlogLMVDAVEC 2014MPDD elderMPDD young
InertiaMPDD (full)98.4184.9866.6785.7180.00
NSP only97.3874.5750.0076.9266.67
w/o conditioning97.8775.0750.0072.7366.67
Global correction97.3580.8466.6772.7366.67
Full IIP (audio + visual)68.9775.8850.0092.3190.91
Shuffled history67.0259.6835.2977.7890.91
Repeated mean state71.9274.8146.1560.0063.64
Visual only62.5062.7840.0082.3586.96
Audio only75.6864.8636.3677.7873.68

Table 7. Decision and reference-construction ablations: F1 score (%). Blue: full configurations.

Context and correction address different personalization steps

Removing zz gives fθ0(s≤t)f_{\theta_0}(s_{\leq t}). Global context replaces recording-specific conditioning with a shared training-set mean, and context swapping assigns another recording’s context to the observed history. Decision correction is a separate operation: global correction replaces b=w⊤u~+cb = w^{\top}\tilde{u} + c with −τ⋆-\tau^\star, while NSP alone sets b=0b = 0.

ConfigurationD-VlogLMVDAVEC 2014MPDD elderMPDD young
InertiaMPDD98.4184.9866.6785.7180.00
w/o conditioning97.8775.0750.0072.7366.67
Global context97.8775.0754.5572.7366.67
Context swapping97.3576.5533.3377.7890.91
Global correction97.3580.8466.6772.7366.67

Table 8. Conditioning and correction controls: F1 score (%).

On LMVD, full F1 is 84.98, versus 75.07 without conditioning and 80.84 with global correction. Elder reaches 85.71 with the full model, compared with 72.73 for both controls. These benefits are not uniform: AVEC2014 ties global correction, and young’s swapped-context result exceeds full fusion.

Reference construction benefits from temporal order and multiple modalities

Ordered prediction benefits LMVD and elder. Shuffling preserves the recording’s states while disrupting their order, whereas repeating the mean removes within-recording changes. LMVD reaches 75.88 F1 with full IIP, 59.68 after shuffling and 74.81 with a repeated mean. Elder reaches 92.31, 77.78 and 60.00, respectively. Young’s shuffled history reaches 90.91, matching full IIP. Its strong static control also shows that recording-level state information can be discriminative. AVEC2014 remains weak across predictor designs.

Learned prediction improves on simplified references. On LMVD, full IIP reaches 75.88 F1, compared with 70.36 for a static MLP, 67.82 for persistence and 65.50 for a linear AR predictor. The corresponding elder scores are 92.31, 66.67, 58.82 and 53.33. These contrasts support learning a conditional next-state reference beyond static mapping, state repetition or linear prediction on these cohorts.

ConfigurationD-VlogLMVDAVEC2014MPDD-elderMPDD-young
AccuracyF1 scoreAccuracyF1 scoreAccuracyF1 scoreAccuracyF1 scoreAccuracyF1 score
Full IIP (causal Transformer)64.0068.9773.7075.8862.5050.0095.4592.3190.9190.91
Static MLP65.6869.1567.6070.3656.250.0068.1866.67100.00100.00
Persistence62.7263.1664.1067.8256.2546.1568.1858.8245.4533.33
Linear AR predictor63.3167.3761.5065.5075.0033.3368.1853.3381.8281.82
Shuffled history62.7267.0257.2059.6831.2535.2981.8277.7890.9190.91
Repeated mean state66.2771.9273.1074.8156.2546.1563.6460.0063.6463.64
Visual only53.8562.5060.7062.7825.0040.0086.3682.3586.3686.96
Audio only73.3775.6860.3064.8656.2536.3681.8277.7877.2773.68

Table 9. Complete IIP predictor and modality controls (%). Blue: full IIP.

Audio and vision provide complementary reference evidence on LMVD and MPDD. Joint IIP reaches 75.88 F1 on LMVD, exceeding visual-only 62.78 and audio-only 64.86, and exceeds either modality on both MPDD cohorts. Reference utility and the need for true temporal order therefore vary across cohorts.

ConfigurationAccuracyF1 score
Visual-only NSP73.1475.39
Audio-only NSP92.5792.82
Audio–visual NSP95.4395.74
Audio–visual–text NSP97.1497.38
Audio–visual NSP + IIP95.4395.74
InertiaMPDD98.2998.41

Table 10. D-Vlog NSP modality and correction controls (%).

D-Vlog has strong direct acoustic and language cues. Adding transcripts to audiovisual NSP raises F1 from 95.74 to 97.38, with correction reaching 98.41.

Descriptor informativeness differs from fusion complementarity

Correction inputD-VlogAVEC 2014MPDD elderMPDD young
NSP only97.3850.0076.9266.67
NSP + RR97.8966.6777.7885.71
NSP + force⁡AR\operatorname{force}_{\mathrm{AR}}97.3554.5566.6766.67
NSP + ρ(J)\rho(J)97.8750.0072.7370.00
NSP + RR, force⁡AR\operatorname{force}_{\mathrm{AR}}97.3828.5777.7885.71
NSP + RR, ρ(J)\rho(J)97.8928.5777.7885.71
NSP + force⁡AR\operatorname{force}_{\mathrm{AR}}, ρ(J)\rho(J)97.3528.5766.6772.73
InertiaMPDD98.4166.6785.7180.00

Table 11. Fusion feature controls: F1 score (%).

A useful descriptor need not require the full feature combination. RR alone matches full correction on AVEC2014 and exceeds it on MPDD-young, whereas the full correction is strongest among these inputs on D-Vlog and elder. Standalone informativeness and fusion complementarity are therefore distinct.

Pathway weighting changes how reference evidence is used

Scaling learns α\alpha in αm+b\alpha m+b. Convex fusion uses αm+(1−α)q(u)\alpha m+(1-\alpha)q(u), and feature-dependent gating replaces α\alpha with α(u)\alpha(u), where q(u)q(u) is the standalone IIP score.

Pathway balance matters when IIP is strong. Convex fusion reaches 93.33 F1 on elder and 100.00 on young, compared with 85.71 and 80.00 for fixed additive correction. Allowing the relative pathway weights to change can therefore make better use of the strong standalone IIP scores on MPDD. Fixed additive correction remains strongest among these rules on both vlog cohorts, so a more flexible rule does not uniformly improve screening. These controls separate the informativeness of the reference descriptors from the decision rule used to combine them with NSP.

Decision ruleD-VlogLMVDAVEC 2014MPDD elderMPDD young
NSP only97.3874.5750.0076.9266.67
IIP only68.9775.8850.0092.3190.91
Global correction97.3580.8466.6772.7366.67
InertiaMPDD98.4184.9866.6785.7180.00
Scaling97.8775.2050.0048.2866.67
Convex fusion97.3877.0660.0093.33100.00
Feature-dependent gating97.3877.8536.3675.0090.91

Table 12. Pathway-weighting controls: F1 score (%).

D-Vlog decisions can survive simplified dynamics

These single-run diagnostics hold NSP margins fixed and refit correction heads to test whether simplified references preserve the screening decisions.

ControlObserved relation to the diagnostic reference
Zero or global contextSame test decisions as true context in one setting
Persistence predictorReproduces the conditional predictor’s test decisions
PCA / Mahalanobis scoresMatch confusion counts, each differing on two recordings
Global / block time shufflingRetain screening performance
Repeated mean-state inputRetains strong screening performance without temporal variation

Table 13. D-Vlog diagnostic controls with a fixed population margin.

Strong NSP margins make several reference variants decision-equivalent. Label-independent keyword-aligned masking of cached audiovisual intervals gives 96.84 F1 versus 96.88 for its diagnostic reference, testing those intervals rather than transcript-token removal from the full model.

Reference coverage affects descriptor quality

Reference coverage affects discrimination. Duration matching preserves strong MPDD IIP performance, whereas healthy-pool subsampling weakens LMVD and elder. The cohort-dependent response to interval masking further supports learning references that cover the available expression patterns.

Numerical accuracy and extraction cost

Thirty JVP steps balance spectral accuracy and cost. Relative errors are 4.25% and 4.53% on D-Vlog and LMVD, with faster extraction than exact eigendecomposition. Further iterations increase runtime without consistently reducing error.

Incremental descriptor-extraction cost IIP extraction from cached backbone features takes approximately 0.23 seconds per recording, including predictor evaluation and JVP iterations, with peak GPU memory below 1 GiB in the recorded runs. Fitting the correction head takes approximately 0.2–0.3 seconds on CPU. Backbone extraction is excluded from these measurements.

Case studies and visualization

Residual traces localize predictive disagreement. Figure 5 illustrates the temporal information compressed into RR. The healthy trajectory follows its reference through marked fluctuations, whereas the depressed example shows sustained residual elevations. Movement alone is insufficient: a changing trajectory can be well predicted, while a smooth one can remain offset from its reference. The full-state residual localizes disagreement even when the displayed projection looks similar. Recording-level labels identify the screening category, while residual traces indicate when the observed continuation differs from the learned reference. These traces guide audiovisual inspection of isolated and recurring disagreement. Averaging these errors yields RR, and residual decay summarizes the persistence of their fluctuations. Together with the spectral descriptor, these summaries support recording-level screening. They locate predictive disagreement without segment-level symptom attribution. § 4.9.1 includes the complete multi-dataset cases.

Image not extracted

Figure 5. D-Vlog case studies. Healthy (top) and depressed (bottom) recordings. Green: healthy reference. Red: subject trace or residual. Shading: healthy residual 5–95% range.

Correction depends on joint score geometry. Figure 6 connects the descriptors to screening decisions. The top row shows continuous NSP margins. The bottom row places each recording relative to b=−mb=-m. D-Vlog separates strongly along mm, while its IIP view overlaps substantially. LMVD leaves more scope for adjustment in the plotted run. Table 2 reports the quantitative pathway comparison. In the plotted AVEC2014 run, both classes lie on the negative-score side, so feature variation alone does not ensure a useful decision boundary. On MPDD-elder, correction moves one missed positive across the boundary. It improves NSP without surpassing standalone IIP. Marginal densities describe coordinate separation, whereas the joint (m,b)(m,b) location determines the additive decision.

Image not extracted

Figure 6. Test-set pathway views from single-run exports. Columns: cohorts. Rows: NSP margins, IIP features and InertiaMPDD (m,b)(m, b) with boundary b=−mb=-m. Blue circles: healthy. Orange triangles: depressed. Dark outlines: changed decisions.

For a schematic example, recordings with the same m=0.4m=0.4 but corrections b=−0.7b=-0.7 and b=0.2b=0.2 receive final scores −0.3-0.3 and 0.60.6. The recording-dependent correction therefore changes the decision even when population evidence is identical.

Case Studies across Datasets

Visualization settings D-Vlog bands use healthy test recordings with the displayed case excluded. Pointwise 5th, 50th and 95th percentiles summarize aligned one-second traces. Its two latent curves share a first-principal-component projection fitted to the healthy pool.

The MPDD pair comes from an earlier case-study run using the official partition and contains training-partition examples. Its healthy-training reference band excludes the displayed subject. Quantitative MPDD tables use cohort-specific 80:20 retraining. These cases illustrate prediction agreement under their visualization settings. Reference bands and latent projections are descriptive context for reading the trajectories.

Predictive agreement is more informative than visible stillness LMVD further separates predictive agreement from visible stillness. The healthy recording fluctuates with intermittent residual peaks, whereas the relatively smooth depressed trace remains offset from its reference. The relevant contrast is prediction agreement rather than the amount of visible motion.

Image not extracted

Figure 7. Case studies across datasets. Healthy/depressed pairs from D-Vlog, MPDD-young, AVEC2014 and LMVD. Green: healthy reference. Red: subject trace or residual. Shading: healthy residual 5–95% range.

Similar projections can conceal full-state disagreement In MPDD-young, the healthy trajectory follows long trends and abrupt changes with mainly local error excursions. The depressed example follows the broad projected shape but has a more persistently elevated residual. The full-state norm captures disagreement beyond the single coordinate shown in the latent curve.

AVEC examples explain the feature without defining the cohort Both AVEC2014 trajectories follow the reference’s large-scale changes, while the depressed example has a residual offset. This illustrates the temporal meaning of RR without overriding the cohort’s weak standalone separation.

Conclusion

InertiaMPDD explores how pretrained multimodal representations can supply measurable evidence for depression screening without fine-grained emotion labels. Its world-model-inspired healthy reference produces reference-derived emotional dynamics descriptors for standalone screening and individual correction. The experiments support three findings. A predictive reference makes behavioral departure useful for screening, with recurring MPDD deviation contrasts and complementary vlog evidence. Reference construction draws on both temporal order and multimodal content: LMVD benefits from ordered prediction and joint audio–visual inputs, while strong static controls on MPDD-young show that informative reference evidence can also reside in recording-level state patterns. Reference informativeness and fusion benefit remain distinct, motivating cohort-sensitive use of the two pathways. AVEC2014’s weak standalone separation identifies reference coverage as a key boundary. These candidate proxy descriptors warrant further validation of their specificity, repeatability and transfer.

References

  1. [1]Md. Rezwamul Haque, Md. Milon Islam, S. M. Taslim Uddin Raju, Hamdi Altaheri, Lobna Nassar, and Fakhri Karray. MMFormer: Multimodal fusion transformer network for depression detection. arXiv preprint arXiv:2508.06701, 2025.
  2. [2]Jincheng Li, Menglin Zheng, Jiongyi Yang, Yihui Zhan, and Xing Xie. DynMultiDep: A dynamic multimodal fusion and multi-scale time series modeling approach for depression detection. Journal of Imaging, 12(1):29, 2026.DOI
  3. [3]Misha Sadeghi, Robert Richer, Bernhard Egger, Lena Schindler-Gmelch, Lena H. Rupp, Farnaz Rahimi, Matthias Berking, and Bjoern Eskofier. Harnessing multimodal approaches for depression detection using large language models and facial expressions. npj Mental Health Research, 3(1):66, 2024.DOI
  4. [4]Lei Wang, Chao Wang, Chen Li, Tetsuya Murai, Yan Bai, Zhen Song, Shuo Zhang, Qian Zhang, Yan Huang, and Xiaojun Bi. AI-assisted multi-modal information for the screening of depression: A systematic review and meta-analysis. npj Digital Medicine, 8(1):523, 2025.DOI
  5. [5]Fangjie Zhu, Haifeng Lu, Sicheng Zhao, Runhao Zeng, and Xiping Hu. DepressionAgent: Reading, listening, seeing, and deliberating multimodal evidence for depression risk assessment. arXiv preprint arXiv:2608.13891, 2026.arxiv.org/abs/2608.13891
  6. [6]Linhai Zhang, Ziyang Gao, Deyu Zhou, and Yulan He. Explainable depression detection in clinical interviews with personalized retrieval-augmented generation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 9927–9944, 2025.arxiv.org/abs/2503.01315
  7. [7]Peter Kuppens, Nicholas B. Allen, and Lisa B. Sheeber. Emotional inertia and psychological maladjustment. Psychological Science, 21(7):984–991, 2010.DOI
  8. [8]Peter Koval, Madeline L. Pe, Kristof Meers, and Peter Kuppens. Affect dynamics in relation to depressive symptoms: variable, unstable or inert? Emotion, 13(6):1132, 2013.DOI
  9. [9]Marlies Houben, Wim Van Den Noortgate, and Peter Kuppens. The relation between short-term emotion dynamics and psychological well-being: A meta-analysis. Psychological Bulletin, 141(4):901–930, 2015.DOI
  10. [10]Jeewoo Yoon, Chaewon Kang, Seungbae Kim, and Jinyoung Han. D-Vlog: Multimodal vlog dataset for depression detection. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 12226–12234, 2022.
  11. [11]Lang He, Kai Chen, Junnan Zhao, Yimeng Wang, Ercheng Pei, Haifeng Chen, Jiewei Jiang, Shiqing Zhang, Jie Zhang, Zhongmin Wang, Tao He, and Prayag Tiwari. LMVD: A large-scale multimodal vlog dataset for depression detection in the wild. Information Fusion, 126:103632, 2026.arxiv.org/abs/2407.00024
  12. [12]Changzeng Fu, Zelin Fu, Qi Zhang, Xinhe Kuang, Jiacheng Dong, Kaifeng Su, Yikai Su, Wenbo Shi, Junfeng Yao, Yuliang Zhao, Shiqi Zhao, Jiadong Wang, Siyang Song, Chaoran Liu, Yuichiro Yoshikawa, Björn Schuller, and Hiroshi Ishiguro. The first MPDD challenge: Multimodal personality-aware depression detection. In Proceedings of the 33rd ACM International Conference on Multimedia, pages 13924–13929, 2025.arxiv.org/abs/2505.10034
  13. [13]Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette Chang, Sungbok Lee, and Shrikanth S. Narayanan. IEMOCAP: Interactive emotional dyadic motion capture database. Language Resources and Evaluation, 42(4):335–359, 2008.DOI
  14. [14]David Ha and Jürgen Schmidhuber. Recurrent world models facilitate policy evolution. In Advances in Neural Information Processing Systems, 2018.arxiv.org/abs/1809.01999
  15. [15]Yann LeCun. A path towards autonomous machine intelligence. OpenReview, 2022.
  16. [16]Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with a joint-embedding predictive architecture. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15619–15629, 2023.arxiv.org/abs/2301.08243
  17. [17]Mido Assran, Adrien Bardes, David Fan, Quentin Garrido, et al. V-JEPA 2: Self-supervised video models enable understanding, prediction and planning. arXiv preprint arXiv:2506.09985, 2025.
  18. [18]Michel Valstar, Björn Schuller, Kirsty Smith, Timur Almaev, Florian Eyben, Jarek Krajewski, Roddy Cowie, and Maja Pantic. AVEC 2014: 3D dimensional affect and depression recognition challenge. In Proceedings of the 4th International Workshop on Audio/Visual Emotion Challenge, pages 3–10, 2014.DOI
  19. [19]Jonathan Gratch, Ron Artstein, Gale Lucas, Giota Stratou, Stefan Scherer, Angela Nazarian, Rachel Wood, Jill Boberg, David DeVault, Stacy Marsella, David Traum, Skip Rizzo, and Louis-Philippe Morency. The distress analysis interview corpus of human and computer interviews. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC), pages 3123–3128, 2014.DOI
  20. [20]Bowen Zhou, Marc-André Fiedler, and Ayoub Al-Hamadi. CAF-Mamba: Mamba-based cross-modal adaptive attention fusion for multimodal depression detection. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 12107–12111, 2026.
  21. [21]Caijian Hua, Zhuofu Wen, Liuying Li, and Xia Zhou. ASYM: Multimodal depression recognition via mamba-enhanced attentive feature fusion. Frontiers in Psychiatry, 17:1704005, 2026.DOI
  22. [22]Xiangyun Zheng, Yingjian Liu, Zonghai Zha, Minghuan Lv, and Hao Wang. AttMamba: Mamba with attention mechanism for multimodal depression detection. Neurocomputing, 684:133570, 2026.DOI
  23. [23]S. Liu, Y. Huang, and X. Yuan. PAVNet: A personality-aware audio-visual fusion network for depression detection. In IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 5868–5875, 2025.DOI
  24. [24]Changzeng Fu, Yiming Zhang, Ming Shang, Shiqi Zhao, Alexis Meneses, Zhaojie Luo, Chaoran Liu, Yuichiro Yoshikawa, Hiroshi Ishiguro, and Björn Schuller. MPDD-AVG: Multimodal personality-aware depression detection via audio-visual interview and gait analysis. In Proceedings of the 34th ACM International Conference on Multimedia, 2026.DOI
  25. [25]Changzeng Fu, Kaifeng Su, Yikai Su, Fengkui Qian, Yixuan Zhang, Chaoran Liu, Siyang Song, Le Yang, Xiaoyong Lv, Peng Shan, et al. M3ADD: A novel benchmark for physiology signal-based automatic depression detection with multimodal multitask multievent framework. In ICASSP 2025–2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2025.DOI
  26. [26]Changzeng Fu, Yi Wang, Yiming Zhang, Kaifeng Su, Yikai Su, Yunze Zhang, Shiwen Zhao, Chaoran Liu, and Shiqi Zhao. M3DEP: Individual difference-aware depression detection from EEG-EMG signals via multievent-multimodal-multitask learning. In Proceedings of the 34th ACM International Conference on Multimedia, pages 2035–2043, 2026.
  27. [27]Wen Liu, Weixin Luo, Dongze Lian, and Shenghua Gao. Future frame prediction for anomaly detection—a new baseline. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6536–6545, 2018.arxiv.org/abs/1712.09867
  28. [28]Mingxuan Hu, Ziqi Liu, Hongbo Ma, Jiaqi Liu, Xinlan Wu, and Yangbin Chen. Mitigating gender bias in depression detection via counterfactual inference. arXiv preprint arXiv:2512.01834, 2026.arxiv.org/abs/2512.01834
  29. [29]Adrien Bardes, Jean Ponce, and Yann LeCun. VICReg: Variance-invariance-covariance regularization for self-supervised learning. In International Conference on Learning Representations (ICLR), 2022.arxiv.org/abs/2105.04906
  30. [30]Ingrid A. van de Lemput, Marieke Wichers, Angélique O. J. Cramer, Denny Boorsboom, Francis Tuurlinckx, Peter Kuppens, Egbert H. van Nes, Wolfgang Viechtbauer, Erik J. Giltay, Steven H. Aggen, Catherine Derom, Nele Jacobs, Kenneth S. Kendler, Han L. J. van der Maas, Michael C. Neale, Frenk Peeters, Evert Thiery, Peter Zachar, and Marten Scheffer. Critical slowing down as early warning for the onset and termination of depression. Proceedings of the National Academy of Sciences, 111(1):87–92, 2014.
  31. [31]Aapo Hyvärinen, Hiroaki Sasaki, and Richard E. Turner. Nonlinear ICA using auxiliary variables and generalized contrastive learning. In Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), pages 859–868, 2019.arxiv.org/abs/1805.08651
  32. [32]Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Toward causal representation learning. Proceedings of the IEEE, 109(5):612–634, 2021.DOI
  33. [33]Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR), 2022.arxiv.org/abs/2106.09685
  34. [34]OpenAI. Whisper large-v3 model card. Accessed November 7, 2026.
  35. [35]Qwen Team. Qwen2.5-Omni technical report. arXiv preprint arXiv:2503.20215, 2025.
  36. [36]Licai Zhou, Zhenyu Liu, Zixuan Shangguan, Xiaoyan Yuan, Yang Li, and Bin Hu. TAMFN: Time-aware attention multimodal fusion network for depression detection. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 31:669–679, 2022.DOI
  37. [37]Licai Zhou, Zhenyu Liu, Xiaoyan Yuan, Zixuan Shangguan, Yang Li, and Bin Hu. CAINNET: Neural network based contextual attention and information interaction mechanism for depression detection. Digital Signal Processing, 137:103986, 2023.DOI
  38. [38]Lang He, Zheng Li, Prayag Tiwari, Cui Cao, Jingjing Xue, Feng Zhu, and Di Wu. Depressformer: Leveraging video swin transformer and fine-grained local features for depression scale estimation. Biomedical Signal Processing and Control, 96:106490, 2024.DOI
  39. [39]Jiaxin Ye, Junjie Zhang, and Hongming Shan. DepMamba: Progressive fusion mamba for multimodal depression detection. In ICASSP 2025 – IEEE International Conference on Acoustics, Speech and Signal Processing, pages 1–5, 2025.arxiv.org/abs/2409.15936
  40. [40]Bowen Zhou, Marc-André Fiedler, and Ayoub Al-Hamadi. Cross-modal fusion mamba for multimodal depression detection. In 2025 14th International Symposium on Image and Signal Processing and Analysis (ISPA), pages 225–230. IEEE, 2025.DOI
  41. [41]Bowen Zhou, Marc-André Fiedler, and Ayoub Al-Hamadi. STCM-Mamba: Multimodal spatio-temporal cross-modal mamba for depression detection. IEEE Access, 13:189494–189505, 2025.DOI
  42. [42]Min Shi, Xin Qiao, Lang He, and Rui Gao. An interaction network based on multiscale spatiotemporal feature fusion for depression prediction. IEEE Transactions on Affective Computing, 2026.
  43. [43]Meng Ma, Chao Wang, Shuai Liu, Wei Zhao, and Xin Jiang. BHDD: Facial behaviors modeling and hierarchical multimodal fusion for depression detection from vlog data. IEEE Transactions on Affective Computing, 2026.DOI
  44. [44]U.S. Food and Drug Administration. About biomarkers and qualification. Biomarker Qualification Program. Accessed November 7, 2026.

Paper details

vlnc:W_hp79vy4h
Version
v2 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 13

downloads · 1

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents