betaFeedback

WorksComputer Science

Are Joint-Embedding Predictive Objectives Effective for Longitudinal EHRs?

Abstract

Masked language modelling (MLM) is the dominant pretraining objective for transformer encoders applied to longitudinal electronic health records (EHRs). However, reconstructing individual events may overemphasise noisy or weakly informative observations and does not explicitly encourage learning higher-level patient states. Joint-Embedding Predictive Architectures (JEPAs) instead learn by predicting representations in latent space and have shown strong performance in a variety of domains. In this paper, we investigate to what extent JEPA can be adapted to EHR sequence representation. Our empirical investigation using MIMIC-IV shows that although all three of our JEPA formulations train stably without representational collapse, they consistently underperform standard MLM. We find that target size, not target heterogeneity, increases prediction error, yet reducing the target size does not recover downstream performance. This suggests that stable latent prediction alone is insufficient for competitive EHR representation learning and highlights the challenge of defining clinically meaningful latent targets for EHR.

Introduction

Electronic Health Records (EHRs) are longitudinal records of patient-healthcare interactions. Recent work has increasingly focused on EHR foundation models, which aim to learn general-purpose patient representations through self-supervised learning and can be adapted to a range of downstream clinical prediction tasks [1, 2, 3, 4]. Downstream performance therefore hinges on the quality of these representations.

Due to the sequential nature of EHRs, transformer encoders have become a common backbone for EHR representation learning. A dominant objective for pretraining has been BERT-style masked language modelling (MLM), in which the model is trained to reconstruct masked medical events from their surrounding context [5, 6, 7, 8]. However, exact token reconstruction may not provide the ideal learning signal for longitudinal clinical data. EHR sequences contain routine, repeated, administrative, and partially unpredictable events, yet MLM treats reconstruction of each masked token as equally important. This may encourage representations that capture the identity of local events rather than the true patient state.

In contrast, clinicians use their medical knowledge to abstract away irrelevant detail and focus on key events, trends, and interactions to form a higher-level representation of the patient state. This motivation is closely aligned with Joint-Embedding Predictive Architectures (JEPAs), which provide an alternative form of self-supervision [9, 10]. Rather than reconstructing observations in the input space, JEPA trains a predictor to estimate latent representations of unobserved targets from an available context. This approach has shown strong representation-learning performance in vision [10, 11], world modelling [12] and language [13]. Latent prediction is intended to produce semantically meaningful features and avoid unnecessary reconstruction of unpredictable low-level detail. This is particularly appealing for EHRs, where multiple observed event sequences may correspond to similar underlying clinical states and trajectories.

However, defining a meaningful and predictable JEPA target in longitudinal EHRs is non-trivial. Unlike image patches, EHR events are irregularly sampled, vary substantially in density, and already represent semantically meaningful concepts. Temporal proximity therefore does not necessarily imply semantic similarity, and future clinical events may depend not only on patient state but also on healthcare decisions.

In this work, we ask whether the I-JEPA-style latent prediction framework [10] can be transferred effectively to longitudinal EHR sequences. We evaluate multiple target-construction strategies, including span masking, future masking and random masking of events. We investigate (1) whether these JEPA formulations can be trained stably without representational collapse, (2) how the downstream performance compares, and (3) which objective and training characteristics may explain the observed differences.

Methodology

Medical Event Representation

A patient trajectory is represented as a time-ordered sequence of clinical events, X=(e1,e2,…,eN)X = (e_{1}, e_{2}, \ldots, e_{N}), where each event is a triplet ei=(ci,vi,ti)e_{i} = (c_{i}, v_{i}, t_{i}) comprising a discrete medical code ci∈Vc_{i} \in V, an optional continuous value vi∈Rv_{i} \in\mathbb{R}, and timestamp tit_{i}. Patient age aia_{i} is derived from tit_{i} and date of birth. Representing time and continuous values is an important design choice in EHR modelling [14, 15, 16]. Rather than discretising continuous recordings into tokens, we retain them directly in the event representation using triplet embeddings [14]. The input embedding for event eie_{i} is

hi=Ecode(ci)+Proj⁡val(vi)⋅I[vi≠∅]+Proj⁡age(ai),(1)h_{i} = E_{\mathrm{code}}(c_{i}) + \operatorname{Proj}_{\mathrm{val}}(v_{i}) \cdot\mathbb{I}[v_{i} \ne\varnothing] + \operatorname{Proj}_{\mathrm{age}}(a_{i}), \tag*{(1)}

where EcodeE_{\mathrm{code}} is a learned code embedding and I\mathbb{I} indicates value presence. Sequence order is encoded using Rotary Position Embeddings (RoPE). All models use the same transformer backbone with 6 layers, 8 attention heads, hidden dimension 768, and maximum context length N=1024N = 1024, ensuring that downstream differences reflect the pretraining objective rather than model capacity.

Objectives

JEPA performance depends strongly on context–target design [13], and as such, we evaluate three EHR target constructions reflecting different assumptions about what should be predictable.

Our first objective, Span JEPA, follows the I-JEPA paradigm as directly as possible [10]. We treat the EHR as a one-dimensional sequence of NN clinical events. The complete uncorrupted sequence is passed through the exponential moving average (EMA) target encoder to obtain event-level representations [zy1,…,zyN][z_{y_{1}}, \ldots, z_{y_{N}}]. We sample MM contiguous target spans from these representations. The (i)(i)-th target span is defined by the set of event positions BiB_{i}, giving the corresponding target representations zyj,j∈Biz_{y_{j}}, j \in B_{i}. The target events are removed from the input leaving the remaining

Diagram of the JEPA architecture and target constructions

Figure 1. The context sequence is encoded by the online encoder, while the complete target sequence is processed by an exponential-moving-average (EMA) target encoder with gradients stopped through the target branch. A predictor combines the context representations with learnable mask embeddings to estimate target latents, which are matched to the EMA target representations using mean squared error. We evaluate three target constructions: contiguous span masking, prospective prediction of future events, and random masking of individual events.

EHR as a one-dimensional sequence of NN clinical events. The complete uncorrupted sequence is passed through the exponential moving average (EMA) target encoder to obtain event-level representations [zy1,…,zyN][z_{y_{1}}, \ldots, z_{y_{N}}]. We sample MM contiguous target spans from these representations. The (i)(i)-th target span is defined by the set of event positions BiB_{i}, giving the corresponding target representations zyj,j∈Biz_{y_{j}}, j \in B_{i}. The target events are removed from the input leaving the remaining events as context. The context encoder produces a set of event representations, which are passed to a shallow two-layer transformer predictor together with learnable mask tokens containing positional information for the target locations. As in I-JEPA, the predictor is applied separately for each target span, producing z^yj\hat{z}_{y_j}, j∈Bij \in B_i. The prediction objective is the mean squared distance between the predicted and target representations across the MM target spans:

LJEPA=1M∑i=1M1∣Bi∣∑j∈Bi∥z^yj(i)−zyj∥22.(2)\mathcal{L}_{\mathrm{JEPA}} = \frac{1}{M} \sum_{i=1}^{M} \frac{1}{\lvert B_i \rvert} \sum_{j \in B_i} \left\lVert\hat{z}_{y_j}^{(i)} - z_{y_j} \right\rVert_2^2. \tag*{(2)}

While span masking provides the closest analogue to I-JEPA, EHRs are inherently temporal, and clinically meaningful information may instead lie in how the patient evolves over time. We therefore consider a prospective formulation, Future JEPA, motivated by temporally predictive JEPA and world-model approaches [11]. All events observed up to a time point TT form the context, while events satisfying [T<tj≤T+24,h][T < t_j \le T + 24, \mathrm{h}] form a single future target region B1B_1. This is equivalent to setting M=1M = 1 in the JEPA objective above. The architecture and loss are otherwise unchanged.

Finally, we evaluate a random-mask JEPA formulation, Random JEPA, that uses the same random sampling strategy as MLM, but optimises the JEPA objective with M=1M = 1. This provides a controlled comparison with MLM under the same corruption scheme, differing only in whether the model predicts discrete medical codes or continuous target representations.

Together, these objectives test three increasingly different assumptions about predictable structure in EHRs: (1) whether latent clinical information can be recovered from neighbouring events, (2) whether it can be predicted prospectively from earlier patient history, and (3) whether latent reconstruction offers an advantage over token reconstruction when the corruption pattern itself is held fixed.

For all JEPA experiments, we use an EMA target encoder with stop-gradient, rather than the shared-encoder SIGReg alternative [17]. In preliminary experiments, SIGReg was unstable and collapsed, with substantially reduced target-embedding variance. The EMA formulation remained stable and non-collapsed, allowing us to assess the JEPA objective without conflating downstream performance with optimisation failure. Standard MLM is retained as the primary baseline.

Experiments and Results

Each objective is trained for 50 epochs on the full MIMIC-IV training cohort using the same optimisation setup. Our downstream tasks include ICU mortality, ICU readmission, hospital mortality, and hospital readmission using the same 80:10:10 patient-level split as pretraining. Representation quality is assessed using frozen-encoder linear probes with mean pooling, using the checkpoint with the lowest validation pretraining loss, reporting AUROC and AUPRC. The full hyperparameter sweep is provided in Appendix § 5.6.

MLM outperformed all JEPA formulations across downstream tasks, with Future JEPA performing closest to MLM, followed by the two others (Table 1). To determine whether this performance gap was caused by representational collapse, we additionally tracked effective rank and feature standard deviation of the target-encoder embeddings throughout pretraining [18]. Both remained stable while training and validation prediction losses decreased (Appendix § 5.3, Figure 3), indicating that the JEPA models converged to non-collapsed solutions despite learning representations that were less useful for downstream clinical prediction.

LossMaskingICU Mortality AUROCICU Mortality AUPRCICU Readmission AUROCICU Readmission AUPRCHospital Mortality AUROCHospital Mortality AUPRCHospital Readmission AUROCHospital Readmission AUPRC
MLMRandom0.7740.2410.6950.2690.8780.2660.7200.416
JEPASpan0.7150.1720.6420.2190.8450.2100.7000.401
JEPAFuture0.7570.2160.6810.2590.8700.2490.7080.409
JEPARandom0.7150.1790.6380.2150.8410.2140.6910.390

Table 1. Performance (AUROC and AUPRC) on downstream clinical prediction tasks on MIMIC-IV for different pretraining objectives and masking strategies.

Training dynamics for the three JEPA formulations over 50 epochs

Figure 3. Training dynamics for the three JEPA formulations over 50 epochs. Rows correspond to Span, Future, and Random JEPA, while columns show prediction loss, effective rank (RankMe) of the target-encoder embeddings, and feature standard deviation. Training and validation prediction losses decrease throughout pretraining, while effective rank and feature variance remain non-degenerate, providing evidence that the models do not undergo representational collapse.

We further investigated which properties of an EHR target were associated with latent-prediction difficulty. We performed a post-hoc multivariable regression on per-target prediction error using target size, unique-code fraction, event-type diversity, target–context code overlap, and code rarity as covariates (Appendix 5.4). Target size, a key design parameter in I-JEPA [10], was the only factor consistently positively associated with prediction error across all three formulations. In contrast, unique-code fraction was negatively associated with prediction error, while event-type diversity had comparatively small effects. To investigate the effect of target size, we varied the masking ratio to alter target size for Span JEPA. Although smaller targets reduced prediction difficulty, this produced no consistent improvement in downstream performance (Figure 2).

Sensitivity of Span-JEPA downstream performance to target masking ratio across the four downstream clinical prediction tasks

Figure 2. Sensitivity of Span-JEPA downstream performance to target masking ratio across the four downstream clinical prediction tasks.

Discussion

Direct JEPA-style latent prediction can be optimised successfully on longitudinal EHRs without representational collapse, yet the resulting representations remain weaker than MLM on the evaluated downstream tasks. The fact that reducing target size also fails to improve downstream performance suggests that the limitation is not simply optimisation difficulty, but may instead lie in how the prediction target is defined.

A likely explanation is a mismatch in the semantic unit of prediction. In I-JEPA, image patches are relatively low-level observations, and predicting larger regions can encourage abstraction toward higher-level structure [10]. However, EHR tokens already have a very specific semantic meaning associated to them. Predicting masked event-level latents requires reconstructing the semantic meaning of each event, rather than definitely encouraging compression into a coherent patient-state representation. In this setting, increasing the amount of masked information does not necessarily increase semantic abstraction in the same way that predicting larger image regions can. By contrast, MLM retains an explicit token-level objective that directly preserves event identity, which itself is clinically informative. This may also explain why recent JEPA-style EHR work retains explicit token-level supervision alongside latent prediction [19].

The stronger performance of Future JEPA suggests that temporal prediction may provide a more appropriate inductive bias than reconstructing arbitrary internal targets. However, future EHR observations are only partially determined by latent patient state and are also shaped by clinical decisions. The target may therefore contain information about how care is delivered as well as how the patient evolves. Irregular sampling further complicates target construction, since a fixed number of events can span very different clinical intervals, while a fixed temporal window can contain widely varying amounts of information. Consequently, neither token contiguity nor temporal proximity necessarily corresponds to a coherent patient-state target.

These observations suggest that an EHR-specific JEPA may need to operate at a higher semantic level. Rather than predicting one latent representation per recorded event, groups of events could first be aggregated into a smaller number of patient-state representations, with the predictive objective modelling transitions between these states over time. Such targets could better separate persistent clinical state from individual observations, potentially matching the abstraction motivating JEPA more closely.

Overall, our findings suggest that the central challenge in transferring JEPA to longitudinal EHRs is not simply preventing collapse or tuning target difficulty, but defining a prediction unit that corresponds to clinically meaningful patient state. In the formulations studied here, successful latent prediction alone is insufficient to outperform token-level reconstruction.

Appendix

Responsible Use

This study evaluates JEPA techniques for retrospective prediction from longitudinal EHRs and does not establish suitability for clinical deployment. Experiments are limited to MIMIC-IV, and performance may not generalise across institutions, patient populations, or care settings. Generative predictions may exhibit uncertainty, miscalibration, and clinically consequential errors. Before any clinical application, models would require external validation, subgroup fairness assessment, calibration analysis, and prospective evaluation under appropriate clinical oversight. Outputs should not be used to guide patient care without such validation and human supervision and as such, this work should be used for research purposes only.

Acknowledgements

This work is supported by UK Research and Innovation (UKRI AI Centre for Doctoral Training in Digital Healthcare grant number EP/Y030974/1), NIHR Imperial Biomedical Research Centre and the NIHR Invention for Innovation grant “Artificial Intelligence to support cancer early diagnosis in general practice" (NIHR207533).

Training Dynamics

Regression

To investigate which properties of an EHR target are associated with latent-prediction difficulty, we perform a post-hoc regression analysis on the validation set for each JEPA formulation. For every target, we record the JEPA mean squared prediction error (MSE) together with structural characteristics of the target and its relationship to the available context. Because the per-target MSE distribution is strongly right-skewed, we use log⁡(1+MSE)\log(1 + \mathrm{MSE}) as the regression outcome. All predictor variables are standardised to zero mean and unit variance before fitting, allowing their coefficients to be compared on a common scale. We fit ordinary least-squares regression models using HC3 heteroskedasticity-robust standard errors and report 95% confidence intervals.

ModelPredictorStd. Coef.95% CIRobust pp-valueVIF
Span JEPATarget size0.121[0.120, 0.122]< 0.0013.03
Unique-code fraction−0.013[−0.015, −0.012]< 0.0013.00
Event-type diversity0.001[0.001, 0.002]< 0.0011.56
Target–context code overlap−0.014[−0.015, −0.013]< 0.0012.28
Code rarity−0.012[−0.012, −0.011]< 0.0011.50
Future JEPATarget size0.067[0.065, 0.068]< 0.0013.32
Unique-code fraction−0.069[−0.071, −0.068]< 0.0012.91
Event-type diversity−0.001[−0.002, 0.001]0.3791.86
Target–context code overlap−0.021[−0.022, −0.020]< 0.0011.96
Code rarity−0.023[−0.025, −0.022]< 0.0011.56
Random JEPATarget size0.060[0.059, 0.062]< 0.0016.37
Unique-code fraction−0.040[−0.041, −0.039]< 0.0016.56
Event-type diversity−0.011[−0.011, −0.010]< 0.0011.26
Target–context code overlap−0.018[−0.018, −0.017]< 0.0013.11
Code rarity−0.009[−0.010, −0.008]< 0.0011.20

Table 2. Detailed regression results for JEPA prediction error. Coefficients are standardised; confidence intervals and pp-values use HC3 robust standard errors.

The regression covariates include target size, unique-code fraction, event-type diversity, target–context code overlap, and code rarity. Target size is defined as the number of events being predicted. Unique-code fraction is the number of unique medical codes in the target divided by the total number of target events. Event-type diversity is measured using normalised entropy over the event types present in the target. Target–context code overlap is the proportion of unique target codes that are also observed in the context, while code rarity is calculated as the mean negative log-frequency of the target codes using frequencies estimated from the pretraining set. We assess multicollinearity using pairwise correlations and variance inflation factors (VIFs). The regression can be written schematically as

log⁡(1+MSEi)=β0+β1log⁡(1+Ntarget,i)+β2Ui+β3Hi+β4Oi+β5Ri+ϵi(3)\log(1 + \mathrm{MSE}_i) = \beta_0 + \beta_1 \log(1 + N_{\mathrm{target},i}) + \beta_2 U_i + \beta_3 H_i + \beta_4 O_i + \beta_5 R_i + \epsilon_i \tag*{(3)}

where Ntarget,iN_{\mathrm{target},i} is target size, UiU_i is unique-code fraction, HiH_i is event-type diversity, OiO_i is target–context code overlap, RiR_i is code rarity.

Context and Target selection

For span masking, let NN be the number of real (non-padding) events in the training window, and let rr be the mask ratio. The masked-token budget is B=⌈Nr⌉B = \lceil Nr \rceil, clipped to [0,N][0, N]. The number of spans is 4 when B≥40B \ge40, and max⁡(1,⌊B/10⌋)\max(1, \lfloor B/10 \rfloor) otherwise. Each span has length has a rough length of ⌊B/S⌋\lfloor B/S \rfloor, where SS is the number of spans, and the minimum span length is 10 tokens. Its starting position is drawn independently and uniformly among the positions where a block of that length fits, so spans may overlap. Events that fall in no span are the context.

For future prediction, timestamps are measured in hours from the first event in the same window. The cut TT is drawn uniformly from event times that leave at least 10 context events at times ≤T\le T and at least 8 target events in (T,T+24h](T, T + 24\mathrm{h}]. Events after T+24hT + 24\mathrm{h} are neither context nor target. If no such cut exists, the last 10% of events in the window are the target and the preceding events are the context.

Hyperparameters

Table 3: Hyperparameter sweep used during pretraining for the three JEPA objectives. Transformer and predictor settings were held fixed.

HyperparameterValues
Learning rate1×10−51 \times10^{-5}, 5×10−55 \times10^{-5}, 1×10−41 \times10^{-4}, 5×10−45 \times10^{-4}
Mask ratio0.150.15, 0.300.30, 0.500.50
Batch size6464, 128128, 256256
EMA coefficient0.9960.996
Transformer (fixed)
Hidden size768768
Layers66
Attention heads88
Feed-forward width30723072
Dropout0.10.1
Context length10241024
Normalizationpre-norm, final LayerNorm
Position encodingRoPE
Attentionbidirectional
Predictor (fixed)
Hidden size768768
Layers22
Attention heads88
Feed-forward width30723072
Dropout0.10.1
Position encodingRoPE
Normalizationpre-norm, final LayerNorm

Table 3.

EMA update: θtgt←0.996θtgt+0.004θcontext\theta_{\mathrm{tgt}} \leftarrow0.996\theta_{\mathrm{tgt}} + 0.004\theta_{\mathrm{context}}

MIMIC Dataset Information

Metric / TaskTrainValidationTest
Count%Count%Count%
Unique Patients291,70280.0%36,46310.0%36,46210.0%
ICU Mortality
Case (Positive)5,8817.8%7057.7%7157.7%
Control (Negative)69,16992.2%8,50892.3%8,62892.3%
ICU Readmission
Case (Positive)10,74514.2%1,34514.5%1,35014.3%
Control (Negative)64,97885.8%7,94785.5%8,07985.7%
Hospital Mortality
Case (Positive)8,0612.9%1,0142.9%9882.8%
Control (Negative)271,05697.1%34,27197.1%33,84097.2%
Hospital Readmission
Case (Positive)86,85920.4%11,07420.6%11,48021.2%
Control (Negative)339,55979.6%42,65979.4%42,67678.8%

Table 4.

Linear Probing

ModelICU MortalityICU ReadmissionHospital MortalityHospital Readmission
AUROC
MLM0.774 [0.757, 0.792]0.695 [0.677, 0.712]0.878 [0.867, 0.889]0.720 [0.711, 0.728]
Span JEPA0.715 [0.696, 0.734]0.642 [0.625, 0.658]0.845 [0.832, 0.858]0.700 [0.690, 0.711]
Future JEPA0.757 [0.739, 0.775]0.681 [0.663, 0.697]0.870 [0.859, 0.882]0.708 [0.698, 0.719]
Random JEPA0.715 [0.696, 0.734]0.638 [0.622, 0.653]0.841 [0.827, 0.854]0.691 [0.681, 0.702]
AUPRC
MLM0.241 [0.214, 0.274]0.269 [0.246, 0.294]0.266 [0.231, 0.306]0.416 [0.398, 0.436]
Span JEPA0.172 [0.154, 0.198]0.219 [0.202, 0.238]0.210 [0.178, 0.249]0.401 [0.376, 0.428]
Future JEPA0.216 [0.193, 0.247]0.259 [0.238, 0.285]0.249 [0.214, 0.289]0.409 [0.383, 0.437]
Random JEPA0.179 [0.159, 0.204]0.215 [0.199, 0.232]0.214 [0.182, 0.252]0.390 [0.367, 0.416]

Table 5. Downstream linear-probe performance on AUROC and AUPRC with brackets denoting 95% percentile bootstrap confidence intervals from 1,000 stratified resamples

References

  1. [1]Ethan Steinberg, Ken Jung, Jason A. Fries, Conor K. Corbin, Stephen R. Pfohl, and Nigam H. Shah. Language Models Are An Effective Patient Representation Learning Technique For Electronic Health Record Data. 5 2020.
  2. [2]Pawel Renc, Yugang Jia, Anthony E. Samir, Jaroslaw Was, Quanzheng Li, David W. Bates, and Arkadiusz Sitek. Zero shot health trajectory prediction using transformer. npj Digital Medicine, 7(1):256, 9 2024. ISSN 2398-6352. doi: 10.1038/s41746-024-01235-0.DOI
  3. [3]Artem Shmatko, Alexander Wolfgang Jung, Kumar Gaurav, Søren Brunak, Laust Hvas Mortensen, Ewan Birney, Tom Fitzgerald, and Moritz Gerstung. Learning the natural history of human disease with generative transformers. Nature, 9 2025. ISSN 0028-0836. doi: 10.1038/s41586-025-09529-3.DOI
  4. [4]Haresh Rengaraj Rajagopalan, Xiang Gao, Weicheng Zhu, Shih-Lun Huang, Long Chen, Kyunghyun Cho, Cem M. Deniz, and Narges Razavian. Foundation Models for Clinical Records at Health System Scale. 7 2025.
  5. [5]Yikuan Li, Shishir Rao, Jose Roberto Ayala Solares, Abdelaali Hass aine, Dexter Canoy, Yajie Zhu, Kazem Rahimi, and Gholamreza Salimi-Khorshidi. BEHRT: Transformer for Electronic Health Records. 7 2019.arxiv.org/abs/1907.09538
  6. [6]Yikuan Li, Mohammad Mamouei, Gholamreza Salimi-Khorshidi, Shishir Rao, Abdelaali Hass aine, Dexter Canoy, Thomas Lukasiewicz, and Kazem Rahimi. Hi-BEHRT: Hierarchical Transformer-based model for accurate prediction of clinical events using multimodal longitudinal electronic health records. 6 2021.arxiv.org/abs/2106.11360
  7. [7]Laila Rasmy, Yang Xiang, Ziqian Xie, Cui Tao, and Degui Zhi. Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction. npj Digital Medicine, 4(1):86, 5 2021. ISSN 2398-6352. doi: 10.1038/s41746-021-00455-y.DOI
  8. [8]Chao Pang, Xinzhuo Jiang, Krishna S Kalluri, Matthew Spotnitz, RuiJun Chen, Adler Perotte, and Karthik Natarajan. CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction tasks. 11 2021.arxiv.org/abs/2111.08585
  9. [9]Yann LeCun. A Path Towards Autonomous Machine Intelligence. OpenReview, 2022. URL https://openreview.net/pdf?id=BZ5a1r-kVsf.
  10. [10]Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture. 4 2023.arxiv.org/abs/2301.08243
  11. [11]Mido Assran, Adrien Bardes, David Fan, Quentin Garrido, Russell Howes, Mojtaba, Komeili, Matthew Muckley, Ammar Rizvi, Claire Roberts, Koustuv Sinha, Artem Zholus, Sergio Arnaud, Abha Gejji, Ada Martin, Francois Robert Hogan, Daniel Dugas, Piotr Bojanowski, Vasil Khalidov, Patrick Labatut, Francisco Massa, Marc Szafraniec, Kapil Krishnakumar, Yong Li, Xiaodong Ma, Sarath Chandar, Franziska Meier, Yann LeCun, Michael Rabbat, and Nicolas Ballas. V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning. 6 2025.arxiv.org/abs/2506.09985
  12. [12]Lucas Maes, Quentin Le Lidec, Damien Scieur, Yann LeCun, and Randall Balestriero. LeWorld-Model: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels. 6 2026.
  13. [13]Hai Huang, Yann LeCun, and Randall Balestriero. LLM-JEPA: Large Language Models Meet Joint Embedding Predictive Architectures. 10 2025.arxiv.org/abs/2509.14252
  14. [14]Rafi Al Attrach, Rajna Fani, David Restrepo, Yugang Jia, and Peter Schüffler. Rethinking Tokenization for Clinical Time Series: When Less is More. 12 2025.
  15. [15]Maria Elkjaer Montgomery, Christian Igel, Mikkel Fruelund Odgaard, Martin Sillesen, and Mads Nielsen. How Should Transformers Encode Numeric Values in Electronic Health Records? In Forty-third International Conference on Machine Learning, 2026. URL https://openreview.net/forum?id=YzIscRoNUj.
  16. [16]Lin Lawrence Guo, Santiago Eduardo Arciniegas, Joseph Jihyung Lee, Adam Paul Yan, George Tomlinson, Jason Fries, and Lillian Sung. Tokenization Tradeoffs in Structured EHR Foundation Models. 3 2026.
  17. [17]Randall Balestriero and Yann LeCun. LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics. 11 2025.arxiv.org/abs/2511.08544
  18. [18]Quentin Garrido, Randall Balestriero, Laurent Najman, and Yann Lecun. RankMe: Assessing the Downstream Performance of Pretrained Self-Supervised Representations by Their Rank. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 10929–10974. PMLR, 8 2023. URL https://proceedings.mlr.press/v202/garrido23a.html.
  19. [19]Irsyad Adam, Zekai Chen, David Laprade, Shaun Porwal, David Laub, Erik Reinertsen, Arda Pekis, and Kevin Brown. The Patient is not a Moving Document: A World Model Training Paradigm for Longitudinal EHR. 1 2026.

Paper details

vlnc:W_rn62uhrc
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 8

downloads · 1

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents