betaFeedback

WorksComputer Science

Where to Predict in Retinal OCT? Anatomy-Guided Target Selection for I-JEPA

Abstract

Self-supervised learning aims to learn meaningful representations from unlabeled data, potentially allowing models to capture subtle anatomical variations across patients and reduce the amount of labeled data needed. Image-based Joint-Embedding Predictive Architecture (I-JEPA) learns by predicting target-region representations from visible context rather than reconstructing pixels. However, its uniform target placement does not account for the anatomical structure of retinal OCT, where retinal tissue occupies only part of each B-scan. We investigate anatomy-guided target selection using image intensity and predicted retinal segmentation while retaining the I-JEPA architecture and latent prediction objective. We compare anatomy-guided rectangular targets, anatomy-shaped targets and coverage-constrained placement with uniform masking, and evaluate the learned representations using linear probing for glaucoma classification on FairVision. We found that rectangular targets guided by image intensity achieved the highest linear-probe AUC in our comparisons. We show that anatomically meaningful target selection is promising for retinal OCT representation learning, while greater anatomical specificity did not consistently improve performance.

Introduction

Optical coherence tomography (OCT) visualizes retinal layers and supports glaucoma assessment (Maetschke et al., 2019; Ran et al., 2019; Noury et al., 2022). Self-supervised pretraining is widely used in medical imaging foundation models to learn transferable representations and improve downstream performance [2] (Zhou et al. 2023; Qiu et al. 2024). Image-based Joint-Embedding Predictive Architecture (I-JEPA) in particular has gained interest with extension to both robotics and medical images [16], motivated by learning more semantically meaningful features by predicting target representations instead of pixels. The spatial organization of OCT makes the choice of pretraining task particularly relevant: retinal tissue forms a relatively narrow region within a B-scan, surrounded by the vitreous (the transparent gel anterior to the retina) and other low-intensity background regions. Our interest in anatomical guidance also arose from exploratory feature-occlusion maps of glaucoma classifiers initialized from uniform-masking I-JEPA and then fine-tuned with diagnosis labels, in which selected maps suggested prediction sensitivity near retinal structures (Appendix § 13).

Standard I-JEPA places rectangular targets uniformly across the image. In retinal OCT, many such targets include little retinal tissue. This raises our central question: can I-JEPA benefit from selecting prediction targets according to retinal anatomy rather than uniformly across the image? Figure 1 illustrates the change in target placement. Semantic and anatomy-guided masked modelling provide positive motivation for this question (Li et al., 2022; Kakogeorgiou et al., 2022; Li et al., 2024; Wang et al., 2025). However, selecting a target also removes information from the context. The useful learning task depends on both what is predicted and what remains visible.

Top: placement guides. Bottom: target masks.

Figure 1. Top: placement guides. Bottom: target masks. Colours and numbers identify target groups; overlaps are allowed. Illustrative B-scan adapted from Kulyabin et al., OCTDL (arXiv:2312.08255v3), under CC BY 4.0; overlays added.

GenAI for Health Workshop @ NeurIPS 2026, Sydney.

We investigate this question through five masking families: uniform RANDOM, the segmentation-free CENTROID, segmentation-guided ENVELOPE, anatomy-shaped targets (two versions) and coverage-constrained COVER, the last three guided by MIRAGE retinal segmentation (Morano et al., 2025). They vary target location, shape or coverage within the unchanged I-JEPA architecture and latent objective. We evaluate the resulting B-scan representations by pooling them across an OCT volume and fitting a linear glaucoma classifier. We first pretrain our model for 25 epochs as initialization. Then we apply different masking strategies for the rest of pretraining, and evaluate glaucoma classification performance. We found that simpler methods based on image intensity outperform masking methods using more detailed anatomical guidance.

Our contributions are as follows:

  • An empirical comparison of intensity-guided, segmentation-guided and uniform target selection for I-JEPA on retinal OCT.

  • Linear-probe results showing higher glaucoma classification AUC with anatomy-guided rectangular targets in the original runs, while more anatomically specific strategies did not consistently improve performance.

Related work

Masked image modelling and JEPAs. Masked image modelling learns visual representations by predicting hidden content from visible context. Reconstruction-based approaches define this prediction in terms of pixels or discrete visual tokens [13, 5] (Xie et al. 2022). A complementary line of work uses teacher outputs or latent representations as prediction targets, moving the learning task into representation space [3, 1] (Zhou et al. 2022). Masked prediction has also been explored in video and volumetric data [6, 10] (Taleb et al. 2020), while JEPA variants investigate different objectives and prediction structures [4, 14]. Alongside these choices, target selection determines which image regions the model learns to predict. Our study focuses on this spatial aspect in retinal OCT, retaining the I-JEPA objective while guiding target selection using retinal anatomy.

Region selection and prediction structure. DSeq-JEPA combines attention-derived region selection with sequential prediction [14]. Its highest-ranked region supplies context for subsequent predictions, and its ablation favours the joint design over region selection alone. We instead use retinal anatomy with parallel I-JEPA prediction, where the guidance map, selected targets and available context jointly define the learning task. I-JEPA’s own masking analysis emphasizes both sufficiently large targets and informative context [1].

Informed masking. Mask selection can shape self-supervision by emphasizing particular image regions. Existing approaches derive masks from learned policies (Shi et al., 2022), semantic or attention-based guidance (Li et al., 2022; Kakogeorgiou et al., 2022), or prediction difficulty [9] (Wang et al. 2023). In medical imaging, anatomical structure also guides masking [8, 15] (Li et al. 2024). These approaches motivate anatomy-guided target selection, although their prediction objectives, guidance sources and context construction differ from those studied here.

Medical and retinal self-supervised learning. Medical pretraining can incorporate modality-specific structure into region selection. US-JEPA uses a frozen ultrasound teacher, with a region-conditioned variant restricting context and target sampling to valid ultrasound content (Radhachandran et al., 2026). In retinal imaging, foundation models pretrain on large retinal corpora (Zhou et al., 2023; Qiu et al., 2024; Liu et al., 2026), while OCT-specific work explores masked image modelling (Pissas et al., 2024). We use image intensity and MIRAGE retinal segmentation (Morano et al., 2025) to guide target selection within the EMA-based I-JEPA framework, retaining its architecture and parallel prediction structure.

Background and target-selection framework

We first define the I-JEPA prediction task, then describe how each masking strategy selects its targets. The architecture and latent prediction loss are shared; differences in guide sources, target size and final context are part of the strategy comparisons.

I-JEPA preliminaries

Let xx be a B-scan partitioned into NN patches, and let CC denote the indices of the patches supplied to the context encoder. Target group TmT_m contains KmK_m patch indices, with m=1,…,Mm = 1, \ldots, M. We treat TmT_m as an index list because some implementations repeat indices when equalizing target lengths. The context excludes the union of target positions. Writing pCp_C and pTmp_{T_m} for the corresponding positional embeddings, the context and prediction paths are

hC=fθ(x;C),z~m=gϕ(hC,pC,pTm).(1)h_C = f_\theta(x; C), \qquad\tilde{z}_m = g_\phi(h_C, p_C, p_{T_m}). \tag*{(1)}

Here fθ(x;C)f_\theta(x; C) embeds only the selected patches at their original image positions. The predictor projects hCh_C to its own feature dimension and adds its own positional embeddings pCp_C; target queries combine a learned mask token with pTmp_{T_m}. The target encoder sees the full B-scan. Its output is normalized per patch before selecting the target positions:

zm=sg⁡([LN⁡(fθˉ(x))]Tm),θˉ←μθˉ+(1−μ)θ.(2)z_m = \operatorname{sg}\left(\left[\operatorname{LN}(f_{\bar{\theta}}(x))\right]_{T_m}\right), \qquad\bar{\theta} \leftarrow\mu\bar{\theta} + (1-\mu)\theta. \tag*{(2)}

The operator sg⁡\operatorname{sg} stops gradients, LN is LayerNorm across feature dimensions, and μ\mu is the EMA coefficient. The predictor uses the same context for each target group; predictions are not conditioned on earlier target predictions.

Each target and prediction is a Km×DK_m \times D matrix. For feature dimension DD, the implementation minimizes the mean Smooth-L1 loss over target tokens and feature coordinates:

L=1D∑m=1MKm∑m=1M∑k=1Km∑d=1Dρ(z~m,k,d−zm,k,d).(3)\mathcal{L} = \frac{1}{D\sum_{m=1}^{M}K_m}\sum_{m=1}^{M}\sum_{k=1}^{K_m}\sum_{d=1}^{D}\rho(\tilde{z}_{m,k,d}-z_{m,k,d}). \tag*{(3)}

The Smooth-L1 function ρ\rho uses the standard unit transition between its quadratic and linear regions. We call the ∑mKm\sum_m K_m supervised target tokens loss slots; repeated or overlapping indices add loss slots, and the mean is taken over all loss slots of a batch (Appendix D). Figure 2 shows the pretraining and evaluation paths.

Overview of OCT pretraining and glaucoma classification

Figure 2. Overview of OCT pretraining and glaucoma classification. (a) Each OCT volume contains 200 two-dimensional slices (B-scans); we select 100 evenly spaced slices. (b) For each slice, the context encoder processes visible patches, and a predictor estimates the features of masked target regions. Standard I-JEPA uses randomly placed rectangles; our strategies change target selection. A Smooth-L1 loss compares these predictions with features from a target encoder that sees the full slice. The loss trains the context encoder and predictor, while the target encoder follows a moving average of the context encoder’s weights. (c) For evaluation, the trained target encoder is frozen. Features are averaged across patches and slices, and a linear classifier predicts glaucoma for the volume. The illustrative B-scan is sourced as in Figure 1.

Guided target selection

Background patches above and below the retina often have low intensity variation, whereas retinal patches carry the layered structure. Guided strategies direct targets toward retinal structures so that prediction depends more on them. However, even a visually uniform background patch receives a positional embedding, and its latent representation can incorporate tissue information through attention. Such patches may therefore still provide useful prediction targets or visible context. For strategy ss, let As(x)A_s(x) denote its patch-grid guidance: an intensity-derived band for CENTROID, a repaired segmentation envelope for ENVELOPE, or the InnerRetina and choroid soft score maps for the anatomy-shaped and coverage strategies. CENTROID is an intensity-guided retinal localization baseline, not a segmentation model. We write the joint masking procedure as

(C,T)∼Πs(⋅∣As(x)),T=(T1,…,TM).(4)(C,T) \sim\Pi_s(\cdot\mid A_s(x)), \qquad T=(T_1,\ldots,T_M). \tag*{(4)}

Πs\Pi_s includes target placement and context construction, including batch-dependent token equalization. A candidate context block is sampled, target positions are excluded, and token lists are shortened as needed for batching. Only the remaining context tokens enter fθf_\theta. Consequently, tissue outside the target union need not be visible to the encoder. We report unique targets (cells in the target union), tissue hidden (share of tissue cells inside that union), purity (share of target cells on tissue) and context (share of the grid supplied to the context encoder); the target union and the context need not be complementary.

Compared masking strategies

All policies request M=4M=4 target blocks and one context block (Table 1, Figure 1). Rectangle strategies share nominal scale and aspect-ratio ranges, but placement and collation can change unique targets and encoder-visible context. We denote the segmentation scores by Y^=sψ(x)\hat{Y}=s_\psi(x), with guide-specific class definitions (§ 4 and Appendix § 8). We follow Assran et al. [1] for rectangular mask sizes and ratio except for ANATOMY.

policyguide As(x)A_s(x)target shapeplacementtarget length
RANDOMnonerectanglesuniformvariable
ENVELOPEhard MIRAGE enveloperectanglesrejection-sampled on envelopevariable
CENTROIDintensity-centroid bandrectanglesweighted by band overlapvariable
ANATOMY-V1/V2soft MIRAGE scoresconnected componentsgrown on scoresfixed (KK)
COVERsoft MIRAGE scoresrectanglesgreedy coveragevariable

Table 1. Compared strategies. Variable-length rectangle targets are shortened to the shortest target in the batch (Appendix § 11). Delivered target and context budgets are reported in Table 3.

RANDOM (baseline). Masking strategies adopted from I-JEPA (Assran et al., [1]), where four rectangular masks are randomly placed and no image content is used for masking.

ENVELOPE (random-within-retina). We sample the same number of rectangles, using the same size and width-to-height ratio distributions as RANDOM. We then place them randomly where they overlap sufficiently with a repaired retinal envelope estimated by MIRAGE (Morano et al., 2025). The guide uses MIRAGE-Large fine-tuned for layer segmentation on GOALS circumpapillary B-scans (Fang et al., [12]). Its foreground Dice score of 0.925 on held-out GOALS images describes performance on GOALS, not FairVision, which lacks layer annotations and uses a different scan protocol. We combine the predicted inner-retinal and choroid regions and fill the unlabeled middle-retina gap in the original predicted label map, then transform the guide and image together.

CENTROID (segmentation-free). To account for variation in the vertical position of retinal tissue across FairVision B-scans, we first estimate the band center. Let xˉij\bar{x}_{ij} be the mean intensity of the patch in row ii and column jj, minus the smallest patch mean in the image. For columns with ∑ixˉij>ε\sum_i \bar{x}_{ij} > \varepsilon, where ε=10−6\varepsilon= 10^{-6}, the band center in column jj is the intensity-weighted row centroid

cj=∑iixˉij∑ixˉij.(5)c_j = \frac{\sum_i i\bar{x}_{ij}}{\sum_i \bar{x}_{ij}}. \tag*{(5)}

Columns with negligible total weight use the image-wide intensity centroid, falling back to the vertical grid center when the image-wide weight is also negligible. We denote the centers averaged over a three-column neighborhood by c~j\tilde{c}_j. A band of fixed height centered on c~j\tilde{c}_j across the central columns (Appendix § 8) weights the placement of four rectangular targets; it is not itself the target shape. No segmentation model or annotation of any kind is used.

ANATOMY-V1 / ANATOMY-V2. To align target shape with retinal anatomy, we construct target regions from the predicted retinal-class score maps. Connected components are grown on soft MIRAGE retinal-class scores from a frozen feature adapter (Appendix § 8), so target shape follows retinal anatomy. Each component is resampled to exactly K=16K=16 indices, subsampled when larger and topped up with repeated indices when smaller. v2 also bridges diagonal adjacencies.

COVER rr. This strategy aims to balance anatomical target coverage with the retention of retinal information in the visible context. Targets are chosen greedily to cover anatomy while leaving a fraction rr of supported soft guide mass outside the target union. The trained configuration uses r=0.21r=0.21. This condition does not guarantee the same fraction survives final encoder-context collation, and post-placement target truncation changes the intended coverage (Appendix § 11).

Experimental setup

Data. We use FairVision OCT volumes with binary glaucoma labels (Luo et al., 2024) and the fixed released splits. Training and validation contain 6,000 and 1,000 volumes, respectively. Each volume contributes 100 of its 200 B-scans, evenly spaced and resized to 256×256256 \times256 (Figure 2a). The test split contains 3000 volumes (1466 with glaucoma). Diagnosis labels are used only for downstream evaluation, not JEPA pretraining. Demographic attributes are not model inputs.

Pretraining. Patch-level I-JEPA, ViT-Base, 256×256256 \times256 crops, patch size 16, predictor depth 6, EMA target encoder. The original continuations fork from a shared epoch-25 checkpoint (linear-probe AUC 0.8487) of uniform-masking pretraining, with matched nominal optimizer, learning-rate and weight-decay schedules and effective batch size (512). The probability of guided placement rises linearly to full strength between epochs 25 and 30. The lineage of the early ANATOMY-V1 continuation is described in Appendix A. The encoder processes B-scans independently, without attention across B-scans. Table 1 and Appendix A list the remaining differences between strategies.

RANDOM, CENTROID, ENVELOPE and COVER reach epoch 100. ANATOMY-V2 reaches epoch 75; its epoch-100 result is unmeasured. We compare strategies at matched pretraining epochs, with epoch 50 as the common endpoint for the five main strategies. Excluded continuations are listed separately in Appendix A.

Segmentation guides. ENVELOPE uses a GOALS-tuned MIRAGE-Large model with a ConvNeXt decoder and four classes: Elsewhere (other retinal tissue and image background), retinal nerve fiber layer (RNFL), ganglion cell and inner plexiform layers (GCIPL), and choroid. Its foreground predictions are combined and repaired into a retinal envelope; its held-out GOALS foreground Dice is 0.925. The MIRAGE-Base soft guides used by ANATOMY and COVER combine RNFL and GCIPL as InnerRetina, retaining choroid separately. This model also has Elsewhere and ignored void output channels; only the InnerRetina and choroid soft score maps are retained for masking. Appendix A details the label harmonization and adapters.

Additional continuations and controls. The main comparison uses one continuation per strategy. We also report two additional completed continuations each of RANDOM, CENTROID and ENVELOPE from the same epoch-25 checkpoint, plus one uniform-placement control matched to CENTROID’s per-image target and context budgets. Seeds vary data order, crops and mask draws. These additional comparisons are descriptive and do not represent independent initializations. Checkpoint decisions used validation AUC. Details are in Appendix B.

Evaluation. The EMA target encoder is frozen and encodes each B-scan without cropping. We average patch features within each B-scan, then average across the volume’s B-scans and train a LayerNorm–linear head (Figure 2c). Each head uses a 50-epoch budget and is selected by validation AUC. We report test AUC with percentile bootstrap intervals and paired differences (10,000 class-stratified resamples, fitted heads fixed), together with DeLong tests ([11]). These intervals describe test-case sampling, not variation across pretraining runs. AUC differences and their confidence intervals are reported in percentage points (pp), unless otherwise stated. The original RANDOM, CENTROID and ENVELOPE probes used fp16 mixed precision because their evaluation configurations retained the default automatic mixed-precision setting; ANATOMY-V2 and COVER explicitly disabled it and used fp32. In Table 2, each difference uses a RANDOM baseline matched in epoch and probe precision. Repeating linear probing with the same frozen encoders in fp32 changed AUC by at most 2×10−42 \times10^{-4} (0.02 pp). Further precision-control results are described in Appendix H.

Results and analysis

These are strategy comparisons, not factorial ablations: each original strategy has one pretraining continuation and may change several aspects of the prediction task. Section 5.3 reports additional completed continuations.

Comparison of target-selection strategies

Anatomy-guided rectangles. At the common endpoint, epoch 50, ENVELOPE and the segmentation-free CENTROID exceed uniform masking by +1.20 pp (CI [+0.68, +1.73]) and +0.99 pp (CI [+0.51, +1.49]). Across the three evaluated epochs, gains ranged from 0.62–1.20 pp for ENVELOPE and 0.99–1.13 pp for CENTROID (Figure 3; multiple-comparison adjustment in Appendix G). These gains favour anatomy-guided target placement in the original runs. Both strategies also leave more of the grid visible to the context encoder than RANDOM (Table 3), so these comparisons do not separate target location from visible context. The gains do not require targets to contain only retinal tissue: the large rectangles also include background.

Original-run linear probe performance on FairVision

Figure 3. Original-run linear probe performance on FairVision. (a) Test AUC across pretraining epochs (truncated vertical axis). The star marks the shared epoch-25 checkpoint. Solid and dashed lines denote fp16 and fp32 probes. (b) Paired AUC differences (on the original AUC scale) from precision-matched RANDOM, with 10,000-resample percentile bootstrap intervals. Intervals reflect test-case sampling, not variation across pretraining runs.

strategytissue hidden purityunique targetsloss slotscontext (% of grid)AUC (ep. 50)
RANDOM53.2%30.9%113.1160.227.5 ± 0.10.8641
CENTROID59.7%38.2%102.5160.231.5 ± 0.40.8740
ENVELOPE70.7%42.1%110.2160.229.2 ± 0.50.8761
COVER73.3%40.1%120.1160.225.2 ± 0.40.8643
ANATOMY-V279.7%96.9%54.064.063.3 ± 0.40.8654

Table 3. Masks delivered at the training batch size of 64, measured by replaying the sampler code in each strategy’s trained configuration on 6,400 fixed training views. The masks used during the original training were not recorded. Values are means over 3 sampler seeds of 100 batches each, with crops and block sizes paired across strategies. The context column also gives the across-seed s.d. Tissue is a thresholded MIRAGE-derived proxy, not ground-truth annotation. Column definitions are given in Appendix § 8. AUC is at the matched epoch 50 (fp32 probes for COVER and ANATOMY-V2).

Anatomy-shaped targets. At the matched epoch 50, ANATOMY-V2 has lower AUC than ENVELOPE (Δ=−1.07\Delta= -1.07 pp; CI [−1.67-1.67, −0.46-0.46]). At epoch 50, the intervals for both ANATOMY-V2 and COVER versus RANDOM include zero.

Coverage-constrained placement. COVER is close to RANDOM at epoch 50 (+0.02 pp, CI [−0.50-0.50, +0.53]) but falls behind by epoch 100 (−1.68-1.68 pp, CI [−2.33-2.33, −1.05-1.05]), as its own AUC declines after epoch 73 while RANDOM keeps improving (Figure 3). Batching shortens its targets after placement, so the trained policy is not a controlled higher-coverage counterpart to ENVELOPE (Appendix § 11).

In the original single runs, the CENTROID–ENVELOPE difference at the final evaluated epoch is +0.47 pp (CI [+0.04, +0.91]). The paired CENTROID–ENVELOPE difference intervals include zero at epochs 50 and 75. A simple intensity-based guide is therefore a useful comparator for segmentation-guided methods.

Analysis of target and context masks

Table 3 reports the masks that reach the model. At the training batch size, CENTROID leaves 4.0 points more of the grid visible than RANDOM (pooled 95% interval over batches 3.6–4.4), and ENVELOPE 1.6 points more (1.1–2.1). With loss slots equal across the rectangle strategies, CENTROID’s rectangles overlap more and cover fewer unique targets than RANDOM’s (Figure 1). ENVELOPE differs least from RANDOM in target and context budgets but has the largest proportion of target patches within the retina among the rectangle strategies. Appendix § 8 gives the measurement protocol.

ANATOMY-V2 has a high proportion of target patches within the retina but a smaller target union and more total context than the rectangle strategies. Its fixed K=16K=16 indices per target give 4×16=644 \times16 = 64 loss slots, compared with 160.2 for each rectangle strategy, an explicit target-length choice rather than a property of irregular shapes. Appendix § 8 gives the measurement protocol and the per-image geometry before batching, and Appendix § 12 audits the coverage strategy in complete training-sized batches.

Additional continuations and matched-budget control

policytest AUCΔ\Delta (pp) (interval)ULDCpurity (%)
RANDOM0.8624—113.1160.247.170.536.0
RANDOM-CB0.8650+0.26 [−0.15, +0.66]102.2159.257.179.536.7
CENTROID0.8721+0.96 [+0.52, +1.42]102.5160.257.780.846.0
CENTROID − RANDOM-CB+0.70 [+0.30, +1.11]placement at matched budgets

Table 5. Matched-budget control at epoch 50, seed A, one continuation per arm. Δ\Delta (pp) is the paired difference from RANDOM with a test-case interval. Budgets are per image at the training batch size: unique targets (U), loss slots (L), duplicates (D) and context tokens (C). RANDOM and CENTROID come from the audit of Table 3, RANDOM-CB comes from its run’s mask statistics (3,740 batches, mean hidden-area residual 0.00) and RANDOM-CB equals its CENTROID shadow per image (0 mismatches in 960 image draws). Purity uses a hard MIRAGE-envelope tissue proxy (190 views), not comparable with Table 3.

Repeated continuations. Table 4 reports the original encoders and two additional completed continuations per strategy at epoch 50. These runs share the epoch-25 checkpoint and are reported descriptively.

RANDOMCENTROIDENVELOPEΔ\Delta (pp) CENTROIDΔ\Delta (pp) ENVELOPE
original (re-probed)0.86410.87520.8755+1.11+1.14
seed A0.86240.87210.8716+0.96+0.92
seed B0.86350.87540.8710+1.20+0.75

Table 4. Completed continuations at epoch 50: test AUC averaged over five probe-head seeds (fp32; head-seed s.d. at most 0.0011). Original encoders are re-probed with the same protocol. Δ\Delta (pp) is the difference from RANDOM of the same continuation. Comparisons are descriptive; intervals are in Table 7.

policyruntest AUChead s.d.val. AUCΔ\Delta (pp) test (interval)Δ\Delta (pp) val.analysis
RANDOMoriginal0.86410.00050.8467——original
RANDOMseed A0.86240.00050.8495——descriptive
RANDOMseed B0.86350.00080.8443——descriptive
CENTROIDoriginal0.87520.00060.8560+1.11+0.93original
CENTROIDseed A0.87210.00070.8547+0.96 [+0.52, +1.42]+0.52descriptive
CENTROIDseed B0.87540.00050.8566+1.20 [+0.77, +1.64]+1.23descriptive
ENVELOPEoriginal0.87550.00050.8597+1.14+1.30original
ENVELOPEseed A0.87160.00020.8525+0.92 [+0.39, +1.44]+0.30descriptive
ENVELOPEseed B0.87100.00110.8593+0.75 [+0.27, +1.24]+1.50descriptive
RANDOM-CBseed A0.86500.00040.8463+0.26 [−0.15, +0.66]−0.32control

Table 7. Every evaluated encoder at epoch 50. AUC is the mean over five probe-head seeds, with their s.d. Δ\Delta (pp) is the difference from RANDOM of the same seed with a paired, label-stratified test-case bootstrap interval (10,000 resamples), and the validation difference is given for comparison. Intervals describe test-case sampling, not retraining. All comparisons are descriptive.

Matched-budget control. RANDOM-CB receives CENTROID’s per-image unique targets, loss slots and visible context but places its targets uniformly. Its difference from CENTROID therefore reflects placement at equal budgets, and its difference from RANDOM reflects the change of budget. ENVELOPE is not budget-matched in this design. With these budgets, RANDOM-CB differs from RANDOM of the same seed by +0.26 pp. The placement difference, CENTROID minus RANDOM-CB, is +0.70 pp (descriptive). The comparison is a single continuation per arm, and the two policies differ only modestly in the proportion of target patches within the retina (46.0% versus 36.7%), so a null placement difference would be weak evidence that placement does not matter.

Pooling. We compare three alternative pooling schemes on seed A; the patch maximum within B-scans may retain localized responses that the patch mean attenuates. For seed A, the alternative poolings give differences from RANDOM ranging from +1.42–+1.88 pp for CENTROID and from +0.69–+1.84 pp for ENVELOPE (Table 8). These comparisons are exploratory.

poolingRANDOMCENTROIDENVELOPEΔ\Delta CENTROIDΔ\Delta ENVELOPE
mean, mean (primary)0.86240.87210.8716+0.96+0.92
max, mean0.84830.86250.8553+1.42+0.69
mean, max0.84490.86370.8633+1.88+1.84
mean, mean and max0.85530.87090.8719+1.56+1.66

Table 8. Pooling variants for seed A at epoch 50. Pooling is applied to patch tokens within a B-scan, then across B-scans. Test AUC is averaged over five probe-head seeds; Δ\Delta (pp) is the difference from RANDOM. Comparisons are exploratory.

Discussion and limitations

Anatomy-guided rectangular target placement improved glaucoma AUC in the original comparisons. Because the strategies also change target size, overlap and visible context, these results do not isolate the effect of target placement. Masking studies should report the budgets that reach the encoder at the training batch size, not only the requested ones (§ 5.2). Experimental limits. Each original strategy has one pretraining continuation, so its test-case confidence intervals do not establish a ranking over retrainings [7]. The additional experiments use continuations from the same epoch-25 checkpoint. They measure variation across continuations, not across independent initializations. The additional experiments include two completed continuations per primary strategy and one matched-budget control. Their limited number and shared checkpoint do not establish a ranking across independent pretraining runs. The early anatomy-shaped result has incomplete continuation documentation, and ANATOMY-V2 was stopped after inspecting its epoch-75 deficit. Its later trajectory is unknown. Probe optimization and initialization can also affect linear probe comparisons; the reported fixed-budget protocol is not a measurement of the best achievable classifier for each encoder. External foundation models such as RETFound under the same linear probe, other self-supervised objectives such as masked image modelling, and target-selection rules derived from a formal objective are left for future work. The study uses one dataset and one downstream task. Strategies, checkpoints and analyses were chosen after repeated inspection of the same test split, although probe heads were fitted on training data and selected on validation data. Reported intervals are therefore descriptive rather than confirmatory. The additional continuations use the same test split and therefore do not provide an independent external evaluation. Validation-split differences are reported alongside every per-seed test difference (Appendix § 9). FairVision has no layer annotations, so guide accuracy cannot be measured on it. On 300 stratified training volumes, the CENTROID band disagreed grossly with the MIRAGE envelope in 4.8% of B-scans, more often in glaucoma and low-signal scans, and 3.8% of envelopes raised a plausibility flag (Appendix § 10). These are agreement statistics between two automatic guides, not anatomical accuracy. Only the specified coverage setting was pretrained, so we identify neither an optimal coverage level nor a mechanism for its AUC decline.

Broader impact and ethics. This study uses a public, de-identified retinal dataset under its release terms and involves no new recruitment or data collection, and no model reported here is clinically validated or intended for deployment (Appendix § 8).

Future work

Our model encodes B-scans separately and combines their features through mean pooling, without learning interactions between slices. Exploratory attentive probes also showed overfitting. Future work could retain individual slice features while learning relationships between neighboring B-scans before pooling, using V-JEPA-style prediction or 3D OCT models. Segmentation of complete retinal layers across a volume could provide anatomical guidance.

Conclusion

We investigated where I-JEPA should predict in retinal OCT. Anatomy-guided rectangle placement improved glaucoma AUC under linear probing in the original runs, including a simple intensity-centroid strategy. More anatomically specific targeting did not consistently improve these results, and the implementations differ in more than target location, including the context they leave visible. Comparisons that match guidance, target sizes and visible context across independent pretraining runs remain the next step.

Additional experimental details

Pretraining and linear evaluation. The input crop is 256×256256 \times256, with patch size 16 and a ViT-Base encoder. The predictor has depth 6 and hidden dimension 384. Pretraining uses AdamW with peak learning rate 0.00025, a five-epoch warmup, final learning rate 0.000001, weight decay scheduled from 0.04 to 0.4, and EMA coefficient from 0.996 to 1.0. The effective batch size is 512, formed from batches of 64 with gradient accumulation. Rectangle target scales are 0.15–0.20 of image area, aspect ratios 0.75–1.5, and context scales 0.85–1.0. Random resized crops use area scale 0.3–1.0. The guidance probability increases over the epoch-25 to epoch-30 continuation window; fixed-length anatomy targets use K=16K = 16 throughout. The frozen linear head uses AdamW, learning rate 0.0004, weight decay 0.05, batch size 256, seed 42, and a 50-epoch budget with validation-AUC selection and patience 15. Paired analyses rely on the documented shared split and deterministic loader ordering; historical prediction files lack independent case-ID manifests.

B-scan selection and pooling. Each FairVision volume stores 200 B-scans of 200 × 200 pixels. Pretraining and evaluation use the same 100 B-scans per volume, with indices ⌊199k/99⌋\left\lfloor199k/99 \right\rfloor for k=0,…,99k = 0, \ldots, 99 (0, 2, …, 196, 199), each resized bilinearly to 256×256256 \times256 pixels. Pretraining samples every B-scan independently as a random resized crop. Evaluation encodes the full B-scans, averages the 256 patch features of each B-scan and then the 100 B-scan vectors (dimension 768) before the LayerNorm–linear head. The secondary pooling variants use the patch maximum within B-scans, the maximum across B-scans, or the concatenated mean and maximum across B-scans.

Guide sources. ENVELOPE uses a GOALS-tuned MIRAGE-Large model with a ConvNeXt segmentation decoder and four classes: Elsewhere, RNFL, GCIPL, and choroid. Elsewhere includes other retinal tissue and image background. The union of the three foreground classes is repaired across the unlabelled gap between the inner-retinal layers and choroid to form the hard retinal envelope. The anatomy-shaped and coverage strategies use precomputed soft guides from MIRAGE-Base with a ConvNeXt decoder and harmonized GOALS, Duke DME, and AROI annotations. GOALS and Duke DME RNFL/GCIPL labels are combined as InnerRetina, and AROI’s pre-combined inner-retinal band maps to the same class; choroid remains separate. The segmentation head emits four logits: Elsewhere, InnerRetina, choroid, and an ignored void channel. Softmax is applied over all four logits before retaining only the InnerRetina and choroid score maps for guidance. The ANATOMY-V1 guide uses a decoder-feature adapter; ANATOMY-V2 and COVER use an encoder-feature adapter. The models and adapters are frozen when these guides are generated and cached. These guide products are not independently controlled in the cross-strategy comparison. CENTROID uses image intensity rather than a segmentation model: in the trained configuration its band (Equation (5)) is 7 rows high and spans the central 10 columns of the patch grid (region fraction 0.28, lateral fraction 0.6). The adapters optimize feature-similarity objectives on FairVision-derived image caches, not a glaucoma classification loss. The guide models and adapters remain frozen during I-JEPA training. The retained provenance does not independently establish subject-level non-overlap for all data used to train the guidance models.

Included and excluded continuations. The early anatomy-v1 configuration resumes an ENVELOPE epoch-27 checkpoint rather than a documented epoch-25 fork, and its producing launch record is incomplete. Its only result is at epoch 30, where it lies +0.44 pp above ENVELOPE (CI [+0.09, +0.78]). This comparison inherits the lineage uncertainty. Anatomy-v2 results use the continuation with full-precision target-encoder outputs. Probes from its earlier mixed-target-precision continuation are excluded, as are earlier uniform controls that used different target precision and context selection. The main table retains the comparable strategy endpoints, including the adverse COVER trajectory.

Mask measurement. Table 3 replays the sampler code in each strategy’s trained configuration (commit 86ae8823f3afc9fa2c9ed6a3cb0c459b513ba1c5), including the CENTROID band above and ENVELOPE placement on the hard envelope with the fallback rule used in training. The scope is 256 training volumes with 25 views each; every strategy sees the same crops, block sizes and placement seeds. Tissue is guide occupancy of at least 0.25 on the soft MIRAGE envelope, on the 16 × 16 patch grid. Table 6 summarizes per-image and batched geometry. In Table 3, “tissue hidden” is the share of tissue cells in the target union, “purity” the share of target cells on tissue, “unique targets” the number of unique target cells, “loss slots” the supervised target tokens including duplicates, and “context” the share of the grid supplied to the context encoder; in Table 6, “mask ratio” is the number of unique target cells divided by NN. Repeated target indices add loss slots, and the anatomy-shaped and coverage strategies use different guide products from the ENVELOPE run.

strategytissue hiddenpuritymask ratiocontextloss slotscontext
RANDOM53.030.944.042.1159.327.5
CENTROID59.738.339.945.7159.331.5
ENVELOPE70.542.142.943.7159.329.2
COVER73.240.146.739.9159.325.2
ANATOMY-v279.796.821.168.064.063.3

Table 6. Mask geometry under the trained configurations and the protocol of Table 3. Per-image columns are measured one view at a time (6,400 single-view draws per seed), before batching; the last column is the context delivered at batch size 64.

Data use and ethics. This is secondary research on the public, de-identified FairVision release under its CC BY-NC-ND 4.0 data licence. No new participants were recruited or identifying fields added. A separate institutional ethics or exemption determination is not recorded for this project. The models are not clinically validated or intended for patient care. The illustrative OCT B-scan in Figures 1 and 2 comes from the separately attributed CC BY 4.0 OCTDL publication, not from the FairVision evaluation cohort.

Additional continuation and evaluation details

Two additional completed continuations each of RANDOM, CENTROID and ENVELOPE start from the shared epoch-25 checkpoint. Seeds set data order, crops, mask draws and data-loader streams; runs with the same seed share data order only. ENVELOPE retains the original fallback when no admissible window has low overlap: a uniform choice among admissible windows. A uniform-placement control receives CENTROID’s per-image budgets for seed A. Optimizer, schedule horizon and effective batch size match the original continuations, with full-precision teacher targets; each run stops at epoch 50. The original RANDOM and CENTROID continuations used different GPU hardware from the original ENVELOPE continuation and all additional runs.

Linear probing follows the main protocol in fp32, with five independently seeded linear heads fitted to each feature cache. We report their mean AUC and standard deviation; the original encoders are re-probed in the same way. Checkpoint decisions use validation AUC. All additional comparisons are descriptive. Differences from RANDOM of the same seed use paired test-case bootstrap intervals, which describe test-case sampling rather than independent pretraining runs. The pooling comparisons use seed A. Visual-field mean-deviation regression in glaucoma cases and validation-split differences are secondary analyses.

Table 7 lists every evaluated encoder with its paired interval and Table 8 the pooling variants, and Figure 4 shows the spread across runs. In glaucoma cases of the test split (n=1,466n = 1{,}466), ridge regression on the frozen features predicts visual-field mean deviation with a mean absolute error from 3.44 to 3.55 dB across encoders, against 4.65 dB for the training mean (Table 9). Mean deviation partly defines the glaucoma label, so this is a severity endpoint on the same cohort, not an independent task.

policyrunMAERMSESpearmanΔ\Delta MAE (interval)Δ\Delta Spearman
RANDOMseed A3.554.720.52——
RANDOMseed B3.494.640.53——
CENTROIDoriginal3.454.570.53——
CENTROIDseed A3.474.600.53−0.08 [−0.16, −0.00]+0.01
CENTROIDseed B3.464.580.54−0.03 [−0.10, +0.05]+0.01
ENVELOPEoriginal3.464.600.54——
ENVELOPEseed A3.504.640.54−0.05 [−0.13, +0.03]+0.02
ENVELOPEseed B3.444.600.53——
RANDOM-CBseed A3.494.670.52−0.06 [−0.13, +0.01]−0.00
training-mean baseline4.655.96—

Table 9. Visual-field mean deviation (MD, dB) regressed from frozen epoch-50 features in glaucoma cases of the test split (n=1,466n = 1,466), ridge regression with the penalty chosen on validation. Lower MAE and RMSE and higher Spearman correlation are better. Δ\Delta is the difference from RANDOM of the same seed with a paired case-bootstrap interval. MD partly defines the glaucoma label, so this is a severity endpoint on the same cohort, not an independent task. Encoders without a per-case feature-cache identity (RANDOM original) are not included.

Variation across pretraining continuations at epoch 50

Figure 4. Variation across pretraining continuations at epoch 50. (a) Test AUC averaged over five probe-head seeds; error bars show head-seed standard deviation. (b) Paired AUC differences (on the original AUC scale) from RANDOM. Bars summarize seed A. Filled and open markers distinguish additional continuations A and B; half-filled markers denote re-probed originals (O). Letters identify seeds. All continuations share an ancestor checkpoint.

Guide agreement and envelope plausibility

FairVision has no retinal layer annotations, so neither guide can be scored for accuracy on it. We fixed failure rules before computing any rate and compared the CENTROID band with the hard MIRAGE envelope (patch occupancy at least 0.25) on full B-scans of 300 training volumes, 50 from each combination of glaucoma label and tertile of a signal-to-noise proxy, with ten B-scans per volume (3,000 B-scans). The computation was repeated on all 6,000 training volumes. A B-scan counts as a gross disagreement when the band–envelope IoU is below 0.2 or more than half of the band cells contain no envelope. Overall rates are population-weighted. Subgroup rates are unweighted proportions within the stratified sample. Disagreement rates use B-scans with a usable envelope, and intervals are 95% volume-cluster bootstrap intervals.

The band disagreed grossly with the envelope in 4.8% of B-scans (3.7–5.9, all training volumes 5.3%) and was off-center in at least three of ten columns in 0.9%. Disagreement was more frequent in glaucoma than in normal eyes (6.5% versus 2.6%) and in the lowest than in the highest signal tertile (7.5% versus 2.3%). The median IoU was 0.41. Because the envelope is typically about half as thick as the 7-row band, an IoU of this size is expected even for a centered band. Of the envelopes, 3.8% (2.7–5.0) raised at least one plausibility flag: an empty or invalid envelope, three or more components (1.5%), irregular thickness (2.5%) or implausible position (0.4%). These are agreement and plausibility statistics between two automatic guides, not anatomical accuracy. The envelope model’s only quantitative validation is in-domain on GOALS (foreground Dice 0.925).

Target sizing and visible context

Batch equalization. Predictor targets are shortened to the minimum target length in the microbatch, and the loss applies a single mean reduction to the stacked prediction and target tensors, including averaging over images. COVER chooses placements using the anatomical mass inside full candidate rectangles before this equalization. Because target indices are ordered row-major, shortening removes the lower portion of each rectangle rather than a spatially random subset. Coverage at placement time can consequently differ from coverage at the loss.

Measured geometry. Across 24,000 emitted targets, 73.4% remain perfect rectangles. In the batch-level audit of the trained configurations, COVER hides 73.3% of tissue cells against 70.7% for ENVELOPE while supplying less context (Table 3), and the guide masks hidden at the loss is 74.57% against 78.51% at placement (Table 10). Thus the trained COVER policy is not a controlled higher-coverage counterpart to ENVELOPE.

MeasurementCOVER
Views losing tissue targets during equalization300
Valid-guide context-criterion misses358
Views with zero tissue context55
Scored hidden guide mass (%)78.51
Final hidden guide mass (%)74.57
Mean encoder-visible tissue cells10.13

Table 10. Final masks from the trained COVER strategy on 576 fixed training views. Context-criterion misses count only the 573 valid guides. Scored and final coverage refer to supported soft guide mass; context counts include all views.

ENVELOPE shares the same shortening rule but does not optimize an exact coverage level. The anatomy-shaped strategies instead resample targets to a fixed K=16K=16 indices. Equalization therefore has different consequences across the strategies, even when nominal target counts agree.

Final target and context masks

Scope. We replayed the actual guided dataset, paired transforms, DataLoader worker collator and returned mask tensors on the existing 600-view training scope (24 volumes, 25 views each), using fixed new crop draws shared across policies. These are not reconstructions of the original training crops. Drawn block sizes were injected independently of placement; equal seeds alone do not pair policy-dependent rejection sampling. The production-size subset contains 576 views in complete batches of 64, with 573 valid post-crop guides. The shorter final batch is excluded from Table 10. Views within volumes are correlated, so these are descriptive mask statistics, not independent training replications.

What is measured. Target coverage is measured both at scoring and after target collation. Encoder-visible tissue is counted only in the final encoder indices, not in the full-grid target complement. Tissue is a guide occupancy thresholded at 0.25, not clinical ground truth. The context criterion is at least max⁡(⌈0.21Ntissue⌉,min⁡(4,Ntissue))\max(\lceil0.21N_{\mathrm{tissue}}\rceil,\min(4,N_{\mathrm{tissue}})) non-target tissue cells. This occupancy-cell diagnostic is distinct from the policy’s soft-mass constraint on the target complement.

Random fill is not a background quota. At full ramp all usable targets take the guided branch, while the random_legal fill mechanism is conditional on unused target blocks. Some views have no random-fill blocks. Such blocks can include tissue, and guided blocks can include background, so branch labels cannot stand in for background supervision. Target composition is therefore measured from the returned token indices rather than inferred from the branch name.

Motivating attribution analysis and evidence limits

Origin of the target-selection question. Before studying anatomy-guided masking, we inspected feature-occlusion maps from glaucoma classifiers initialized with the uniformly masked I-JEPA baseline and then fine-tuned. The analysis compared mean pooling, cross-attention pooling and an attentive probe. At the B-scan level, a B-scan representation was zeroed before re-evaluating the classifier. At the patch level, an encoded patch was omitted from its B-scan’s patch mean, the mean was recomputed from the remaining patches, and the altered B-scan representation was passed to the pooling module and classification head. The change in logit measures sensitivity to this feature intervention; it is not pixel occlusion or an attention map.

Let FF denote the sequence of B-scan representations supplied to the classifier and ℓ(F)\ell(F) its glaucoma logit. If F(−s,p)F^{(-s,p)} replaces the representation of B-scan ss with the mean of its remaining patch features after omitting patch pp, the attribution is

Δs,p=ℓ(F)−ℓ(F(−s,p)).(6)\Delta_{s,p}=\ell(F)-\ell\left(F^{(-s,p)}\right). \tag*{(6)}

Positive values mean that removing the feature lowers the glaucoma logit; negative values mean that removal raises it. Both signs indicate prediction sensitivity. Selected archived displays are not reproduced because of the dataset licence.

The historical report and selected overlays prompted us to investigate retinal regions as prediction targets. Those examples were selected from correctly classified cases and informative B-scans, so they are not a population-level estimate of anatomical localization. Nor does a response to feature removal establish a clinically causal region or show which pretraining mask is best. Fine-tuning used diagnosis labels, so the maps cannot separate the contribution of self-supervised pretraining from subsequent supervised adaptation.

Table 11: Original-run contrasts: matched epoch and matched precision. Δ\Delta is a paired difference on identical test cases; the interval is a 10,000-resample paired bootstrap and pp is a DeLong test for correlated ROC curves. qq is Benjamini–Hochberg over these nine contrasts. All pp and qq values here are descriptive (§ 6). Every comparison against the null excludes zero.

contrastepochΔ\Delta (pp)95% CI (pp)ppqq
ENVELOPE − RANDOM50+1.20[+0.68, +1.73]<0.0001<0.0001
ENVELOPE − RANDOM75+0.80[+0.28, +1.32]0.00250.0045
ENVELOPE − RANDOM100+0.62[+0.10, +1.13]0.01990.0299
CENTROID − RANDOM50+0.99[+0.51, +1.49]<0.00010.0001
CENTROID − RANDOM75+1.13[+0.63, +1.64]<0.0001<0.0001
CENTROID − RANDOM100+1.09[+0.57, +1.60]<0.0001<0.0001
CENTROID − ENVELOPE50-0.20[-0.69, +0.29]0.41140.4114
CENTROID − ENVELOPE75+0.33[-0.13, +0.79]0.14910.1677
CENTROID − ENVELOPE100+0.47[+0.04, +0.91]0.02940.0378

Table 11. Original-run contrasts: matched epoch and matched precision. Δ\Delta is a paired difference on identical test cases; the interval is a 10,000-resample paired bootstrap and pp is a DeLong test for correlated ROC curves. qq is Benjamini–Hochberg over these nine contrasts. All pp and qq values here are descriptive (§ 6). Every comparison against the null excludes zero.

Available evidence. Archived occlusion curves and heat maps remain available, but the underlying per-volume numerical arrays and a complete normalization record are not retained in the evidence used for this paper. The historical record also documents sensitivity of patch-level displays to numerical precision and colour scaling. The selected image panels are retained as archived visualizations, without restoring their numerical summaries as validated results. We make no spatial-consistency, laterality or clinical landmark claim from those displays. Recovering the producing arrays, checkpoint identities and plotting transforms would be necessary for a reproducible quantitative analysis.

Full paired-contrast table

Benjamini–Hochberg is applied within a declared family consisting of exactly these nine contrasts (the fp16 comparisons among RANDOM, CENTROID and ENVELOPE at epochs 50, 75 and 100) and the remaining 25 exploratory contrasts in the stored statistics are corrected as a separate family. The two families are never pooled, and the qq values printed above are those of the nine-contrast family only. Declaring the family fixes what is corrected together. It does not make these contrasts confirmatory, because the test split was inspected repeatedly before the family was fixed, and the paper reports them as descriptive throughout.

Probe precision control

The original rectangle-strategy probes use mixed precision, while the anatomy-shaped and coverage-strategy probes use fp32. All Δ\Delta values in Table 2 use a baseline matched in both epoch and probe precision. Probe precision is distinct from pretraining target-encoder precision. We repeated the recorded mixed-precision probes with autocast disabled in feature extraction, head fitting and evaluation. Each comparison uses the same frozen encoder. The recorded changes are smaller than the principal strategy differences.

strategyguidance signalepoch 50 AUCepoch 50 Δ (pp)epoch 75 AUCepoch 75 Δ (pp)epoch 100 AUCepoch 100 Δ (pp)
RANDOMnone (null)0.8641—0.8723—0.8746—
ENVELOPEsegmenter (location only)0.8761+1.200.8803+0.800.8807+0.62
CENTROIDintensity centroid, no segmenter0.8740+0.990.8836+1.130.8855+1.09
ANATOMY-V2segmenter (target shape)0.8654+0.130.8612−1.11——
COVER r=0.21segmenter (coverage)0.8643+0.020.8639−0.840.8577−1.68
reported: supervised 3D ResNet (Luo et al., 2024)0.8649 (trained end to end on all B-scans; not this probe)

Table 2. Mean-pooled linear-probe test AUC on FairVision (upper block: fp16; lower block: fp32). Δ is the AUC difference from RANDOM (pp); only positive differences are bold.

policyepochfp16 AUCfp32 AUCΔ\Deltapp
ENVELOPE500.8760640.876063-0.0000010.964
ENVELOPE750.8803070.880305-0.0000020.806
ENVELOPE1000.8807430.880761+0.0000180.167
CENTROID500.8740300.874015-0.0000150.412
CENTROID1000.8854850.885293-0.0001920.767
RANDOM500.8640970.864121+0.0000240.128
RANDOM750.8723020.872302-0.0000010.962
RANDOM1000.8745810.874485-0.000096<0.001<0.001

Table 12. Full fp32 re-probe against the original fp16 result, same frozen encoder in each row. The eight DeLong pp-values are unadjusted diagnostics, not a family-wise test. Differences in this table use the original AUC scale. The largest difference is 2×10−42 \times10^{-4}.

References

  1. [1]Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with a joint-embedding predictive architecture. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15619–15629, 2023. doi: 10.1109/CVPR52729.2023.01499.DOI
  2. [2]Shekoofeh Azizi, Basil Mustafa, Fiona Ryan, Zachary Beaver, Jan Freyberg, Jonathan Deaton, Aaron Loh, Alan Karthikesalingam, Simon Kornblith, Ting Chen, et al. Big self-supervised models advance medical image classification. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021.arxiv.org/abs/2101.05224
  3. [3]Alexei Baevski, Wei-Ning Hsu, Qiantong Xu, Arun Babu, Jiatao Gu, and Michael Auli. data2vec: A general framework for self-supervised learning in speech, vision and language. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 1298–1312. PMLR, 2022. URL https://proceedings.mlr.press/v162/baevski22a.html.arxiv.org/abs/2202.03555
  4. [4]Randall Balestriero and Yann LeCun. LeJEPA: Provable and scalable self-supervised learning without the heuristics, 2025. URL https://arxiv.org/abs/2511.08544.
  5. [5]Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. BEiT: BERT pre-training of image transformers. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=p-BhZS zS9o4.arxiv.org/abs/2106.08254
  6. [6]Adrien Bardes, Quentin Garrido, Jean Ponce, Xinlei Chen, Michael Rabbat, Yann LeCun, Mahmoud Assran, and Nicolas Ballas. Revisiting feature prediction for learning visual representations from video. Transactions on Machine Learning Research, 2024. URL https://openreview.net/forum?id=QaCCuDfBk2.arxiv.org/abs/2404.08471
  7. [7]Xavier Bouthillier, Pierre Delaunay, Mirko Bronzi, Assya Trofimov, Brennan Nichyporuk, Justin Szeto, Nazanin Mohammadi Sepahvand, Edward Raff, Kanika Madan, Vikram Voleti, Samira Ebrahimi Kahou, Vincent Michalski, Tal Arbel, Chris Pal, Gael Varoquaux, and Pascal Vincent. Accounting for variance in machine learning benchmarks. Proceedings of Machine Learning and Systems, 3:747–769, 2021. URL https://proceedings.mlsys.org/paper_files/paper/2021/hash/0184b0cd3cfb185989f858a1d9f5c1eb-Abstract.html.arxiv.org/abs/2103.03098
  8. [8]Alberto M. Ceballos Arroyo, Jisoo Kim, Chu-Hsuan Lin, Lei Qin, Geoffrey S. Young, and Huaizu Jiang. Anatomically-guided masked autoencoder pre-training for aneurysm detection. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 5693–5702, 2026. doi: 10.1109/WACV61042.2026.00552.DOI
  9. [9]Haijian Chen, Wendong Zhang, Yunbo Wang, and Xiaokang Yang. Improving masked autoencoders by learning where to mask. In Pattern Recognition and Computer Vision, pages 377–390. Springer, 2023a. doi: 10.1007/978-981-99-8543-2_31.DOI
  10. [10]Zekai Chen, Devansh Agarwal, Kshitij Aggarwal, Wiem Safta, Mariann Micsinai Balan, and Kevin Brown. Masked image modeling advances 3D medical image analysis. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1970–1980, 2023b. doi: 10.1109/WACV56688.2023.00201.DOI
  11. [11]Elizabeth R. DeLong, David M. DeLong, and Daniel L. Clarke-Pearson. Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics, 44(3):837–845, 1988. doi: 10.2307/2531595. URL https://doi.org/10.2307/2531595.
  12. [12]Huihui Fang, Fei Li, Huazhu Fu, Junde Wu, Xiulan Zhang, and Yanwu Xu. Dataset and evaluation algorithm design for GOALS challenge. In Ophthalmic Medical Image Analysis, pages 135–142. Springer, 2022. doi: 10.1007/978-3-031-16525-2_14.DOI
  13. [13]Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16000–16009, 2022. doi: 10.1109/CVPR52688.2022.01553.DOI
  14. [14]Xiangteng He, Shunsuke Sakai, Shivam Chandhok, Sara Beery, Kun Yuan, Nicolas Padoy, Tatsuhito Hasegawa, and Leonid Sigal. DSeq-JEPA: Discriminative sequential joint-embedding predictive architecture, 2025. URL https://arxiv.org/abs/2511.17354v4. Accepted at ECCV 2026; arXiv:2511.17354v4.
  15. [15]De-Xing Huang, Xiao-Hu Zhou, Mei-Jiang Gui, Xiao-Liang Xie, Shi-Qi Liu, Shuang-Yi Wang, Tian-Yu Xiang, Rui-Ze Ma, Nu-Fang Xiao, and Zeng-Guang Hou. VasOMM: Vascular anatomy-aware masked image modeling for vessel segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 4985–4993, 2026. doi: 10.1609/aaai.v40i6.42503.
  16. [16]Raphael Judkiewicz, Eran Berkowitz, Meishar Meisel, Tomer Michaeli, and Joachim A Behar. Shifting the retinal foundation models paradigm from slices to volumes for optical coherence tomography. NPJ Digit. Med., 9(1), March 2026.

Paper details

vlnc:W_wswwsyax
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 8

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents