betaFeedback

WorksComputer Science

IFC-SSL-Bench: Benchmarking Self-Supervised Learning for Annotation-Efficient Multispectral Imaging Flow Cytometry

Abstract

Imaging Flow Cytometry (IFC) generates large multispectral datasets, yet the need for expert annotations bottlenecks the development of supervised deep learning models. While Self-Supervised Learning (SSL) offers a solution, its application to IFC remains fragmented and lacks systematic benchmarking. To address this, we present IFC-SSL-Bench, a modular framework for evaluating SSL in IFC. We benchmark six SSL approaches across four backbones on two heterogeneous datasets. By assessing downstream classification performance across annotation regimes of the training sets (1%-100%), we quantify annotation efficiency and identify optimal configurations for high-dimensional biological imaging. SSL matches the performance of fully supervised baselines trained on 100% of annotated data while using as little as 5% of annotations for morphological classification, with advantages in low-data regimes below 5%. Across both datasets, the contrastive approach MoCo-v2 paired with ConvNeXt backbones emerged as the top-performing configuration, with ConvNeXtV1-T best for immunological synapse classification and ConvNeXtV2-T for morphological classification. We identify a clear pattern in transfer dynamics: ImageNet initialization benefits morphological classification, consistent with its visual similarity to natural images, whereas random initialization better captures the more domain-specific immunological synapse classification. SSL elevates class-specific F1-scores by over 35 percentage points for structurally complex classes at 1% of annotated data. We synthesize these insights into a unified set of guidelines for deploying self-supervised workflows in high-throughput IFC, and release the benchmark as a resource for future work.

Correspondence: [email protected]

Introduction

Imaging Flow Cytometry (IFC) is a modern technology that combines the quantitative, high-throughput capabilities of Flow Cytometry (FC) with the spatial and morphological detail provided by microscopy [1]. In practice, this is achieved by hydrodynamically focusing cells into a single-file stream that passes through a laser-illuminated imaging zone. High-speed detectors, synchronized with the cell flow, capture multispectral images using combinations of brightfield, darkfield, phase-contrast, and fluorescence channels, to achieve sharp subcellular resolution at high velocities [2, 3]. By capturing multiple images of each cell as it passes through the imaging region, IFC produces high-dimensional image datasets that provide unprecedented single-cell depth [3]. This hybrid technique has advanced research in immunology, oncology, cell biology, and environmental science by allowing detailed examination of heterogeneous cell populations and subtle phenotypic variations with robust statistical confidence [2, 4, 5, 6].

Despite its powerful capabilities, IFC faces significant challenges that stem not only from the inherent biological heterogeneity of cell populations [7], but also from the technical constraints, and therefore hinders effective data analysis. The primary technical hurdle in IFC is the fundamental trade-off to set between throughput and image resolution. While increasing magnification or exposure time gradually improves spatial resolution and fine morphological detail, it inherently reduces acquisition speed. Conversely, high-throughput demands shorter exposure times, which results in reduced image quality with lower magnification and/or low Signal-to-noise ratio (SNR) [8]. In multichannel acquisitions, fluorescence signals are additionally subject to background and to spectral crosstalk between detectors, which can obscure weak markers and distort the measured intensity distributions [9]. The appearance of a given cell type therefore depends not only on its biology but also on how it was acquired, so that the morphological differences of interest are superimposed on variation of purely technical origin.

Beyond these acquisition challenges, the analysis of IFC data also faces various bottlenecks. Traditionally, IFC data analysis has relied on manual gating or software-assisted pipelines using handcrafted feature extraction. Tools such as CellProfiler [10] improved throughput by automating cell feature extraction of hundreds of morphological and intensity-based descriptors. However, these classical approaches still require extensive domain expertise to select and tune relevant features for specific biological assays, making the analysis labor-intensive and susceptible to operator subjectivity [11, 12]. This reliance on predefined features introduces variability and limits scalability, as manual or semi-automated gating is not only time-consuming but also prone to inconsistencies across different operators [2, 13].

With the advent of Deep Learning (DL), many of these challenges have been partially mitigated. DL models can automatically learn complex features from raw data, reducing the dependence on manual feature engineering. This transition from manual feature engineering to DL-based image analysis has also occurred in the field of IFC [14, 15, 16]. Several specialized pipelines have emerged to facilitate this transition, including DeepXperiment for feature importance analysis, Deepometry [18] for multi-channel phenotyping, Deep Cytometry [19] for real-time sorting applications, and Small U-Net [20] for high-throughput segmentation.

Despite these successes, the models still require large amounts of accurately annotated data for supervised training [21, 22]. Given the high cost and expertise required to annotate images, obtaining sufficiently large and diverse annotated datasets remains a significant bottleneck, leading to scarce datasets, limiting generalization performance, and creating a need for annotation-efficient strategies [21]. To address this, Self-Supervised Learning (SSL) methods have emerged as a promising alternative [23, 24]. SSL leverages unannotated data to learn useful representations, reducing the need for costly manual annotations. SSL is a paradigm well suited to IFC, which naturally generates vast amounts of unannotated cellular images at high-throughput scales. The robust performance of SSL in processing high-dimensional data, most notably in natural language processing [25] and video analysis [26], has provided a strong impetus for its application to other fields such as transcriptomics [27, 28] and visual geo-localization [29]. In medical imaging, SSL has reached a high level of maturity: in digital histopathology, for example, the SSL method Barlow Twins (BT) was used to pretrain a model on whole slide images and demonstrated remarkable robustness to cohorts variations and tissue types [30]. Similarly, on widely used benchmarks like MedMNIST [31], SSL has been investigated to provide clear domain guidelines. Bundele et al. [32] benchmark eight SSL methods across eleven MedMNIST datasets, varying annotation fraction and initialization, and find that ImageNet initialization consistently benefits performance, especially at low annotation fractions. Boukdir similarly reports transferable SSL representations across diagnostic domains [33]. More recently in radiology with nnFoundation, self-supervised pretraining at scale has produced general purpose encoders that transfer across more than a hundred downstream tasks [34].

Cellular imaging has followed the same trajectory. On Cell Painting, Kraus et al. trained Masked Autoencoders (MAEs) on millions of images and recovered known relationships between genetic and chemical perturbations [35], while Kim et al. compared self-supervised objectives directly on more than 10,000 compounds and found that DINO outperformed MAE, contrastive learning and the engineered CellProfiler pipeline [36]. Self-distillation has since been scaled across assays and cell types: Moutakanni et al. trained DINOv2 on several million multichannel images from the Human Protein Atlas and Cell Painting, matching supervised performance on protein localization tasks with 1% of the annotations [37], and Zedda et al. assembled 1.25 million single-cell images from 18 datasets to build a dedicated foundation model for red blood cell morphology [38].

However, these successes do not seamlessly translate to IFC, which presents a unique set of challenges. Current literature focuses on isolated methods, demonstrating, for example, that a specific method like MoCo-v2 [39] can be used for leukocyte differential analyses [40], or BT [41] for investigating immunological synapse formation and characterizing antibodies [42]. These studies fail to address whether these choices are optimal, leaving the comparative potential of other paradigms unexplored.

Collectively, these limitations underscore a fundamental gap in the principled evaluation of SSL approaches for IFC, raising the following key research question: How can SSL be effectively applied and optimized for multispectral IFC data?

This central inquiry is decomposed into two specific research questions (RQs):

  • RQ1: Methodological and architectural effectiveness: Which state-of-the-art SSL methods and backbone architectures are most effective for learning reliable and discriminative embeddings from multispectral IFC data?

  • RQ2: Annotation efficiency: How does SSL reduce annotation needs on downstream classification?

By addressing these questions, this work aims to provide a comprehensive benchmark for SSL in the IFC domain, offering insights into the optimal combination of algorithms, architectures, and preprocessing strategies for high-dimensional biological imaging. Our primary contributions are:

  1. A unified benchmarking pipeline for multispectral IFC: We introduce IFC-SSL-Bench, a standardized, modular, reproducible and extensible framework that unifies data processing, pretraining, and downstream evaluation to ensure fair comparisons across diverse SSL methods for IFC.

  1. Comprehensive empirical analysis: We systematically evaluate six state-of-the-art invariance-based SSL methods across four backbone architectures on two distinct SSL datasets, providing the first structured comparison of self-supervised representation learning efficacy in this domain, identifying superior performance regimes.

  1. Annotation-efficient learning: We deliver empirical evidence on the utility of SSL under annotation scarcity. By quantifying performance gains across varying fractions of annotated data, we provide insights in optimizing SSL in annotation-constrained scenarios.

  1. Actionable guidelines & open resource: We derive best-practice recommendations for models, SSL methods, and initialization strategies tailored to IFC data and release our full codebase under an open license at https://github.com/hciays/IFC-SSL-Bench to foster reproducibility and advance future research in SSL applications for IFC.

Results

To address the lack of systematic evaluation for SSL in IFC, we developed a benchmarking framework designed to decouple the influence of SSL paradigms, backbone architectural choices, and domain-specific knowledge.

The Benchmarking Pipeline

Figure 1 describes our proposed modular framework for benchmarking SSL methods on IFC. The pipeline standardizes SSL evaluation through three integrated stages: ❶, a data preparation module (Methods, Data Preparation) performs dataset splitting into an unannotated set DunaD_{\mathrm{una}} for pretraining and an annotated set DannD_{\mathrm{ann}} for downstream analysis, channel selection, and domain-specific transformations. This is followed by ❷, a flexible pretraining module (Methods, Self-Supervised Pretraining) designed to decouple the effects of model architecture, initialization strategy, and SSL methodology. On the self-supervised path, six objectives spanning contrastive, asymmetric, and redundancy reduction families are pretrained on DunaD_{\mathrm{una}}; all of them draw their views from the same augmentation operators and parameters for a given backbone, while the number and spatial scale of views follow each method’s original formulation, since the multi-crop scheme of DINO is intrinsic to its objective rather than a preprocessing choice. In parallel, fully supervised models are trained end-to-end on DannD_{\mathrm{ann}} for each combination of backbone, weights initialization choice, and fraction of annotated data. The self-supervised path therefore isolates the contribution of the objective, and the supervised path provides the reference against which annotation efficiency and representation quality are measured. Finally, ❸, an evaluation module (Methods, Downstream Analysis) responsible for downstream analyses, integrating both quantitative performance metrics and qualitative representation assessments.

Overview of the modular SSL benchmarking framework

Figure 1. Overview of the modular SSL benchmarking framework. The pipeline is organized into three core modules: 1, data preparation. This module covers the dataset split generation, channel selection, and domain-specific transformations. 2, the learning module, which consists of the self-supervised pretraining and supervised baseline. Six self-supervised objectives spanning three families of invariance-based SSL are pretrained on the unannotated set: contrastive (SimCLR, MoCo-v2, MoCo-v3), asymmetric (BYOL, DINO), and redundancy reduction (BT). In parallel, the annotated set is used to train end-to-end supervised baselines for each backbone, weight initialization, and annotated data fraction (1%, 5%, 10%, 25%, 50%, and 100%). 3, downstream analysis evaluates the pretrained encoders on image classification at the same annotated data fractions as the supervised baselines through four evaluation protocols (kNN, linear probing, XGBoost, and full finetuning) quantified by macro F1 scores. This quantitative analysis is complemented by a qualitative assessment of the learned representations. Scale bars in the multichannel sample images represent 5µm.

For our benchmarking, we selected two publicly available IFC datasets that represent distinct biological and computational challenges, ensuring the generalizability of our SSL framework (Methods, Data Preparation). All images and annotations were obtained from the public releases of the original studies. First, the Red Blood Cells (RBC) dataset [43], a specialized single-cell morphological dataset comprising 315,416 multi-channel images (BF1, BF2, and DF) from which 70,331 were annotated by the original authors and categorized into eight distinct red blood cell shapes (Figure 2A, B). Second, we leveraged the Immunological Synapse Formation (ISF) dataset, described by its authors as one of the largest publicly available IFC datasets, containing over 2.89 million images profiling human immunological synapse formation [42, 44] (Figure 2D). A subset of 9,186 images was manually annotated by the original authors into 17 classes. Similarly to [17, 42], we focus on the nine biologically relevant classes (B cell, T cell, T cell with signaling, B cell and T cell in one layer, T cell with B cell fragments, Synapse without signaling, Synapse with signaling, No cell cell interaction, and Multiplets) as well as on the five canonical channels BF, F–Actin, MHCII, CD3, and P–CD3ζ to ensure comparability with established baselines (Figure 2C). We refer to this specific five-channel configuration as ISF–5CH throughout the manuscript. Together, these datasets establish a rigorous cross-task performance landscape, spanning from single-cell morphology to complex intercellular signaling, and allowing for a comprehensive assessment of SSL representation quality. We benchmarked six state-of-the-art SSL methods (SimCLR [45], MoCo-v2 [39], MoCo-v3 [46], DINO [47], BT [41], and Bootstrap Your Own Latent (BYOL) [48]) across four foundational architectures (ResNet50 [49], Vision Transformer (ViT)–S/16 [50], ConvNeXtV1–T [51] and ConvNeXtV2–T [52]), which can be initialized either with random weights

Overview of IFC datasets and distribution of annotated and unannotated subsets

Figure 2. Overview of IFC datasets and distribution of annotated and unannotated subsets. (A) Representative multichannel imagery and class distribution of the annotated RBC dataset [43]. For each class, three imaging channels are shown alongside the sample counts for training, validation, and testing splits. (B) Distribution of the RBC unannotated subset across acquisition batches (bags (CE) and timepoints (Day)). Stacked bars represent the sample volume per CE, partitioned by day. (C) Representative multichannel imagery and class distribution of the annotated ISF dataset [42, 44]. Training, validation, and testing counts are indicated per class. (D) Distribution of the ISF unannotated subset across four independent experiments. Stacked bars illustrate donor diversity within each experiment, showing the contribution of individual donors to the total unannotated pool. Scale bars in (A) and (C) represent 5μm5\mu\mathrm{m}.

(θRnd\theta_{\mathrm{Rnd}}) or ImageNet-supervised pretrained weights (θIN\theta_{\mathrm{IN}}). These SSL methods (Supplementary SSL) represent three established families of self-supervision (contrastive, asymmetric student-teacher, and redundancy reduction) identified by recent studies for achieving state-of-the-art representation quality [53].

We simulate the acute annotation scarcity typical of specialized bioimaging domains, where high-fidelity manual annotations are labor-intensive and scarce, by evaluating each model across a graded sequence of annotated data fractions (1%, 5%, 10%, 25%, 50%, and 100%). To compare downstream adaptation capacity while conserving computational resources, we restricted full-parameter finetuning to the best configurations identified for each weight initialization (θRnd\theta_{\mathrm{Rnd}} or θIN\theta_{\mathrm{IN}}) and evaluation protocol (k-Nearest Neighbor (kNN), linear probing, and XGBoost). The best configurations are systematically selected throughout the manuscript based on achieved macro F1-score under the fully annotated dataset regime (100% of DannD_{\mathrm{ann}}).

Supervised Baseline Performance

To establish baselines, supervised models were trained end-to-end for each backbone architecture under the two initialization regimes θRnd\theta_{\mathrm{Rnd}} and θIN\theta_{\mathrm{IN}}. To quantify the baseline sensitivity to annotation scarcity, each baseline was systematically trained across diminishing annotated fractions of the training subset of DannD_{\mathrm{ann}}, ranging from the full dataset to the low-data regime (1%), and evaluated on the full test subset of DannD_{\mathrm{ann}}. As shown in Table 1 and further visualized in Figure 3, Figure S1A, and Figure S1B, performance patterns varied across architectures, initialization, and data regimes. Across both datasets, all investigated backbones exhibited a highly consistent trajectory: As the fraction of annotated training data increased, test set macro F1-scores rose systematically while the performance gap between θRnd\theta_{\mathrm{Rnd}} and θIN\theta_{\mathrm{IN}} models generally decreased, with architecture dependent exceptions.

BackboneRBCISF–5CH
ResNet50–θRnd\theta_{\mathrm{Rnd}}0.875 ± 0.0300.870 ± 0.007
ResNet50–θIN\theta_{\mathrm{IN}}0.920 ± 0.0060.752 ± 0.021
ConvNeXtV1–T–θRnd\theta_{\mathrm{Rnd}}0.839 ± 0.0060.802 ± 0.018
ConvNeXtV1–T–θIN\theta_{\mathrm{IN}}0.921 ± 0.0050.889 ± 0.007
ConvNeXtV2–T–θRnd\theta_{\mathrm{Rnd}}0.816 ± 0.0190.848 ± 0.002
ConvNeXtV2–T–θIN\theta_{\mathrm{IN}}0.922 ± 0.0110.877 ± 0.011
ViT–S/16–θRnd\theta_{\mathrm{Rnd}}0.752 ± 0.0200.857 ± 0.005
ViT–S/16–θIN\theta_{\mathrm{IN}}0.816 ± 0.0110.893 ± 0.010

Table 1. Test set macro F1-scores of the supervised end-to-end baselines (mean ± std) on 100% annotated data fraction. θIN\theta_{\mathrm{IN}} and θRnd\theta_{\mathrm{Rnd}} denote initialization from ImageNet-supervised weights and random initialization, respectively. The best-performing supervised baseline for each dataset is bolded.

Test set performance of supervised baselines across annotated data fractions, datasets, backbone variants, and weight initialization strategies

Figure 3. Test set performance of supervised baselines for both datasets (RBC and ISF–5CH) across backbone variants, annotated data fractions, and weights initialization strategies

Supervised baseline evaluation under varying initialization, data constraints, and domain-specific transformation

Figure S1. Supervised baseline evaluation under varying initialization, data constraints, and domain-specific transformation. (A) Transfer learning effect (Δ\Delta Macro F1 score = Macro F1 score θRnd\theta_{\mathrm{Rnd}}− Macro F1 score θIN\theta_{\mathrm{IN}}) measured as the difference in macro F1-score between random initialization (θRnd\theta_{\mathrm{Rnd}}) and ImageNet-supervised (θIN\theta_{\mathrm{IN}}) across different annotated fractions for the ISF–5CH and RBC datasets. (B) Macro F1-score under extreme data scarcity (0.1% annotated fraction) on the RBC dataset, comparing all backbone architectures across both initialization strategies. (C) Impact of the domain specific Logicle transformation on the ISF–5CH dataset. The baseline is ConvNeXtV1–TθRnd\theta_{\mathrm{Rnd}} trained end-to-end on 100% of Dann\mathcal{D}_{\mathrm{ann}}.

On the RBC dataset, the impact of initialization revealed a clear pattern: Across all evaluated backbones, θIN\theta_{\mathrm{IN}} initialization consistently yielded performance improvements, with the singular exception of ConvNeXtV2–T at the sparse 1% annotated data fraction (Figure 3, Figure S1A). The magnitude of this initialization advantage varied largely across backbone families. For ResNet50, the performance gap between θIN\theta_{\mathrm{IN}} and θRnd\theta_{\mathrm{Rnd}} was the most constrained across all fractions, reaching its maximum delta in F1-score of 13.5 percentage points (pp) at the 1% annotated data regime. Conversely, the most pronounced initialization sensitivity was observed within the ConvNeXt family. Under the 1% data regime, ConvNeXtV1–T exhibited a 46.3 pp performance advantage under θIN\theta_{\mathrm{IN}} compared to θRnd\theta_{\mathrm{Rnd}}, with this margin narrowing to 31.0, 24.0, and 13.9 at the 5%, 10%, and 25% annotated data fractions, respectively. While ConvNeXtV2–T maintained a performance advantage of at least 10.0 pp under θIN\theta_{\mathrm{IN}} across higher fractions, θRnd\theta_{\mathrm{Rnd}} slightly outperformed θIN\theta_{\mathrm{IN}} at the 1% annotated scarcity level by 2.3 pp. The ViT–S/16 demonstrated a monotonic decrease in initialization sensitivity as the volume of annotated training data scaled, with the θIN\theta_{\mathrm{IN}} advantage steadily declining from 25.9 pp at the 1% annotated fraction to 6.4 pp under full Dann\mathcal{D}_{\mathrm{ann}}.

On the ISF–5CH dataset, the impact of initialization strategies revealed a more heterogeneous distribution of performance across the backbones and data regimes. ResNet50 consistently benefited from θRnd\theta_{\mathrm{Rnd}} initialization, except for the 1% annotated data fraction where no difference was observed (Figure 3). The largest delta in F1-score is observed at 10% of Dann\mathcal{D}_{\mathrm{ann}} with 22.8 pp in favor of θRnd\theta_{\mathrm{Rnd}} (Figure S1A). Conversely, ConvNeXtV1–T demonstrated a consistent preference for θIN\theta_{\mathrm{IN}} across all fractions with a minimal delta of 8.7 pp. Both ConvNeXtV2–T and ViT–S/16 exhibited a transitional behavior: While performance fluctuated between initialization strategies in low-data regimes (1–10% of Dann\mathcal{D}_{\mathrm{ann}}), θIN\theta_{\mathrm{IN}} pretraining provided a minimal performance advantage once the annotated fraction reached 25% or higher (Figure 3, Figure S1A).

To investigate whether integrating domain-specific knowledge enhances feature extraction, we applied during training the domain-specific preprocessing Logicle transformation [54, 55] following the Gating-ML 2.0 standard [56] to the supervised baseline. Contrary to our initial hypothesis, the inclusion of the Logicle transform during training did not yield any performance gains (Figure S1C). Instead, we observed a slight decrease in F1-score, demonstrating that, while non-linear, domain-specific transformations remain a gold standard for human-in-the-loop visualization (Figure 1 Data preparation) and manual gating [55], they are detrimental to deep representational learning.

Interestingly, we observed on the RBC dataset that the supervised baseline already achieved solid performance when trained on just 1% of Dann\mathcal{D}_{\mathrm{ann}}. To identify the breakdown threshold of standard supervised learning and verify if SSL provides a margin of benefit under extreme constraints, we established a 0.1% ultra-low annotated fraction of Dann\mathcal{D}_{\mathrm{ann}} comparison (Figure S1B). Similarly to the ISF–5CH dataset at this scarcity level, the best supervised baseline (ResNet50–θIN\theta_{\mathrm{IN}}) achieves a very low F1-score of 43.7 ± 1.0%. Furthermore, the vulnerability in the newer ConvNeXtV2–T architecture under extreme data scarcity is pronounced, with the performance advantage of θRnd\theta_{\mathrm{Rnd}} against θIN\theta_{\mathrm{IN}} doubling from 2.3 pp Δ\DeltaF1 at the 1% annotated scarcity level to 4.6 pp Δ\DeltaF1 at 0.1% annotated scarcity. This choice of fractions balances the absolute sample size between the two datasets, yielding 37 training samples at the 1% fraction for the ISF–5CH dataset and 43 samples at the 0.1% fraction for the RBC dataset (as low as two samples per class), thereby enabling a fairer and more balanced comparison of representation capacity under equivalent low-data regimes. Having established the performance bounds and architectural sensitivities of the supervised baselines, we next evaluated whether self-supervised pretraining can bridge the observed domain gap and stabilize representation learning across modalities.

Self-Supervised Pretraining from Scratch Bridges the Domain Gap for Immunological Synapses

Across all three major evaluation protocols (Methods, Downstream Analysis) (kNN, linear probing, and XGBoost), we noticed a consistent performance advantage for backbones trained via self-supervision from θRnd\theta_{\mathrm{Rnd}} compared to those initialized with θIN\theta_{\mathrm{IN}}. Specifically, the target-domain SSL approach yielded median improvements in F1-score on the full Dann\mathcal{D}_{\mathrm{ann}} of 2.3 pp for kNN, 2.0 pp for linear probing, and 4.2 pp for XGBoost (Table 2). Observing the general methodological performance across all evaluated backbones, DINO, MoCo-v2, and MoCo-v3 emerge with the highest performance ranks across several annotation scarcity regimes (Figure 4A, B, Table 2), with the best SSL configuration (ConvNeXtV1–T–θRnd\theta_{\mathrm{Rnd}} with MoCo-v2) being only 1.4 pp below the best supervised baseline in F1-score when evaluated by linear probing. The second best (ViT–S/16–θIN\theta_{\mathrm{IN}} with DINO) was also not far from the best supervised baseline (2.1 pp). These narrow gaps provide strong quantitative evidence of the linear separability of the learned feature space. Most notably, DINO achieved on all three evaluation protocols the top ranking across the ViT–S/16, ConvNeXtV1–T, and ConvNeXtV2–T backbones at the 1% annotated data fraction, holding true under both θRnd\theta_{\mathrm{Rnd}} and θIN\theta_{\mathrm{IN}} (Figure 4B, Figure S2, Figure S3, Figure S4). The architectural choice also plays a critical role in how cleanly these self-supervised representations align with the downstream task. The average model rank when pretraining from scratch (θRnd\theta_{\mathrm{Rnd}}) is led by ConvNeXtV1–T, which displays the most stable and robust representations under strict data limits, whereas ViT–S/16 dominates other backbones under θIN\theta_{\mathrm{IN}} (Figure 4C). ResNet50 backbone exhibits the poorest performance and stability. Specifically, it ranks third overall under θRnd\theta_{\mathrm{Rnd}} and becomes last when transferred from θIN\theta_{\mathrm{IN}}, accompanied by wide standard deviation bars, indicating high susceptibility to initialization variance. This volatility is attributed to its high sensitivity to the choice of self-supervised optimization objective (Table 2, Figure 4B). While ResNet50 exhibits strong representations when paired with contrastive methods (SimCLR, MoCo-v2, MoCo-v3), it demonstrates susceptibility to representation collapse or dimensional degradation when subjected to asymmetric or redundancy-reduction (DINO, BT, BYOL) (Table 2, Figure S2, Figure S3, Figure S4), with some or all of the three runs not converging.

Benchmarking self-supervised representations for the ISF–5CH dataset, including heatmaps, ranking plots, finetuning trajectories, and performance-gain table

Figure 4. Benchmarking self-supervised representations for the ISF–5CH dataset. (A) Ranking of combinations of SSL methods and backbones across downstream evaluation protocols. Heatmaps display the rank of 48 model combinations (six SSL methods times four backbones times two initialization strategies) across kNN, linear probing, and XGBoost protocols for each annotated fraction of DannD_{\mathrm{ann}}. Each cell is split to compare initialization θRnd\theta_{\mathrm{Rnd}} (left/green, 24 ranks) vs. θIN\theta_{\mathrm{IN}} (right/blue, 24 ranks). Darker hues indicate superior ranking (rank 1). Markers identify the corresponding backbone architecture for the specific configuration. (B) SSL method performance ranking for each backbone from each annotated fraction based on linear probing evaluation. Markers with bolded edges represent θIN\theta_{\mathrm{IN}} and normal edges θRnd\theta_{\mathrm{Rnd}}. (C) Architectural robustness and initialization bias: Comparison of average backbone rank after linear probing between initializations θRnd\theta_{\mathrm{Rnd}} and θIN\theta_{\mathrm{IN}}. Error bars indicate the inter SSL method standard deviation. (D) Finetuning trajectories of top-ranked configurations. For each initialization mode (θRnd\theta_{\mathrm{Rnd}} or θIN\theta_{\mathrm{IN}}) and evaluation protocol, the best-performing SSL-backbone pairing as indicated in Table 2 was selected. (E) Finetuning performance gain between the best supervised baseline at 100% of DannD_{\mathrm{ann}} (ViT–S/16–θIN\theta_{\mathrm{IN}}) and the best SSL combination (ConvNeXtV1–T–θRnd\theta_{\mathrm{Rnd}} with MoCo-v2). The table displays the absolute improvements (pp delta) in overall macro-F1-score along with the single class showing the most substantial improvement for each annotated fraction of DannD_{\mathrm{ann}}. Fractions on the x-axis in panels (B), (C), and (D) are displayed on a logarithmic scale for clarity in low-data regimes. Ranks (1 = top performer) per annotated fraction (1%–100%) in (B) and (C) were calculated based on test set macro F1-scores using a linear probing protocol.

BackboneMethodkNN θRnd\theta_{\mathrm{Rnd}}kNN θIN\theta_{\mathrm{IN}}Linear probing θRnd\theta_{\mathrm{Rnd}}Linear probing θIN\theta_{\mathrm{IN}}XGBoost θRnd\theta_{\mathrm{Rnd}}XGBoost θIN\theta_{\mathrm{IN}}
ResNet50SimCLR0.633±0.0110.633 \pm0.0110.639±0.0070.639 \pm0.0070.845±0.0200.845 \pm0.0200.799±0.0060.799 \pm0.0060.832±0.0050.832 \pm0.0050.824±0.0050.824 \pm0.005
ResNet50MoCo-v20.710±0.0120.710 \pm0.0120.740±0.0130.740 \pm0.0130.844±0.0150.844 \pm0.0150.831±0.0050.831 \pm0.0050.849±0.0060.849 \pm0.0060.859±0.0120.859 \pm0.012
ResNet50MoCo-v30.722±0.0060.722 \pm0.0060.760±0.0180.760 \pm0.0180.852±0.0090.852 \pm0.0090.856±0.0140.856 \pm0.0140.850±0.0100.850 \pm0.0100.867±0.0120.867 \pm0.012
ResNet50DINO0.495±0.0180.495 \pm0.0180.299±0.0170.299 \pm0.0170.717±0.0050.717 \pm0.0050.433±0.0230.433 \pm0.0230.688±0.0100.688 \pm0.0100.449±0.0060.449 \pm0.006
ResNet50BT0.662±0.0130.662 \pm0.0130.599±0.0030.599 \pm0.003<u>0.857±0.0100.857 \pm0.010</u>0.807±0.0130.807 \pm0.0130.840±0.0100.840 \pm0.0100.820±0.0010.820 \pm0.001
ResNet50BYOL0.290±0.0070.290 \pm0.0070.599±0.0080.599 \pm0.0080.448±0.0100.448 \pm0.0100.660±0.0090.660 \pm0.0090.532±0.0110.532 \pm0.0110.751±0.0040.751 \pm0.004
ConvNeXtV1–TSimCLR0.668±0.0090.668 \pm0.0090.647±0.0040.647 \pm0.0040.862±0.0120.862 \pm0.0120.824±0.0050.824 \pm0.0050.789±0.0030.789 \pm0.0030.760±0.0050.760 \pm0.005
ConvNeXtV1–TMoCo-v20.687±0.0140.687 \pm0.0140.657±0.0040.657 \pm0.0040.879±0.0100.879 \pm0.0100.850±0.0050.850 \pm0.005<u>0.844±0.0070.844 \pm0.007</u><u>0.795±0.0070.795 \pm0.007</u>
ConvNeXtV1–TMoCo-v30.677±0.0060.677 \pm0.0060.637±0.0030.637 \pm0.0030.870±0.0030.870 \pm0.0030.828±0.0020.828 \pm0.0020.827±0.0070.827 \pm0.0070.759±0.0150.759 \pm0.015
ConvNeXtV1–TDINO0.723±0.0120.723 \pm0.0120.691±0.0050.691 \pm0.0050.854±0.0090.854 \pm0.0090.837±0.0100.837 \pm0.0100.806±0.0040.806 \pm0.0040.785±0.0100.785 \pm0.010
ConvNeXtV1–TBT0.665±0.0030.665 \pm0.0030.645±0.0110.645 \pm0.0110.857±0.0140.857 \pm0.0140.832±0.0110.832 \pm0.0110.805±0.0090.805 \pm0.0090.757±0.0120.757 \pm0.012
ConvNeXtV1–TBYOL0.691±0.0120.691 \pm0.0120.666±0.0090.666 \pm0.0090.817±0.0050.817 \pm0.0050.851±0.0090.851 \pm0.0090.805±0.0070.805 \pm0.0070.757±0.0130.757 \pm0.013
ConvNeXtV2–TSimCLR0.676±0.0130.676 \pm0.0130.634±0.0030.634 \pm0.0030.852±0.0080.852 \pm0.0080.822±0.0050.822 \pm0.0050.789±0.0030.789 \pm0.0030.744±0.0010.744 \pm0.001
ConvNeXtV2–TMoCo-v20.665±0.0040.665 \pm0.0040.640±0.0080.640 \pm0.008<u>0.867±0.0080.867 \pm0.008</u>0.836±0.0090.836 \pm0.009<u>0.820±0.0090.820 \pm0.009</u>0.780±0.0040.780 \pm0.004
ConvNeXtV2–TMoCo-v30.676±0.0110.676 \pm0.0110.625±0.0090.625 \pm0.0090.863±0.0080.863 \pm0.0080.800±0.0090.800 \pm0.0090.803±0.0100.803 \pm0.0100.725±0.0130.725 \pm0.013
ConvNeXtV2–TDINO0.703±0.0080.703 \pm0.0080.661±0.0020.661 \pm0.0020.846±0.0060.846 \pm0.0060.826±0.0090.826 \pm0.0090.812±0.0010.812 \pm0.001<u>0.790±0.0120.790 \pm0.012</u>
ConvNeXtV2–TBT0.661±0.0030.661 \pm0.0030.635±0.0100.635 \pm0.0100.863±0.0050.863 \pm0.005<u>0.837±0.0040.837 \pm0.004</u>0.774±0.0080.774 \pm0.0080.752±0.0090.752 \pm0.009
ConvNeXtV2–TBYOL0.654±0.0190.654 \pm0.0190.599±0.0160.599 \pm0.0160.851±0.0080.851 \pm0.0080.824±0.0050.824 \pm0.0050.813±0.0090.813 \pm0.0090.716±0.0180.716 \pm0.018
ViT–S/16SimCLR0.668±0.0070.668 \pm0.0070.646±0.0150.646 \pm0.0150.828±0.0050.828 \pm0.0050.839±0.0100.839 \pm0.0100.770±0.0040.770 \pm0.0040.757±0.0100.757 \pm0.010
ViT–S/16MoCo-v20.672±0.0090.672 \pm0.0090.638±0.0060.638 \pm0.0060.852±0.0100.852 \pm0.0100.845±0.0110.845 \pm0.0110.786±0.0050.786 \pm0.0050.764±0.0070.764 \pm0.007
ViT–S/16MoCo-v30.658±0.0060.658 \pm0.0060.644±0.0110.644 \pm0.0110.846±0.0120.846 \pm0.0120.816±0.0240.816 \pm0.0240.784±0.0080.784 \pm0.0080.759±0.0040.759 \pm0.004
ViT–S/16DINO0.692±0.0130.692 \pm0.0130.709±0.0170.709 \pm0.0170.823±0.0180.823 \pm0.0180.872±0.0030.872 \pm0.0030.786±0.0110.786 \pm0.0110.809±0.0090.809 \pm0.009
ViT–S/16BT0.649±0.0060.649 \pm0.0060.662±0.0040.662 \pm0.0040.843±0.0120.843 \pm0.0120.840±0.0070.840 \pm0.0070.775±0.0050.775 \pm0.0050.770±0.0020.770 \pm0.002
ViT–S/16BYOL0.655±0.0170.655 \pm0.0170.671±0.0070.671 \pm0.0070.859±0.0000.859 \pm0.0000.860±0.0090.860 \pm0.009<u>0.801±0.0100.801 \pm0.010</u>0.777±0.0180.777 \pm0.018
Overall median0.6680.6680.6450.6450.8520.8520.8320.8320.8040.8040.7620.762

Table 2. Evaluation of the SSL methods on the fully annotated ISF–5CH dataset: Test set macro F1-scores (mean ± std) for kNN evaluation, linear probing, and XGBoost. θIN\theta_{\mathrm{IN}} and θRnd\theta_{\mathrm{Rnd}} denote initialization from ImageNet-supervised weights and random initialization, respectively. For each evaluation protocol and weight initialization, the best score per backbone is underlined, and the best score across all backbones is highlighted in bold.

Comparative analysis of SSL strategies across backbones and annotated fractions for the kNN evaluation protocol on the ISF–5CH dataset

Figure S2. Comparative analysis of SSL strategies across backbones and annotated fractions for the kNN evaluation protocol on the ISF–5CH dataset. Each cell displays the test set mean macro F1-score ±\pm standard deviation across three runs for a specific backbone, SSL method, and weights initialization strategy. The left (green) and right (blue) halves of each cell represent random initialization (θRnd\theta_{\mathrm{Rnd}}) and ImageNet pretraining (θIN\theta_{\mathrm{IN}}) respectively, with color intensity mapped to absolute performance [0,1]. To highlight relative performance, we underline the leading SSL method within each specific backbone. Bold entries denote the globally optimal SSL configuration across all evaluated backbones for a given data fraction and initialization mode. Bottom rows indicate the median SSL performance per fraction and initialization. Values have been rounded to two decimal places to enhance legibility relative to Table 2. Furthermore, font colors within cells dynamically switch between black and white based on background intensity to ensure maximum readability of the results.

Comparative analysis of SSL strategies across backbones and annotated fractions for the linear probing evaluation protocol on the ISF–5CH dataset

Figure S3. Figure S3. Comparative analysis of SSL strategies across backbones and annotated fractions for the linear probing evaluation protocol on the ISF–5CH dataset. Each cell displays the test set mean macro F1-score ±\pm standard deviation across three runs for a specific backbone, SSL method, and weights initialization strategy. The left (green) and right (blue) halves of each cell represent random initialization (θRnd\theta_{\mathrm{Rnd}}) and ImageNet pretraining (θIN\theta_{\mathrm{IN}}) respectively, with color intensity mapped to absolute performance [0,1]. To highlight relative performance, we underline the leading SSL method within each specific backbone. Bold entries denote the globally optimal SSL configuration across all evaluated backbones for a given data fraction and initialization mode. Bottom rows indicate the median SSL performance per fraction and initialization. Values have been rounded to two decimal places to enhance legibility relative to Table 2. Furthermore, font colors within cells dynamically switch between black and white based on background intensity to ensure maximum readability of the results.

Comparative analysis of SSL strategies across backbones and annotated fractions for the XGBoost evaluation protocol on the ISF–5CH dataset

Figure S4. Comparative analysis of SSL strategies across backbones and annotated fractions for the XGBoost evaluation protocol on the ISF–5CH dataset. Each cell displays the test set mean macro F1F1-score ±\pm standard deviation across three runs for a specific backbone, SSL method, and weights initialization strategy. The left (green) and right (blue) halves of each cell represent random initialization (θRnd\theta_{\mathrm{Rnd}}) and ImageNet pretraining (θIN\theta_{\mathrm{IN}}) respectively, with color intensity mapped to absolute performance [0,1][0,1]. To highlight relative performance, we underline the leading SSL method within each specific backbone. Bold entries denote the globally optimal SSL configuration across all evaluated backbones for a given data fraction and initialization mode. Bottom rows indicate the median SSL performance per fraction and initialization. Values have been rounded to two decimal places to enhance legibility relative to Table 2. Furthermore, font colors within cells dynamically switch between black and white based on background intensity to ensure maximum readability of the results.

Zooming into specific annotated fractions, these patterns become more pronounced. A closer inspection of the low-data regime reveals that self-supervised pretraining closes the gap to, and in some cases surpasses, traditional supervised learning under extreme annotation scarcity (Figure S2, Figure S3, Figure S4). Specifically, at 1% annotated fraction of DannD_{\mathrm{ann}}, pretraining with DINO or MoCo-v2 combined with downstream linear probing or XGBoost classification outperforms the standard end-to-end supervised baseline. Under θIN\theta_{\mathrm{IN}}, the best supervised baseline (ViT–S/16 with 37.1±3.6%37.1 \pm3.6\% F1-score) was outperformed by 1.7 pp when pretraining a ResNet50 with MoCo-v2 followed by XGBoost, and by 8.9 pp when combining DINO with ConvNeXtV1–T and applying linear probing. This margin is amplified under θRnd\theta_{\mathrm{Rnd}} where DINO with ConvNeXtV2–T achieved 40.8±3.1%40.8 \pm3.1\% after XGBoost and 47.4±6.5%47.4 \pm6.5\% after linear probing, both exceeding the best supervised baseline (ResNet50 with 35.4±5.1%35.4 \pm5.1\%), demonstrating that the learned representations capture highly discriminative signal features even when downstream annotation availability is minimal. From 5%5\% annotated Dann\mathcal{D}_{\mathrm{ann}} onward, ConvNeXtV1–T–θRnd\theta_{\mathrm{Rnd}} with MoCo-v2 pretraining followed by linear probing outperforms the best end-to-end supervised baseline under θRnd\theta_{\mathrm{Rnd}} (Figure S3). This shows that self-supervised pretraining approaches the performance of a supervised baseline or a fully finetuned SSL model without the computational burden of end-to-end training. While frozen latent space evaluations are not expected to exceed the upper bound of full SSL finetuning, they remain highly competitive even under extreme annotation scarcity, a regime where supervised baselines fail to generalize.

Self-Supervised Learning Resolves Complex Immunological Structures in Sparse Annotation Regimes

A persistent challenge in IFC is the understanding of complex structures, particularly the accurate detection of multi-cellular events [57]. Our results demonstrate that SSL mitigates this bottleneck. Following our systematic evaluation of various SSL configurations, we finetuned the leading ones to assess the transferability of their self-supervised features and to quantify their effectiveness as weight initializations for downstream classification. As shown in Figure 4D, finetuning our pretrained models yields improvements over the fully supervised baselines. The best combinations proposed by the XGBoost protocol (ResNet50–θRnd\theta_{\mathrm{Rnd}} and ResNet50–θIN\theta_{\mathrm{IN}} both with MoCo-v3) showed the smallest improvements, with ResNet50–θRnd\theta_{\mathrm{Rnd}} + MoCo-v3 even falling under the supervised baseline from 25% annotated Dann\mathcal{D}_{\mathrm{ann}} onward (Figure 4D). The best finetuning performance was achieved with ConvNeXtV1–T–θRnd\theta_{\mathrm{Rnd}} + MoCo-v2, surpassing the best supervised baseline on full Dann\mathcal{D}_{\mathrm{ann}} at 25% (a little less than 1,000 samples) of Dann\mathcal{D}_{\mathrm{ann}} (90.8±0.1%90.8 \pm0.1\% against 89.3±1.0%89.3 \pm1.0\% F1-score), and achieving 93.0±0.3%93.0 \pm0.3\% on full Dann\mathcal{D}_{\mathrm{ann}}. This demonstrates that self-supervised pretraining serves as a robust initialization state that accelerates downstream convergence. While these global metrics illustrate clear cross-fraction success, a granular, class-specific examination reveals where the true strength of self-supervised representations lies. When investigating the source of improvements between the supervised baseline and our top finetuned model (Figure 4D, E), we noticed that pretraining disentangles complex structures. At 1% and 5% of Dann\mathcal{D}_{\mathrm{ann}} where the largest improvements in F1-score occur, the largest class-specific gain stems from the class "B cell and T cell in one layer" (Figure 2C) with deltas of 38.0 pp and 36.5 pp respectively (Figure 4E). This class corresponds to events in which the B and T cell completely overlap [42]; the resulting colocalized high intensities create ambiguous feature profiles that commonly lead classifiers to assign these examples to single cell classes (in our case B cell or T cell with or without signaling) [58]. At 10% and 25% of Dann\mathcal{D}_{\mathrm{ann}}, the boost comes from classes "T cell with signaling" with ΔF1=22.3\Delta F1 = 22.3 pp and "Synapses without signaling" with ΔF1=13.5\Delta F1 = 13.5 pp respectively (Figure 4E), showing that SSL separated signaling samples more cleanly. Biologically, these results suggest that self-supervised pretraining successfully learns to encode generalized spatial boundaries, cell-to-cell adhesion geometries, and relative morphological proportions from unannotated data. When the network is subsequently finetuned on scarce annotations, it possesses the necessary structural priors to identify complex cellular configurations.

ImageNet Pretraining Enhances Performance for Morphological Cell Shape Classification

On the RBC dataset, conversely to the ISF–5CH dataset, initializing the backbones with θIN\theta_{\mathrm{IN}} yielded better performances, with median ΔF1\Delta F1 for the full Dann\mathcal{D}_{\mathrm{ann}} of 24.8 pp on kNN, 22.5 pp on linear probing, and 22.5 pp on XGBoost (Table 3). This indicates that ImageNet-pretrained initializations provide a substantial advantage to downstream SSL for biological and clinical tasks where feature extraction heavily relies on morphological analysis.

BackboneMethodkNNLinear probingXGBoost
θRnd\theta_{\mathrm{Rnd}}θIN\theta_{\mathrm{IN}}θRnd\theta_{\mathrm{Rnd}}θIN\theta_{\mathrm{IN}}θRnd\theta_{\mathrm{Rnd}}θIN\theta_{\mathrm{IN}}
ResNet50SimCLR0.579 ± 0.0370.796 ± 0.0020.752 ± 0.0180.873 ± 0.0080.698 ± 0.0130.891 ± 0.008
ResNet50MoCo-v20.461 ± 0.0260.829 ± 0.0200.651 ± 0.0010.865 ± 0.0100.641 ± 0.0260.901 ± 0.014
ResNet50MoCo-v30.481 ± 0.0320.843 ± 0.0250.585 ± 0.1750.857 ± 0.0130.663 ± 0.0550.902 ± 0.004
ResNet50DINO0.558 ± 0.0060.701 ± 0.0270.754 ± 0.0300.829 ± 0.0030.755 ± 0.0110.826 ± 0.013
ResNet50BT0.593 ± 0.0060.814 ± 0.0090.774 ± 0.0140.891 ± 0.0120.722 ± 0.0070.904 ± 0.004
ResNet50BYOL0.459 ± 0.0000.547 ± 0.0000.605 ± 0.0000.637 ± 0.0000.647 ± 0.0000.665 ± 0.000
ConvNeXtV1–TSimCLR0.134 ± 0.0280.789 ± 0.0150.325 ± 0.1140.892 ± 0.0080.432 ± 0.1230.879 ± 0.009
ConvNeXtV1–TMoCo-v20.261 ± 0.0460.790 ± 0.0040.263 ± 0.0600.903 ± 0.0070.356 ± 0.0750.896 ± 0.001
ConvNeXtV1–TMoCo-v30.192 ± 0.0070.791 ± 0.0270.179 ± 0.0380.900 ± 0.0110.250 ± 0.0170.880 ± 0.014
ConvNeXtV1–TDINO0.183 ± 0.0000.678 ± 0.0000.073 ± 0.0000.828 ± 0.0000.228 ± 0.0000.791 ± 0.000
ConvNeXtV1–TBT0.246 ± 0.0850.726 ± 0.0140.311 ± 0.0310.857 ± 0.0210.434 ± 0.0560.841 ± 0.015
ConvNeXtV1–TBYOL0.185 ± 0.0000.469 ± 0.3440.124 ± 0.0000.423 ± 0.5270.249 ± 0.0000.469 ± 0.344
ConvNeXtV2–TSimCLR0.773 ± 0.0140.794 ± 0.0070.891 ± 0.0120.902 ± 0.0080.871 ± 0.0040.889 ± 0.007
ConvNeXtV2–TMoCo-v20.748 ± 0.0000.785 ± 0.0070.903 ± 0.0070.910 ± 0.0080.863 ± 0.0100.875 ± 0.001
ConvNeXtV2–TMoCo-v30.828 ± 0.0070.753 ± 0.0140.905 ± 0.0000.884 ± 0.0000.904 ± 0.0000.853 ± 0.007
ConvNeXtV2–TDINO0.185 ± 0.0000.590 ± 0.1430.135 ± 0.0300.694 ± 0.1750.238 ± 0.0020.726 ± 0.098
ConvNeXtV2–TBT0.780 ± 0.0060.817 ± 0.0070.893 ± 0.0030.898 ± 0.0080.862 ± 0.0080.886 ± 0.001
ConvNeXtV2–TBYOL0.098 ± 0.0070.094 ± 0.0080.127 ± 0.0070.137 ± 0.0300.305 ± 0.0010.230 ± 0.035
ViT–S/16SimCLR0.504 ± 0.0040.630 ± 0.0160.675 ± 0.0360.806 ± 0.0280.612 ± 0.0220.755 ± 0.014
ViT–S/16MoCo-v20.508 ± 0.0200.612 ± 0.0190.657 ± 0.0300.768 ± 0.0230.623 ± 0.0340.722 ± 0.003
ViT–S/16MoCo-v30.488 ± 0.0110.610 ± 0.0170.639 ± 0.0090.774 ± 0.0010.604 ± 0.0120.720 ± 0.011
ViT–S/16DINO0.413 ± 0.0150.707 ± 0.0620.492 ± 0.0420.793 ± 0.0390.522 ± 0.0090.788 ± 0.049
ViT–S/16BT0.474 ± 0.0320.645 ± 0.0040.632 ± 0.0020.800 ± 0.0020.587 ± 0.0230.748 ± 0.018
ViT–S/16BYOL0.330 ± 0.0200.576 ± 0.1260.321 ± 0.0590.699 ± 0.1130.411 ± 0.0060.676 ± 0.101
Overall median0.4680.7160.6180.8430.6080.833

Table 3. Evaluation of the SSL methods on the fully annotated RBC dataset: Test set macro F1-scores (mean ± std) for kNN evaluation, linear probing, and XGBoost. θIN\theta_{\mathrm{IN}} and θRnd\theta_{\mathrm{Rnd}} denote initialization from ImageNet-supervised weights and random initialization, respectively. For each evaluation protocol and weight initialization, the best score per backbone is underlined, and the best score across all backbones is highlighted in bold.

Further analysis revealed that contrastive methods (SimCLR, MoCo-v2 and MoCo-v3) and BT perform better, particularly when using the ConvNeXtV2–T backbone initialized with θIN\theta_{\mathrm{IN}} (Figure 5A). Similar to the ISF–5CH dataset, linear probing already achieves competitive performance, with the top SSL combination (ConvNeXtV2–T–θIN\theta_{\mathrm{IN}} with MoCo-v2) being only 1.2 pp below the best supervised baseline in F1-score (ConvNeXtV2–T–θIN\theta_{\mathrm{IN}}), followed by ConvNeXtV2–T–θRnd\theta_{\mathrm{Rnd}} + MoCo-v3 with 1.7 pp (Table 3). A visible trend is the general poor performance of BYOL (Table 3, Figure 5A, B), confirming reported instability issues [59]. We additionally observe that DINO performs poorly on the smaller RBC dataset compared to its performance on the larger ISF–5CH dataset, unless it is paired explicitly with a ViT (Table 2, Table 3). This pattern is consistent with DINO having been developed and tuned on ViTs, for which its multi-crop strategy and high-dimensional prototype head were introduced. The convolutional configurations share the objective but not the architecture it was designed around. Because method specific hyperparameters were mostly held at their published defaults, we cannot separate an intrinsic interaction between objective and architecture from a suboptimal configuration for the convolutional backbones.

Benchmarking self-supervised representations for the RBC dataset

Figure 5. Benchmarking self-supervised representations for the RBC dataset. (A) Ranking of combinations of SSL methods and backbones across downstream evaluation protocols. Heatmaps display the rank of 48 model combinations (six SSL methods times four backbones times two initialization strategies) across kNN, linear probing, and XGBoost protocols for each annotated fraction. Each cell is split to compare initialization θRnd\theta_{\mathrm{Rnd}} (left/green, 24 ranks) vs. θIN\theta_{\mathrm{IN}} (right/blue, 24 ranks). Darker hues indicate superior ranking (rank 1). Markers identify the corresponding backbone architecture for the specific configuration. (B) SSL method performance ranking for each backbone from each annotated fraction based on linear probing evaluation. Markers with bolded edges represent θIN\theta_{\mathrm{IN}} and normal edges θRnd\theta_{\mathrm{Rnd}}. (C) Architectural robustness and initialization bias: Comparison of average backbone rank after linear probing between initializations θRnd\theta_{\mathrm{Rnd}} and θIN\theta_{\mathrm{IN}}. Error bars indicate the inter SSL method standard deviation. (D) Finetuning trajectories of top-ranked configurations. For each initialization mode (θRnd\theta_{\mathrm{Rnd}} or θIN\theta_{\mathrm{IN}}) and evaluation protocol, the best-performing SSL-backbone pairing as indicated in Table 3 was selected. (E) Finetuning performance gain between the best supervised baseline at 100% of DannD_{\mathrm{ann}} (ConvNeXtV2–T–θIN\theta_{\mathrm{IN}}) and the best SSL combination for each annotated fraction from (E). The table displays the absolute improvements (percentage point delta) in overall macro-F1-score along with the single class showing the most substantial improvement for each annotated fraction of DannD_{\mathrm{ann}}. Fractions on the x-axis in panels (B), (C), and (D) are displayed on a logarithmic scale for clarity in low-data regimes. Ranks (1 = top performer) per annotated fraction (0.1%–100%) in (B) and (C) were calculated based on test set macro F1-scores using a linear probing protocol.

Assessing the general backbone ranking (Figure 5C) reveals the following: First, ConvNeXtV2–T emerged as the overall superior backbone, achieving the top average rank and reduced performance variance. This performance gap was particularly pronounced under θRnd\theta_{\mathrm{Rnd}}, where ConvNeXtV2–T consistently performed best across all evaluated data fractions. Second, ConvNeXtV1–T performed poorly under θRnd\theta_{\mathrm{Rnd}}, consistently ranking last across all annotated data fractions, yet benefited disproportionately from θIN\theta_{\mathrm{IN}} initialization, which effectively closed the performance gap and brought its ranking in line with other competitive backbones. Third, the ViT–S/16 exhibited poor overall performance, particularly under θIN\theta_{\mathrm{IN}} where it ranked last alongside exhibiting very high performance variance. Finally, ResNet50 demonstrated stable performance, maintaining a consistent overall average rank of two under both random and ImageNet initializations. Notably, under θIN\theta_{\mathrm{IN}}, ResNet50 excelled as the top-performing model up to the 5% annotated data fraction, after which its ranking slipped slightly to second place and even third place. This indicates that while classical residual architectures remain highly competitive and sample-efficient in low-data regimes, modern convolutional backbones like ConvNeXtV2–T ultimately scale better as more annotated data become available.

A closer look at the evaluation of the specific annotated fraction enables deriving some key insights. At ultra-low scarcity levels, applying kNN to ResNet50–θIN\theta_{\mathrm{IN}} + MoCo-v3 is only 1 pp below the best supervised baseline ResNet50–θIN\theta_{\mathrm{IN}} (Figure S5). Most notably, BT achieves remarkable performances (Figure S6). Under θIN\theta_{\mathrm{IN}} and with 0.1% of Dann\mathcal{D}_{\mathrm{ann}}, ConvNeXtV2–T + BT followed by linear probing improves the best supervised baseline (ResNet50–θIN\theta_{\mathrm{IN}} with 43.7 ± 1.0% macro F1-score) by 11.2 pp (Figure 5B, Figure S6). Similarly, at 1% of Dann\mathcal{D}_{\mathrm{ann}}, ResNet50 pretrained with BT followed by linear probing improved the supervised baseline (ResNet50–θIN\theta_{\mathrm{IN}} with 79.5 ± 0.9% macro F1-score) by 4.3 pp. Furthermore, we notice that representation learning via self-supervised pretraining from random initializations θRnd\theta_{\mathrm{Rnd}} yields more improvements than ImageNet initializations θIN\theta_{\mathrm{IN}}. Specifically, ConvNeXtV2–T–θRnd\theta_{\mathrm{Rnd}} pretrained with MoCo-v3 consistently outperformed the ResNet50–θIN\theta_{\mathrm{IN}} supervised baseline under both XGBoost and linear probing evaluation protocols across almost all data fractions (Figure 5B, Figure S7, Figure S6). This superior performance was stable for the XGBoost evaluation protocol, and at the 1%, 5%, 10%, 50%, and 100% annotated fractions for the linear probing evaluation protocol. At the 25%, MoCo-v2 emerged as the top-performing configuration, while BT excelled in the highly resource-constrained 0.1% regime. This variation suggests that while MoCo-v3 provides the most robust representations overall, alternative SSL objectives can offer specialized advantages under specific annotation scarcity constraints.

Comparative analysis of SSL strategies across backbones and annotated fractions for the kNN evaluation protocol on the RBC dataset

Figure S5. Comparative analysis of SSL strategies across backbones and annotated fractions for the kNN evaluation protocol on the RBC dataset. Each cell displays the test set mean macro F1-score ±\pm standard deviation across three runs for a specific backbone, SSL method, and weights initialization strategy. The left (green) and right (blue) halves of each cell represent random initialization (θRnd\theta_{\mathrm{Rnd}}) and ImageNet pretraining (θIN\theta_{\mathrm{IN}}) respectively, with color intensity mapped to absolute performance [0,1][0,1]. To highlight relative performance, we underline the leading SSL method within each specific backbone. Bold entries denote the globally optimal SSL configuration across all evaluated backbones for a given data fraction and initialization mode. Bottom rows indicate the median SSL performance per fraction and initialization. Values have been rounded to two decimal places to enhance legibility relative to Table 3. Furthermore, font colors within cells dynamically switch between black and white based on background intensity to ensure maximum readability of the results.

Comparative analysis of SSL strategies across backbones and annotated fractions for the linear probing evaluation protocol on the RBC dataset

Figure S6. Comparative analysis of SSL strategies across backbones and annotated fractions for the linear probing evaluation protocol on the RBC dataset. Each cell displays the test set mean macro F1-score ± standard deviation across three runs for a specific backbone, SSL method, and weights initialization strategy. The left (green) and right (blue) halves of each cell represent random initialization (θRnd\theta_{\mathrm{Rnd}}) and ImageNet pretraining (θIN\theta_{\mathrm{IN}}) respectively, with color intensity mapped to absolute performance [0,1]. To highlight relative performance, we underline the leading SSL method within each specific backbone. Bold entries denote the globally optimal SSL configuration across all evaluated backbones for a given data fraction and initialization mode. Bottom rows indicate the median SSL performance per fraction and initialization. Values have been rounded to two decimal places to enhance legibility relative to Table 3. Furthermore, font colors within cells dynamically switch between black and white based on background intensity to ensure maximum readability of the results.

Comparative analysis of SSL strategies across backbones and annotated fractions for the XGBoost evaluation protocol on the RBC dataset

Figure S7. Comparative analysis of SSL strategies across backbones and annotated fractions for the XGBoost evaluation protocol on the RBC dataset. Each cell displays the test set mean macro F1-score ±\pm standard deviation across three runs for a specific backbone, SSL method, and weights initialization strategy. The left (green) and right (blue) halves of each cell represent random initialization (θRnd\theta_{\mathrm{Rnd}}) and ImageNet pretraining (θIN\theta_{\mathrm{IN}}) respectively, with color intensity mapped to absolute performance [0,1]. To highlight relative performance, we underline the leading SSL method within each specific backbone. Bold entries denote the globally optimal SSL configuration across all evaluated backbones for a given data fraction and initialization mode. Bottom rows indicate the median SSL performance per fraction and initialization. Values have been rounded to two decimal places to enhance legibility relative to Table 3. Furthermore, font colors within cells dynamically switch between black and white based on background intensity to ensure maximum readability of the results.

Annotation-Efficient Finetuning: Matching Baselines with Only 5% annotated Data

To assess the potential performance gains from finetuning, we selected the top-performing backbone and SSL method combinations identified during our initial evaluation for downstream finetuning. These optimal configurations included: ConvNeXtV2–T–θRnd\theta_{\mathrm{Rnd}} + MoCo-v3, ResNet50–θIN\theta_{\mathrm{IN}} + MoCo-v3, ConvNeXtV2–T–θIN\theta_{\mathrm{IN}} + MoCo-v2, and ResNet50–θRnd\theta_{\mathrm{Rnd}} + BT (Table 3, Figure 5D). Every finetuned SSL combination consistently outperformed the supervised baseline ConvNeXtV2–T–θIN\theta_{\mathrm{IN}}, demonstrating the utility of representation learning. The most substantial gains were concentrated in extreme low-data regimes. At 0.1% and 1% annotated fractions, we observed remarkable absolute test set macro F1-score increases of 45.7 pp and 45.9 pp, respectively, between the supervised baselines and the corresponding top-performing SSL combinations (Figure 5D, E).

A granular, class-specific diagnostic analysis revealed that this low-data superiority was driven by an improvement in resolving highly challenging cellular phenotypes (Figure 5E). Specifically, at 0.1% DannD_{\mathrm{ann}}, the detection of crenated discs increased by 62.2 pp in F1-score compared to the baseline. Similarly, at 1% of DannD_{\mathrm{ann}}, we observed an F1-score enhancement of 57.0 pp for crenated spheroids. For the remaining annotated fractions, the class-specific performance gains were mostly driven by a drastic reduction in error rates for the "undecidable" cell category (Figure 5E). This category poses the steepest challenge due to its high intra-class variance, containing heterogeneous cell structures, edge-cases, and poorly defined morphological signatures [43].

The fine-grained performance trajectories across annotated fractions further exposed how architectural properties interact with self-supervision methods: First, at 0.1% DannD_{\mathrm{ann}}, the overall best-performing combination was ResNet50–θRnd\theta_{\mathrm{Rnd}} + BT. Second, the vulnerability of ConvNeXtV2–T–θIN\theta_{\mathrm{IN}} at very low data regimes observed in the supervised baselines (Figure 3, Figure S1B) was mitigated but did not completely disappear. From 5% annotated fraction onward, the pretrained SSL configurations stabilized, and ConvNeXtV2–T–θIN\theta_{\mathrm{IN}} + MoCo-v2 emerged as best performer with a macro F1-score of 92.3 ± 1.0, equaling the best supervised baseline trained on full DannD_{\mathrm{ann}} (ConvNeXtV2–T–θIN\theta_{\mathrm{IN}} with 92.2 ± 1.1% macro F1-score) (Table 1). This sudden inflection point underscores the synergistic potential of pairing advanced backbone feature extractors with optimized SSL methods, ultimately enabling highly efficient downstream annotation utilization.

Qualitatively Mapping the Self-Supervised Latent Space

To qualitatively assess the structure and semantics of the learned representations, we visualized the high-dimensional latent space of our best-performing SSL combinations for both θRnd\theta_{\mathrm{Rnd}} and θIN\theta_{\mathrm{IN}} under linear probing using a 2D-Uniform Manifold Approximation and Projection (UMAP) (Figure 6). Because low-dimensional projections distort distances and cannot support quantitative claims, we clustered the embeddings with kk-means and scored the resulting partition against the annotations using the Adjusted Rand Index (ARI) and the Normalized Mutual Information (NMI) (Table 4). On the ISF–5CH dataset, we additionally clustered the embeddings of each experiment separately and compared the resulting within experiment scores against a control in which images were assigned at random to groups of identical size (Table 4).

Latent space visualizations of pretrained backbones for ISF–5CH and RBC datasets

Figure 6. Latent space visualization of the pretrained backbones on full Duna\mathcal{D}_{una} for the (A) ISF–5CH and (B) RBC datasets. We selected for each initialization strategy (θIN\theta_{\mathrm{IN}} or θRnd\theta_{\mathrm{Rnd}}) the best SSL combination after linear probing evaluation and color-coded test samples with annotations. ARI stands for Adjusted Rand Index and NMI for Normalized Mutual Information. Scale bars in illustratives images of (B) represent 5μm5\mu\mathrm{m}. Experiments in (A) represent the individual experiments as described in [42, 44].

GroupingnnkkConvNeXtV1–T–θRnd\theta_{\mathrm{Rnd}} + MoCo-v2 ARIConvNeXtV1–T–θRnd\theta_{\mathrm{Rnd}} + MoCo-v2 NMIViT–S/16–θIN\theta_{\mathrm{IN}} + DINO ARIViT–S/16–θIN\theta_{\mathrm{IN}} + DINO NMI
Annotated class1,16990.0780.1820.1110.259
Experiment1,16940.8110.7920.2120.349
Class within experiment
Experiment 125290.1880.3530.2010.413
Experiment 213990.1600.3980.3060.533
Experiment 348690.1460.2940.1700.347
Experiment 429290.1540.3530.2190.425
Weighted mean1,16990.1590.3340.2050.403
Random control1,16990.0860.2330.1110.288

Table 4. Assessing the cluster structure of the learned representations on the ISF–5CH dataset with ARI and NMI. Unsupervised kk-means clustering of embeddings with kk set to the number of groups and nn representing the number of images. Class within experiment is computed separately for each experiment; the weighted mean averages over experiments proportionally to the number of images. The random control repeats the within-experiment analysis on size-matched random groupings (10 repetitions).

A detailed examination of the individual datasets reveals specific topological behaviors that further illustrate the clinical utility and robustness of self-supervised representations:

  • Experimental Batch Effects vs. Class Clustering: On the ISF–5CH dataset, the latent space projection for both SSL combinations exhibit prominent batch effects governed by the distinct experiments as described in [42, 44] (Figure 6A). This is confirmed when looking at the clustering results on annotations and experiments (Table 4). When grouping by annotated classes, the best SSL combination, ConvNeXtV1–T–θRnd\theta_{\mathrm{Rnd}} + MoCo-v2 achieves an ARI of 0.078 and an NMI of 0.182. These values rise to 0.811 and 0.792 respectively when grouping by experiments. This clustering behavior, typical of multi-batch biological assays with varying experimental conditions, could superficially appear to degrade semantic organization. However, when looking closer into individual batch sub-clusters, we observe a coherent internal organization consisting of boundaries between immunological classes. For example, we notice a separation between T cell images and others

(Figure 6A). This is further noticeable in ConvNeXtV1–T–θRnd\theta_{\mathrm{Rnd}} + MoCo-v2, with higher ARI and NMI within individual experiments (Table 4), which, averaged (ARI = 0.159, NMI = 0.334), surpassed the original grouping by annotated classes and the random control. More importantly, we notice the aforementioned complexity of the class "B cell and T cell in one layer" (B cell and T cell completely overlapping as shown in Figure 2C), which correlates with the substantial improvements noticed in finetuning (Figure 4E). Additionally, the ViT–S/16–θIN\theta_{\mathrm{IN}} + DINO displays a superior capacity to mitigate these persistent batch variations (Table 4). This batch-reconciliation capability is particularly visible in the better clustering based on annotated classes compared to ConvNeXtV1–T–θRnd\theta_{\mathrm{Rnd}} + MoCo-v2 (Figure 6A). This may suggest that the ViT–S/16–θIN\theta_{\mathrm{IN}} + DINO successfully leverages subtle visual features beyond the fluorescent signal to achieve precise B-cell isolation, demonstrating that out-of-domain pretraining enables the network to look past surface-marker ambiguities to capture deep, structural cellular lineages.

  • Pretext-Guided Annotation and Morphology Mapping: On the RBC dataset, the latent spaces of both models showed several clusters. While the randomly initialized latent space ConvNeXtV2–T–θRnd\theta_{\mathrm{Rnd}} + MoCo-v3 struggled with diffuse and noisy clusters, the ImageNet-pretrained configuration ConvNeXtV2–T–θIN\theta_{\mathrm{IN}} + MoCo-v2 established better segregated topology. Although this pretrained latent space displays some local neighborhood overlap, we observe that it successfully disentangles visually similar morphological variants (Figure 6B), specifically resolving crenated spheres from crenated spheroids. This structural division directly aligns with the class-specific performance gains we obtained during finetuning (Figure 5E). Furthermore, the two distinct clusters of smooth discs, crenated discoids, and side cells present in ConvNeXtV2–T–θRnd\theta_{\mathrm{Rnd}} + MoCo-v3 are merged under ConvNeXtV2–T–θIN\theta_{\mathrm{IN}} + MoCo-v2. Of great interest is the distribution of the "undecidable" class. Rather than behaving as an unstructured, random noise overlay, the "undecidable" cells organize into tight, localized sub-pockets embedded directly within the boundaries of major, well-defined phenotypic clusters. This localized segregation suggests that these ambiguous cells share strong, continuous morphological features with their neighboring defined classes. Consequently, these structural mappings serve as a valuable visual taxonomy tool, helping pathologists and experimentalists pinpoint exactly which biological phenotypes these uncertain structures are most morphologically related to, thereby accelerating downstream annotator consensus and streamlining semi-supervised annotating pipelines.

Taken together, these latent space topologies confirm that qualitative feature clustering directly correlates with our quantitative downstream evaluations. They demonstrate that while target-domain self-supervised pretraining (θRnd\theta_{\mathrm{Rnd}}) refines specialized features, the structural priors inherited from natural image pretraining (θIN\theta_{\mathrm{IN}}) could improve the fine-grained morphological boundaries that ultimately drive high-precision downstream analysis.

Discussion and Guidelines

Annotation remains the principal bottleneck in IFC analysis, where acquisition routinely outpaces expert annotating by orders of magnitude. We benchmarked six SSL objectives across four backbones and two initialization strategies on two IFC datasets, holding preprocessing and pretraining scale fixed so that comparisons isolate the objective and the initialization. Self-supervised pretraining reduced annotation requirements substantially and successfully disentangles complex biological phenotypes.

Weights Initialization should be tailored to Task Complexity

Despite some studies suggesting that backbone initialization with ImageNet-supervised weights consistently enhances biomedical image representations [60], recent evaluations have begun to reveal its limitations, specifically showing that supervised natural image priors do not consistently extend to complex downstream tasks and can even degrade robustness under certain SSL combinations [32]. Our results solidify these findings. Crucially, our extensive evaluation reveals that the optimal backbone initialization strategy is highly domain-dependent rather than universal: While fine-grained geometric features in the RBC dataset benefit significantly from natural image weights (θIN\theta_{\mathrm{IN}}) (Table 3, Figure S5, Figure S6, Figure S7), more complex cell structural phenotypes from the ISF–5CH dataset are better resolved via in-domain self-supervision from random initialization (θRnd\theta_{\mathrm{Rnd}}) (Table 2, Figure S2, Figure S3, Figure S4). This pattern was present for each annotation scarcity level, helping address RQ1, RQ2 and providing a proposal for optimizing representation quality under raw imaging constraints, which we leverage here to establish a concrete guideline for backbone initialization.

A Strong Recipe for Learning Linearly Separable Features: MoCo-v2 + ConvNeXt

While claiming, in response to RQ1, that a singular self-supervised paradigm can dominate diverse biological imaging tasks may appear ambitious, our empirical evidence systematically validates this reality. Across our benchmarking experiments under the strict linear probing protocol, the combination of MoCo-v2 with a modern ConvNeXt backbone emerged as the most robust baseline (Table 2, Table 3), thus aligning with recent applications of SSL on biomedical imaging [32, 40]. On the RBC dataset, this peak configuration was achieved using a ConvNeXtV2–T backbone with θIN\theta_{\mathrm{IN}}. Conversely, on the ISF–5CH dataset, the optimal pipeline shifted to a ConvNeXtV1–T backbone trained completely from scratch. These results additionally yield a clear backbone architectural prescription: Practitioners should prioritize modern ConvNeXt or ViT–S/16 backbones over classical residual networks. In our evaluations, the standard ResNet50 showed fluctuations in performance. On the one hand, ResNet50 combined with contrastive methods MoCo-v2 or MoCo-v3 was effective. On the other hand, its performance was worse on the asymmetric methods DINO and BYOL, particularly on the ISF–5CH dataset. This shows ResNet50’s dependency on SSL objectives, suggesting that large-kernel spatial mixing of ConvNeXt or the global receptive fields of ViTs are likely to be less dependent on SSL objectives and better at capturing the fine-grained hierarchical textures of cellular imaging. Similar to other benchmarks [61], we found that some specific SSL combinations are highly compatible. For example, pairing a ViT–S/16 with the DINO framework yielded a potent synergy that achieves competitive results across both evaluation datasets.

Our hierarchical benchmarking and ranking showed a transition under extreme downstream annotation scarcity, where specialized self-supervised paradigms revealed their algorithmic advantages. On the large ISF–5CH dataset, DINO exhibits on modern backbones like ConvNeXts and ViT–S/16 impressive performances at ultra-low data annotation fractions (Figure 4B), showcasing the strength of its local to global self-distillation framework in clustering complex cellular features using minimal supervision [47]. Conversely, on the relatively medium scale RBC dataset, BT dominates the ultra-low regime. In contrast, BYOL exhibits severe performance fluctuations and consistently underperforms across both datasets. This deficit aligns with established literature highlighting its sensitivity to the hyperparameters [59] and suggests that it is susceptible to partial dimensional collapse when exposed to the highly homogeneous and repetitive background profiles of IFC.

Finetuning enables Annotation Efficiency

In our experiments to address RQ2, self-supervised pretraining on unannotated data translated into large annotation efficiency during downstream finetuning. For example, with roughly 1,000 annotated samples on the ISF–5CH dataset, a simple finetune performance exceeded the fully supervised baselines (Figure 4D). On the RBC dataset, we matched the 100% annotation fraction baseline using only 5% of the annotations (Figure 5D). These results imply that, when abundant unannotated IFC data are available, practitioners should pretrain encoders with SSL and then finetune on a very small number of annotated examples: This workflow reliably enables highly annotation-efficient learning, as it held on both datasets despite their differences in cell type, marker panel, class definition and size, which suggests it is not specific to a single assay. We did not, however, evaluate transfer of pretrained encoders between independent IFC studies. Because acquisition panels in IFC are experiment-specific, differing in the number, identity and intensity scaling of channels, encoders pretrained on one panel cannot be applied directly to another without channel adaptation. The backbones released here are therefore intended as initialization for studies using comparable channel configurations, and cross-study transfer, including channel-adaptive pretraining, remains an open question.

Practical Implementation and Normalization Pitfalls

Our experiments highlighted the utility of qualitative performance assessment (Figure 6).

For example, we observed on the ISF dataset that the learned representations were clustered by experiment, suggesting the presence of experiment-level batch effects. However, boundaries and class-specific clusters were still visible in the individual experiment clusters. This serves as strong recommendation for characterizing the latent space qualitatively, as it may provide more insights on the features learned during pretraining.

Our findings suggest that engineering complex domain-specific preprocessing pipelines may yield diminishing returns. Specifically, we advise against introducing cytometric channel scales like the Logicle transformation [54, 55]. While standard in classical flow cytometry analysis, integrating Logicle scaling into visual deep learning pipelines fails to bring any downstream performance improvements over simple z-score normalization.

Additionally, while we did not evaluate channel importance and/or reduction strategies in this study, seminal deep learning work in IFC has systematically demonstrated that omitting channels or restricting models to single modalities discards highly complementary physical and spatial features, leading to a marked degradation in performance [17]. We therefore advise practitioners to avoid restricting themselves to a single modality unless justified.

Limitations and Future Work

While this study establishes baselines for representation learning in IFC, several limitations should be noted.

Regarding scope, our benchmarking framework covers discriminative SSL only; generative and reconstruction-based approaches, such as masked autoencoders and diffusion-based features, were not evaluated. We likewise did not investigate explicit batch-correction techniques to mitigate the multi-run domain shifts reported above, did not ablate domain-specific transformations, and did not quantify the contribution of individual imaging channels. Each of these would require a further multiplication of an already large configuration grid. We also only used z-score normalization during our benchmarking and therefore cannot conclude if this is optimal. Ablating several techniques could be a promising direction.

Regarding the evidence, our conclusions rest on two IFC datasets. Although these differ in cell type, marker panel, class definition and size, two datasets are a narrow basis for claims about IFC in general. Furthermore, each configuration was trained with three seeds, which is sufficient to report the observed variability but limits the resolution at which closely ranked methods can be separated. Addressing these points, in particular through evaluation on additional independent IFC cohorts, remains a promising direction for future work.

Methods

Our modular pipeline illustrated in Figure 1 provides a controlled environment to isolate the factors contributing to representation quality in multispectral IFC data. The following sections detail our methodology in a bottom-up approach: Beginning with the selection and preprocessing of publicly available datasets, followed by the architectural configurations of the backbones, the implementation of diverse SSL paradigms and finally the standardized evaluation protocols used to measure downstream performance.

Data Preparation

At this stage, the raw multispectral IFC data are ingested as a collection of multichannel tensors, where each sample x∈RC×H×Wx \in\mathbb{R}^{C \times H \times W} with CC the number of channels, HH the image height and WW the image width. To facilitate benchmarking, the dataset D\mathcal{D} is partitioned into two disjoint subsets: an unannotated set Duna\mathcal{D}_{\mathrm{una}} for self-supervised pretraining and a annotated set Dann\mathcal{D}_{\mathrm{ann}} reserved for downstream supervised evaluation.

Our framework incorporates a channel selection module to isolate specific biomarkers and a preprocessing suite that supports domain-specific transformations (e.g., Logicle [54, 55] or Hyperlog [62] scaling). While the framework allows for these specialized preprocessing techniques, they were excluded from the primary benchmarking experiments, specifically for those consisting solely of brightfield BF and/or darkfield DF imagery. We reserved the exploration of these domain-specific techniques for our ablation studies, focusing instead on standard channel-wise z-score normalization on the full dataset and spatial resizing (224×224224 \times224) to ensure a uniform input format across all SSL paradigms. We selected for our benchmarking two publicly available IFC datasets:

  • RBC dataset [43], a publicly available medium-scale IFC dataset of single-cell red blood cell images. The dataset consists of an unannotated subset of 245,085 images from three bags on three different days each [43, 63], and a annotated subset of 70,331 samples classified in seven expert-defined morphological categories (Crenated disc, Crenated discoid, Crenated sphere, Crenated spheroid, Side, Smooth disc, Smooth sphere) and one undecidable category [43]. Each sample contains two brightfield channels (BF1 and BF2) and one darkfield channel (DF). For this benchmark, we utilize all available channels BF1, BF2, and DF to ensure a robust evaluation of representation capacity. According to [43], samples were acquired using the Amnis ImageStreamX Mark II and preprocessed with the corresponding software IDEAS. Dataset statistics and representative class examples are shown in Figure 2A and B. The splits were taken from the published data in the following manner: The folder named "Training Test 2 Canada Swiss" was used for the training, "Test 3 4 Holdout Canada Swiss" was used for the validation, and "Test 1 Consensus ground truth" was used for test. For this dataset, the unannotated partition set Duna\mathcal{D}_{\mathrm{una}} consisted of four-channel (an additional DF to the three channels aforementioned in the annotated subset) images of varying spatial dimensions, whereas the annotated subset Dann\mathcal{D}_{\mathrm{ann}} comprised uniform 48×4848 \times48 pixel images restricted to three aforementioned channels. To maintain channel consistency between the pretraining and evaluation phases, pretraining on Duna\mathcal{D}_{\mathrm{una}} was restricted to these same three channels.

  • ISF dataset, one of the largest publicly available IFC dataset [42, 44]. It profiles immunological synapse formation between human memory CD4⁺ T cells and Epstein-Barr virus (EBV) B lymphoblastoid cells (B-LCLs) [42, 44]. T cells from nine donors were co-cultured with Staphylococcus aureus enterotoxin A (SEA)-pulsed or untreated B-LCLs and imaged on an Amnis ImageStreamX Mark II across four independent experiments, yielding more than 2.89 million images [42, 44] as shown in Figure 2D. Each sample contains brightfield (BF) and fluorescence channels (CD45, Antibody, F-actin, MHCII, CD3, P–CD3ζ) plus a live/dead stain (L/D). A subset of 9,186 images was manually annotated by an expert immunologist into 17 classes [17, 42]. For this study, we focus on the nine biologically relevant classes (B cell, T cell, T cell with signaling, B cell and T cell in one layer, T cell with B cell fragments, Synapse without signaling, Synapse with signaling, No cell cell interaction, and Multiplets) as illustrated in Figure 2C, yielding a filtered set of 5,848 samples. The ignored classes consisted mostly of undesired samples (Unknown, Dead cell, Out of focus, Unfocused synapses, Cropped images, No synapse, Unfocused cells, and CD3+MHCII+cells). The obtained subset is split for the downstream tasks into training, validation, and test with a ratio of 64-16-20.

All images across both the RBC and ISF datasets were acquired at 60× magnification with a native spatial resolution of 0.33 micrometers per pixel. We follow the original studies and use the same channels for the benchmarking. All images of both datasets were resized to 224×224224 \times224.

To benchmark downstream models under varying degrees of annotation scarcity, Dann\mathcal{D}_{\mathrm{ann}} was partitioned into nested subsets representing 1%, 5%, 10%, 25%, 50%, and 100% of the total available training set. To prevent class imbalance from biasing evaluation at ultra-low regimes (such as 1%), all subsets were generated using stratified random sampling to preserve the parental class distributions. We maintained consistent random seeds across experiments to ensure that smaller target subsets (e.g., 1%) were strictly nested subsets of larger ones (e.g., 5%), isolating data volume as the sole experimental variable.

Self-Supervised Pretraining

As illustrated in Figure 1, the second stage of our pipeline is a fully decoupled pretraining module designed to evaluate the synergy between model architecture and self-supervised objectives systematically. The construction of the pretraining environment follows a structured and modular hierarchy.

Backbones. To ensure the generalizability of our benchmarking results, we integrate four distinct backbone architectures that represent the core paradigms of modern computer vision. We employ ResNet50 [49] as a convolutional baseline to assess feature extraction via standard residual learning. To contrast this traditional approach with global dependency modeling, we incorporate a ViT–S/16 [50], which utilizes self-attention mechanisms to process imagery as sequential patches. Furthermore, we include ConvNeXtV1–T [51] and its successor, ConvNeXtV2–T [52], which represent modernized convolutional designs that adopt Transformer-inspired hierarchies, such as larger kernels, while maintaining convolutional efficiency. Critically, we specifically utilize the Tiny and Small variants of ConvNeXts and ViT–S/16 to maintain a comparable parameter budget across the different architectural families, ensuring that performance variations are attributable to structural differences rather than sheer model capacity. As detailed in Table 5, these models span a wide spectrum of depths, and receptive field sizes. Each architecture is evaluated under two initialization regimes: random (θRnd\theta_{\mathrm{Rnd}}), to measure the raw representation learning on IFC data, and ImageNet-pretrained (θIN\theta_{\mathrm{IN}}), to quantify the degree of knowledge transfer from natural image domains to the biological landscape.

Backbone# ParametersBlocksKernel/Patch size
ResNet50∼23.5\sim23.5 M163×33 \times3
ConvNeXtV1–T∼27.9\sim27.9 M187×77 \times7
ConvNeXtV2–T∼27.9\sim27.9 M187×77 \times7
ViT–S/16∼21.7\sim21.7 M1216×1616 \times16

Table 5. Overview of the benchmarked backbone architectures. Parameter counts expressed in millions.

To address the mismatch between the three-channel input of ImageNet-pretrained models (Cin=3C_{\mathrm{in}} = 3) and the multispectral nature of IFC data (Cin=NC_{\mathrm{in}} = N), we utilize a standardized weight-averaging adaptation. Let the pretrained first-layer kernels be Wpre∈RD×3×K×KW_{\mathrm{pre}} \in\mathbb{R}^{D \times3 \times K \times K} where DD is the number of filters, 3 the RGB depth, and KK the kernel size. To initialize the new weight tensor Wtarget∈RD×Cin×K×KW_{\mathrm{target}} \in\mathbb{R}^{D \times C_{\mathrm{in}} \times K \times K} without losing the pre-trained feature detectors, we apply channel-wise mean replication. We first compute the average filter across the three RGB channels to create a gray-equivalent filter template Wμ∈RD×1×K×KW_{\mu} \in\mathbb{R}^{D \times1 \times K \times K} as indicated in Equation (1) where Wpre(j)W_{\mathrm{pre}}^{(j)} is the weight slice for the jj-th color channel.

Wμ=13∑j=13Wpre(j)(1)W_{\mu} = \frac{1}{3}\sum_{j=1}^{3} W_{\mathrm{pre}}^{(j)} \tag*{(1)}

We then replicate this mean filter across all CinC_{\mathrm{in}} channels of the target architecture: Wtarget(i)=Wμ,i∈{1,…,Cin}W_{\mathrm{target}}^{(i)} = W_{\mu}, i \in\{1, \ldots, C_{\mathrm{in}}\}. Because this study aims to compare self-supervised objectives and backbones rather than to optimise channel handling, we opted for this simple trick and did not use any of the current specialized methods [64, 65, 66].

Supervised Baseline. To contextualize the quality of the learned self-supervised representations, we established a supervised baseline for each dataset, backbone, and initialization strategy. These baseline models are trained under different scarcity levels with weighted cross-entropy loss and weighted sampling, representing, for each scarcity level, the empirical performance baseline. By comparing results obtained from SSL against this ceiling, we can quantify the supervised-SSL gap and evaluate how effectively our framework captures the feature space required for biological classification under various annotation scarcity level. The corresponding hyperparameter values are detailed in Table 7.

HyperparameterBaselineFinetuning
Image size224 ×\times 224224 ×\times 224
Batch size128128
Epochs200200
Learning Rate (LR) end-to-end1e−31e^{-3}–
LR backbone–1e−41e^{-4}
LR head–1e−31e^{-3}
Weight decay1e−21e^{-2}1e−41e^{-4}
Early stopping patience5 epochs5 epochs
OptimizerAdamWAdamW
LR schedulerCosineAnnealingLRCosineAnnealingLR

Table 7. Hyperparameters used for supervised trainings. The batch size was accordingly adjusted to eight when the training set had less then 128 samples. "–" indicates unavailability.

Self-Supervised Learning Methods. Our framework supports a diverse and extensible registry of SSL methods, each characterized by its architectural head and loss function. We however focus here on invariance-based SSL. This registry therefore incorporates symmetric contrastive approaches like SimCLR [45], which uses a shared Multilayer Perceptron (MLP) projection head to maximize agreement between two augmented views, and momentum-based variants such as MoCo-v2 [39] and MoCo-v3 [46]. While MoCo-v2 utilizes a queue of negative samples to decouple contrastive memory from batch size, MoCo-v3 adopts an asymmetric formulation with an additional prediction head and synchronized batch statistics. The framework also encompasses asymmetric teacher-student architectures like BYOL [48], which relies on a predictor and stop-gradient operations to prevent collapse without negative samples, and DINO [47], which utilizes a multi-crop strategy for robust self-distillation. Furthermore, the redundancy-reduction approach of BT [41] is supported to explicitly penalize correlation across embedding dimensions. By mostly preserving these method specific components, our framework enables a fair and systematic evaluation of how diverse self-supervised methods impact feature quality and downstream performance. A detailed explanation of the benchmarked SSL methods and their corresponding training hyperparameters is provided in the Supplementary Information (Supplementary Section ’SSL Methods’ and Supplementary Table 9).

ParametersSimCLRMoCo-v2MoCo-v3BYOLBTDINO
Common
Batch size128
Embed. dim256
Epochs200
Optimization Batch Normalization (BN)
OptimizerLARSSGDSGDLARSLARSSGD
LR0.150.0150.0150.150.150.015
Weight decay1e−61\mathrm{e}{-6}1e−41\mathrm{e}{-4}1e−41\mathrm{e}{-4}1e−61\mathrm{e}{-6}1e−61\mathrm{e}{-6}1e−41\mathrm{e}{-4}
Optimization Layer Normalization (LN)
OptimizerAdamW
LR3e−43\mathrm{e}{-4}
Weight decay0.1
Method-Specific
Temperature0.20.20.2––0.1
EMA–0.9960.9960.996–0.996
Memory bank–65536––––
Bottleneck–––––64
Lambda––––0.005–

Table 9. Hyperparameters used during self-supervised pretrainings. Common values to all methods are centralized (under MoCo-v3). "–" indicates unavailability. Pretraining on the ISF–5CH dataset occurred for only 10 epochs.

Pretraining Data Augmentation. To ensure a rigorous and controlled benchmarking environment, we implement a standardized data augmentation protocol across all evaluated SSL methods. For methods following a standard two-view strategy (i.e., SimCLR, MoCo-v2, MoCo-v3, BT, and BYOL), images are processed at a global resolution of 224×224224 \times224 pixels. For the multi-crop method DINO, we maintain consistent view configurations, utilizing two global views at 224×224224 \times224 along with 6 local crops at 96×9696 \times96. The geometric transformation pipeline consists of horizontal and vertical flips, rotations, and translations. Photometric augmentations include Gaussian blur and noise to account for IFC optical characteristics. All parameters for these augmentations can be found in Table 8.

Augmentation techniqueTwo-viewDINO
Global image size224×224224 \times224224×224224 \times224
Local image size–96×9696 \times96
Number of views (global, local)2,02,02,62,6
Horizontal flip probability0.50.50.50.5
Vertical flip probability0.50.50.50.5
Global crop scale0.08−10.08 - 10.4−10.4 - 1
Local crop scale–0.05−0.40.05 - 0.4
Rotation0−1800 - 1800−1800 - 180
Translation0.150.150.150.15
Gaussian blur probability0.50.51.0,0.1,0.51.0, 0.1, 0.5
Gaussian blur kernel23×2323 \times2323×2323 \times23
Gaussian blur sigma0.1−20.1 - 20.1−20.1 - 2
Gaussian noise probability0.20.20.20.2
Gaussian noise sigma0.10.10.10.1

Table 8. List of data augmentation parameters for self-supervised pretraining. The methods using a two-view strategy are SimCLR, MoCo-v2 & v3, BYOL, and BT. DINO uses a multi-crop strategy and the probability of applying Gaussian blur follows the original publication (probability of 1.0 for the first global crop, 0.1 for the second global crop and 0.5 for all local crops). "–" indicates unavailability.

In contrast to standard self-supervised pipelines for natural images, we exclude color jittering and solarization as they can destroy biologically relevant intensity information [67]. By maintaining these augmentation parameters constant across all experiments, including synchronized crop scales and noise sigmas, we ensure that performance variations are attributable strictly to the self-supervised paradigms and backbone configurations under study, rather than disparate data-view policies.

Downstream Analysis

To evaluate the quality of the learned representations, we follow the established standard in the field [68] and adopt a multi-tiered evaluation protocol across three standard regimes:

  • kNN classification on the l2-normalized extracted feature representations from the frozen backbone with k=5k = 5 and cosine distance as the proximity metric and neighbor contributions weighted uniformly. This non-parametric approach enables evaluating local feature space density.

  • Linear probing with a multinomial logistic regression classifier trained directly on the z-score-normalized extracted feature representations from the frozen backbone. Optimization was carried out using the Limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) solver, configured with 1,000 maximum iterations to guarantee full gradient convergence across all evaluation partitions.

  • End-to-end finetuning to determine the maximum achievable performance for each task. All hyperparameters can be found in Table 7.

Additionally, we incorporated an XGBoost classifier as a computationally efficient proxy for characterizing non-linear information within the learned representations. Tree-based ensembles like XGBoost are architectural staples in bioimaging and particularly in IFC [42], due to their robustness on tabular feature sets and lower computational overhead compared to end-to-end neural network optimization. XGBoost was applied with the multi-class softmax objective, 500 boosting rounds and library defaults. Full end-to-end finetuning was performed only on the top-performing SSL configurations identified from the evaluation protocols kNN, linear probing, and XGBoost for each initialization strategy (random vs. ImageNet-pretrained). Quantitative performance is measured using the macro F1-score, ARI, and NMI, while the qualitative structure of the latent space is visualized through UMAP [69] embeddings. For UMAP, embeddings were l2-normalized and high-dimensional distances were calculated using cosine similarity, 15 nearest neighbors and a minimum distance parameter of 0.4.

Training and Implementation Details

All models were developed in PyTorch 2.7 and trained on NVIDIA A100 GPUs with 40 GB VRAM. We maintained a fixed seeding strategy to ensure reproducibility of the stochastic components. Final results represent the aggregate of three runs, reported via mean and standard deviation. Further implementation details can be found in the supplementary materials.

To adapt the self-supervised pretraining to our high-throughput IFC data under practical hardware constraints, we utilized distinct epoch budgets for the two datasets. For the RBC dataset (245,085 images), all models were pretrained for 200 epochs as indicated in Table 9. Conversely, pretraining on the larger ISF–5CH dataset (more than 2.89 million images) was restricted to 10 epochs. We justify this highly compact 10-epoch regime on the larger ISF–5CH dataset through both empirical stability and domain-specific visual characteristics. Unlike natural image datasets, such as ImageNet-1k, which exhibit high spatial entropy and diverse semantic classes, IFC data are highly structured, consisting exclusively of roundish cellular morphologies on a uniform background. Under these specialized conditions, the pretraining task is primarily governed by signal learning, specifically, the extraction of micro-textures, marker localization, and signal intensity distributions across channels.

To verify the correctness and stability of the benchmarking pipeline, we use the BloodMNIST [70, 71], a publicly available dataset of the MedMNIST collection [31]. BloodMNIST contains blood cell images annotated into eight classes (Basophil, Eosinophil, Erythroblast, Immature Granulocyte, Lymphocyte, Monocyte, Neutrophil, and Platelet). BloodMNIST is not an IFC dataset and was acquired with the CellaVision DM96 system and consists of standard three-channel RGB images. As indicated in Table 6, we achieve similar performances to those from the benchmark [31].

BackboneBloodMNIST
MedMNIST ResNet50 [1]0.950
MedMNIST ResNet18 [1]0.963
ResNet50–θRnd\theta_{\mathrm{Rnd}}0.987 ±\pm 0.000
ResNet50–θIN\theta_{\mathrm{IN}}0.990 ±\pm 0.000
ConvNeXtV1–T–θRnd\theta_{\mathrm{Rnd}}0.947 ±\pm 0.000
ConvNeXtV1–T–θIN\theta_{\mathrm{IN}}0.990 ±\pm 0.000
ConvNeXtV2–T–θRnd\theta_{\mathrm{Rnd}}0.967 ±\pm 0.000
ConvNeXtV2–T–θIN\theta_{\mathrm{IN}}0.990 ±\pm 0.001
ViT–S/16–θRnd\theta_{\mathrm{Rnd}}0.955 ±\pm 0.000
ViT–S/16–θIN\theta_{\mathrm{IN}}0.958 ±\pm 0.000

Table 6. Accuracies of the supervised end-to-end baselines (mean ±\pm std) on the BloodMNIST dataset. θIN\theta_{\mathrm{IN}} and θRnd\theta_{\mathrm{Rnd}} denote initialization from ImageNet-supervised weights and random initialization, respectively. We report accuracies here for consistency with the MedMNIST convention [1]. The best-performing supervised baseline for each dataset is bolded.

Data and Code Availability

All datasets analyzed in this study are publicly available from the repositories associated with their original publications and no new datasets were generated. All code developed for this study, covering self-supervised pretraining, benchmarking and analysis, is publicly available under the MIT License at https://github.com/hciays/IFC-SSL-Bench. The repository includes the configuration file used across all experiments, so that all results can be reproduced from the public data. Pretrained weights for the best-performing configuration identified in this study for each dataset will be deposited at Zenodo under the CC BY 4.0 license and released upon acceptance of the manuscript. The complete set of checkpoints across all method, backbone, initialization and seed combinations is not deposited owing to its size; all configurations can be reproduced from the code and configuration files in the repository, and individual checkpoints are available from the lead contact upon reasonable request. Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon reasonable request.

Acknowledgements

The Helmholtz Association funds this project under the research school "Helmholtz Information and Data Science School for Health (HIDSS4Health)", the program "Natural, Artificial and Cognitive Information Processing (NACIP)" and the Initiative and Networking Fund on the HAICORE@KIT partition. The state of Baden-Württemberg supports this project through bwHPC.

Author Contributions

The authors have accepted responsibility for the entire content of this manuscript and approved its submission. We describe here the individual contributions: Conceptualization: A.J.Y.S., T.L., N.F., S.A., L.D., D.H., R.M.; Methodology: A.J.Y.S., R.-T.F., T.L., R.M.; Software: A.J.Y.S., R.-T.F.; Validation: A.J.Y.S., R.-T.F., T.L., N.F., L.K.K.S., S.A., R.M.; Investigation: A.J.Y.S., R.-T.F., T.L., N.F., L.K.K.S., S.A., R.M.; Resources: A.S., V.H., D.H., R.M.; Writing-Original Draft: A.J.Y.S., R.-T.F.; Writing-Review & Editing: A.J.Y.S., R.-T.F., T.L., N.F., L.K.K.S., S.A., L.D., A.S., V.H., D.H., R.M.; Supervision: A.S., V.H., D.H., R.M.; Project administration: A.S., V.H., D.H., R.M.; Funding Acquisition: A.S., V.H., D.H., R.M.

Declaration of Generative AI and AI-Assisted Technologies in the Writing Process

During the preparation of this manuscript, the authors used ChatGPT and Grammarly to improve the language and style. After using these, the authors reviewed and edited the content as needed and take full responsibility for the content of the manuscript.

Declaration of Interests

The authors declare no competing interests.

Framework Implementation Validation and Supervised Trainings

Table 6. Accuracies of the supervised end-to-end baselines (mean ±\pm std) on the BloodMNIST dataset. θIN\theta_{\mathrm{IN}} and θRnd\theta_{\mathrm{Rnd}} denote initialization from ImageNet-supervised weights and random initialization, respectively. We report accuracies here for consistency with the MedMNIST convention [1]. The best-performing supervised baseline for each dataset is bolded.

Table 7. Hyperparameters used for supervised trainings. The batch size was accordingly adjusted to eight when the training set had less then 128 samples. "–" indicates unavailability.

Self-Supervised Learning

Methods

Our analysis was focused on the following discriminative SSL frameworks: SimCLR [2], MoCo-v2 [3], MoCo-v3 [4], BYOL [5], Barlow Twins [6], and DINO [7]. Each method was selected to evaluate different representational objectives, ranging from contrastive learning to redundancy reduction. The specific operations of these methods are summarized below. Our implementation adhered to the respective original technical specifications, preserving the primary architectural components, such as projection/prediction heads, and specialized objective functions. By utilizing a centralized configuration system for hyperparameters (e.g., temperature scaling, weight decay, and momentum coefficients), we ensured a systematic and fair comparison across different backbones and datasets. Our analysis focused on the following approaches:

  • SimCLR (Simple Framework for Contrastive Learning) [2] is implemented as a two-view contrastive framework. It learns representations by maximizing the agreement between differently augmented views of the same image in a latent space. We utilize the normalized temperature-scaled cross-entropy loss to increase the similarity between positive pairs while minimizing agreement with other samples in the batch.

  • MoCo-v2 (Momentum Contrast version 2) [3] utilizes a momentum-based design featuring a momentum encoder and a queue-based memory bank. This mechanism decouples the number of negative samples from the training batch size. For backbones utilizing Batch Normalization, batch shuffling is applied to maintain consistent normalization statistics between the online and momentum branches. The momentum branch is updated via an Exponential Moving Average (EMA) of the online network, with a stop-gradient operation to ensure stable feature evolution.

  • MoCo-v3 [4] builds upon the momentum-based formulation but removes the queue mechanism in favor of a purely asymmetric architecture and batch-synchronized design. In this implementation, the online branch includes an additional MLP prediction head, while the momentum branch acts as a slowly updated teacher network. Like the previous iteration, it processes two augmented views per sample and employs an EMA-based weight update strategy to prevent representation collapse.

  • DINO (Self-distillation with no labels) [7] employs a multi-crop self-distillation strategy. Unlike the two-view approaches above, it generates two global views and several local crops for each sample. The student network is trained to predict the outputs of a momentum-updated teacher network, which processes only the global views. A stop-gradient is applied to the teacher, which is instead updated via an EMA of the student weights, allowing the model to distill knowledge and extract consistent features across different crops without the need for negative samples.

  • Barlow Twins [6] implements a redundancy-reduction objective function that aligns the representations of two augmented views while explicitly penalizing cross-correlations between embedding dimensions. By pushing the cross-correlation matrix toward the identity matrix, the method learns features that are both invariant to distortions and maximally informative.

  • BYOL (Bootstrap Your Own Latent) [5] relies on an asymmetric teacher-student architecture to learn representations without the need for negative pairs. The framework minimizes the mean squared error between the online networks prediction and the target networks representation of the same image. The online network is equipped with a predictor head, while the target network is updated via an EMA of the online parameters, ensuring that the target provides a consistently evolving representation for the student to match.

Unless stated otherwise, training hyperparameters were held constant across datasets, backbones, annotated data fractions and seeds, so that differences between methods are not confounded by differences in training budget. The complete data augmentation pipeline with its corresponding parameter ranges, and the global training hyperparameters, are given in Table 8 and Table 9, respectively. The learning rate was warmed up linearly over the first 5% of optimization steps and then decayed to zero following a cosine schedule, with the total number of steps computed as the number of epochs times the number of mini-batches per epoch. The optimizer was the one deliberate exception, chosen to match the normalization scheme of the backbone and the reference implementation of each objective. For the BN backbone ResNet50, LARS was used for SimCLR, BYOL and BT, and SGD with momentum for MoCo-v2, MoCo-v3, and DINO, following the recipes published with those methods. For the LN architectures (ConvNeXts, and ViT), AdamW was used throughout, as in the original training recipes for these backbones and in the transformer variants of MoCo-v3 and DINO. All optimiser settings are listed in Table 9.

Some method specific settings necessarily depart from those reported in the original publications. For consistency across the benchmark and to keep the total compute tractable, we harmonized the depth and width of the projection heads, the prototype dimension of DINO, the batch size and the number of pretraining epochs rather than reproducing each method’s published configuration. These values were chosen to be identical across methods wherever the objective permits, so that observed differences reflect the objectives themselves rather than unequal training budgets; the complete set of values is stated in Table 9.

For completeness, the configuration files used to produce every reported run, together with all remaining implementation details, are available in the accompanying code repository (see data and code availability).

Supplementary evaluation results. To provide a comprehensive overview of the interaction between architectural choice, SSL algorithms, and initialization strategies, we present the full performance results across all experimental configurations.

References

  1. [1]Basiji, D. A., Ortyn, W. E., Liang, L., Venkatachalam, V., and Morrissey, P. (2007). Cellular image analysis and imaging by flow cytometry. Clinics in Laboratory Medicine, 27(3):653–670.pubmed.ncbi.nlm.nih.gov/17658411
  2. [2]Rees, P., Summers, H. D., Filby, A., Carpenter, A. E., and Doan, M. (2022). Imaging flow cytometry: a primer. Nature Reviews Methods Primers, 2(1):86.DOI
  3. [3]Ugawa, M. and Ota, S. (2024). Recent Technologies on 2D and 3D Imaging Flow Cytometry. Cells, 13(24).DOI
  4. [4]Dunker, S., Motivans, E., Rakosy, D., Boho, D., Mäder, P., Hornick, T., and Knight, T. M. (2021). Pollen analysis using multispectral imaging flow cytometry and deep learning. New Phytologist, 229(1):593–606.DOI
  5. [5]Dunker, S., Boyd, M., Durka, W., Erler, S., Harpole, W. S., Henning, S., Herzschuh, U., Hornick, T., Knight, T., Lips, S., et al. (2022). The potential of multispectral imaging flow cytometry for environmental monitoring. Cytometry Part A, 101(9):782–799.DOI
  6. [6]Vonficht, D., Jopp-Saile, L., Yousefian, S., Flore, V., Simó Vesperinas, I., Teuber, R., Avanesyan, B., Luo, Y., Röthemeier, C., Grünschläger, F., et al. (2025). Ultra-high-scale cytometry-based cellular interaction mapping. Nature Methods, 22(9):1887–1899.DOI
  7. [7]Hu, Z., Bhattacharya, S., and Butte, A. J. (2022). Application of Machine Learning for Cytometry Data. Frontiers in Immunology, Volume 12 - 2021.DOI
  8. [8]Hua, X., Han, K., Mandrachia, B., Radmand, A., Liu, W., Kim, H., Yuan, Z., Ehrlich, S. M., Li, K., Zheng, C., et al. (2024). Light-field flow cytometry for high-resolution, volumetric and multiparametric 3D single-cell analysis. Nature Communications, 15(1):1975.DOI
  9. [9]McKinnon, K. M. (2018). Flow cytometry: an overview. Current Protocols in Immunology, 120(1):5–1.DOI
  10. [10]Carpenter, A. E., Jones, T. R., Lamprecht, M. R., Clarke, C., Kang, I. H., Friman, O., Guertin, D. A., Chang, J. H., Lindquist, R. A., Moffat, J., et al. (2006). Cellprofiler: image analysis software for identifying and quantifying cell phenotypes. Genome Biology, 7(10):R100.
  11. [11]Stirling, D. R., Carpenter, A. E., and Cimini, B. A. (2021). CellProfiler Analyst 3.0: accessible data exploration and machine learning for image analysis. Bioinformatics, 37(21):3992–3994.DOI
  12. [12]Julian, T., Tang, T., Hosokawa, Y., and Yalikun, Y. (2023). Machine learning implementation strategy in imaging and impedance flow cytometry. Biomicrofluidics, 17(5).DOI
  13. [13]Nassar, M., Doan, M., Filby, A., Wolkenhauer, O., Fogg, D. K., Piasecka, J., Thornton, C. A., Carpenter, A. E., Summers, H. D., Rees, P., and Henning, H. (2019). Real-time Identification of White Blood Cells Using Machine Learning. Cytometry Part A, 95(8):836–842.
  14. [14]Krajisic, P., Hornick, T., and Dunker, S. Comparison of Deep Learning Architectures for three different Multispectral Imaging Flow Cytometry Datasets. In Ganzha, M., Maciaszek, L. A., Paprzycki, M., and Slezak, D., editors, Position Papers of the 18th Conference on Computer Science and Intelligence Systems, FedCSIS 2023, Annals of Computer Science and Information Systems, pages 59–66, (2023).
  15. [15]Kranich, J., Chlis, N.-K., Rausch, L., Latha, A., Schifferer, M., Kurz, T., Kia, A. F.-A., Simons, M., Theis, F. J., and Brocker, T. (2020). In vivo identification of apoptotic and extracellular vesicle-bound live cells using image-based deep learning. Journal of Extracellular Vesicles, 9(1):1792683.DOI
  16. [16]Delikouyn, K., Chen, Q., Wei, L., Myao, S. K., Krell, J., Schlegel, M., Kuan, W. S., Soong, J. T. Y., Schneider, G., da Costa, C. P., Knolle, P. A., Renia, L., Cove, M. E., Lee, H. K., Diepold, K., and Hayden, O. Real-time deep learning phase imaging flow cytometer reveals broad cell aggregate biomarkers for haematology diagnostics. (2025).
  17. [17]Boushehri, S. S., Korni­vetc, A., Winter, D. J., Kazeminasab, S., Essig, K., Schmich, F., and Marr, C. (2024). PXPermute reveals staining importance in multichannel imaging flow cytometry. Cell Reports Methods, 4(2).DOI
  18. [18]Doan, M., Barnes, C., McQuin, C., Caicedo, J. C., Goodman, A. A., Carpenter, A. E., and Rees, P. (2021). Deepometry, a framework for applying supervised and weakly supervised deep learning to imaging cytometry. Nature Protocols, 16(7):3572–3595.DOI
  19. [19]Li, Y., Mahjoubar, A., Chen, C. L., Niazi, K. R., Pei, L., and Jalali, B. (2019). Deep cytometry: deep learning with real-time inference in cell sorting and flow cytometry. Scientific Reports, 9(1):11088.DOI
  20. [20]Kaliman, S., Alajangi, R., Sbaa, N., Müller, P., Ströhlein, N., Harmon, J., Krä̈ter, M., Guck, J., and Abuhattum, S. (2025). Small U-Net for Fast and Reliable Segmentation in Imaging Flow Cytometry. Cytometry Part A, 107(7):450–463.DOI
  21. [21]Hong, T., Peng, M., Kim, Y., Schellhorn, H. E., and Fang, Q. (2025). Automated cell profiling in imaging flow cytometry with annotation-efficient learning. Optics & Laser Technology, 181:111992.
  22. [22]Wang, J., Zhu, H., Wang, S.-H., and Zhang, Y.-D. (2021). A Review of Deep Learning on Medical Image Analysis. Mobile Networks and Applications, 26(1):351–380.DOI
  23. [23]Zeng, X., Abdullah, N., and Sumari, P. (2024). Self-supervised learning framework application for medical image analysis: a review and summary. BioMedical Engineering OnLine, 23(1):107.DOI
  24. [24]Rani, V., Kumar, M., Gupta, A., Sachdeva, M., Mittal, A., and Kumar, K. (2024). Self-supervised learning for medical image analysis: a comprehensive review. Evolving Systems, 15(4):1607–1633.DOI
  25. [25]Min, B., Ross, H., Sulem, E., Veyseh, A. P. B., Nguyen, T. H., Saiz, O., Agirre, E., Heintz, I., and Roth, D. (2023). Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey. ACM Computing Surveys, 56(2):1–40.arxiv.org/abs/2111.01243
  26. [26]Schiappa, M. C., Rawat, Y. S., and Shah, M. (2023). Self-Supervised Learning for Videos: A Survey. ACM Computing Surveys, 55(13s):1–37.DOI
  27. [27]Ovcharenko, O., Barkmann, F., Toma, P., Daunhauer, I., Vogt, J. E., Schelter, S., and Boeva, V. scSSL-Bench: Benchmarking Self-Supervised Learning for Single-Cell Data. In Singh, A., Fazel, M., Hsu, D., Lacoste-Julien, S., Berkenkamp, F., Maharaj, T., Wagstaff, K., and Zhu, J., editors, Forty-second International Conference on Machine Learning, ICML 2025, Proceedings of Machine Learning Research. PMLR / OpenReview.net, (2025).arxiv.org/abs/2506.10031
  28. [28]Richter, T., Bahrami, M., Xia, Y., Fischer, D. S., and Theis, F. J. (2025). Delineating the effective use of self-supervised learning in single-cell genomics. Nature Machine Intelligence, 7(1):68–78.DOI
  29. [29]Xiao, J., Zhu, G., and Loianno, G. VG-SSL: Benchmarking Self-Supervised Representation Learning Approaches for Visual Geo-Localization. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 6667–6677. IEEE, (2025).
  30. [30]Liu, B., Polack, M., Coudray, N., Claudio Quiros, A., Sakellaropoulos, T., Le, H., Karimkhan, A., Crobach, A. S., van Krieken, J. H. J., Yuan, K., et al. (2025). Self-supervised learning reveals clinically relevant histomorphological patterns for therapeutic strategies in colon cancer. Nature Communications, 16(1):2328.DOI
  31. [31]Yang, J., Shi, R., Wei, D., Liu, Z., Zhao, L., Ke, B., Pfister, H., and Ni, B. (2023). MedMNIST v2 - A large-scale lightweight benchmark for 2D and 3D biomedical image classification. Scientific Data, 10(1):41.
  32. [32]Bundele, V., Sarta, K., Kargi, B., Çal, O. A., Tezören, K., Ghaderi, Z., and Lensch, H. (2025). Evaluating Self-Supervised Learning in Medical Imaging: A Benchmark for Robustness, Generalizability, and Multi-Domain Impact. arXiv preprint arXiv:2412.19124.arxiv.org/abs/2412.19124
  33. [33]Boukdir, A. (2026). An Empirical Study of Self-Supervised Contrastive Pre-training Across Multiple Backbone Architectures for Medical Image Classification. Arabian Journal for Science and Engineering, pages 1–20.DOI
  34. [34]Harsy, C. U., Wald, T., Gotkowski, K., Kirchhoff, Y., Knopp, M., Rokuss, M., Stegmeier, E., Schader, P., Trofimova, D., Stock, R., et al. (2026). nfnoundation: 3d foundation models for radiology. arXiv preprint arXiv:2609.26924.arxiv.org/abs/2609.26924
  35. [35]Kraus, O., Kenyon-Dean, K., Saberïan, S., Fallah, M., McLean, P., Leung, J., Sharma, V., Khan, A., Balakrishnan, J., Celik, S., et al. Masked autoencoders for microscopy are scalable learners of cellular biology. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11757–11768. IEEE, (2024).DOI
  36. [36]Kim, V., Adaloglou, N., Osterland, M., Morelli, F. M., Halawa, M., König, T., Gnu tt, D., and Marin Zapata, P. A. (2025). Self-supervision advances morphological profiling by unlocking powerful image representations. Scientific Reports, 15(1):4876.DOI
  37. [37]Moutakanni, T., Couprie, C., Yi, S., Doron, M., Chen, Z. S., Moshkov, N., Gardes, E., Caron, M., Touvron, H., Joulin, A., et al. (2025). Cell-dino: Self-supervised image-based embeddings for cell fluorescent microscopy. PLOS Computational Biology, 21(12):e1013828.DOI
  38. [38]Zedda, L., Loddo, A., Di Ruberto, C., and Marr, C. RedDino: A foundation model for red blood cell analysis . In proceedings of Medical Image Computing and Computer Assisted Intervention – MICCAI 2025, volume LNCS 15963. Springer Nature Switzerland, (2025).DOI
  39. [39]Chen, X., Fan, H., Girshick, R. B., and He, K. (2020). Improved Baselines with Momentum Contrastive Learning. arXiv preprint arXiv:2003.04297.arxiv.org/abs/2003.04297
  40. [40]Zhang, Y., Huang, X., Wang, Z., Du, X., Fang, Z., Chen, S., Chen, Y., Huo, X., Wang, K., Chen, X., and Chen, J. (2026). Generalizable Self-supervised Learning for Imaging Flow Cytometry on Multi-dataset Leukocyte Differential. Microsyst. Nanoeng., 12(1).DOI
  41. [41]Zbontar, J., Jing, L., Misra, I., LeCun, Y., and Deny, S. Barlow Twins: Self-Supervised Learning via Redundancy Reduction. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, Proceedings of Machine Learning Research, pages 12310–12320. PMLR, (2021).arxiv.org/abs/2103.03230
  42. [42]Shetab Boushehri, S., Essig, K., Ch., N.-K., Herter, S., Bacac, M., Theis, F. J., Glasmacher, E., Marr, C., and Schmich, F. (2023). Explainable machine learning for profiling the immunological synapse and functional characterization of therapeutic antibodies. Nature Communications, 14(1):7888.
  43. [43]Doan, M., Sebastian, J. A., Caicedo, J. C., Siegert, S., Roch, A., Turner, T. R., Mykhailova, O., Pinto, R. N., McQuin, C., Goodman, A., Parsons, M. J., Wolkenhauer, O., Hennig, H., Singh, S., Wilson, A., Acker, J. P., Rees, P., Kolios, M. C., and Carpenter, A. E. (2020). Objective assessment of stored blood quality by deep learning. Proceedings of the National Academy of Sciences, 117(35):21381–21390.
  44. [44]Essig, K., Shetab Boushehri, S., Marr, C., Schmich, F., and Glasmacher, E. An imaging flow cytometry dataset for profiling the immunological synapse of therapeutic antibodies, (2022).DOI
  45. [45]Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. E. A Simple Framework for Contrastive Learning of Visual Representations. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, Proceedings of Machine Learning Research, pages 1597–1607. PMLR, (2020).arxiv.org/abs/2002.05709
  46. [46]Chen, X., Xie, S., and He, K. An Empirical Study of Training Self-Supervised Vision Transformers. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, pages 9620–9629. IEEE, (2021).arxiv.org/abs/2104.02057
  47. [47]Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., and Joulin, A. Emerging Properties in Self-Supervised Vision Transformers. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, pages 9630–9640. IEEE, (2021).arxiv.org/abs/2104.14294
  48. [48]Grill, J. J., Strub, F., Altché, F., Tallec, C., Richemond, P. H., Buchatskaya, E., Doersch, C., Pires, B. Á., Guo, Z., Azar, M. G., Piot, B., Kavukcuoglu, K., Munos, R., and Valko, M. Bootstrap Your Own Latent - A New Approach to Self-Supervised Learning. In Advances in Neural Information Processing Systems 33, NeurIPS 2020, (2020).arxiv.org/abs/2006.07733
  49. [49]He, K., Zhang, X., Ren, S., and Sun, J. Deep Residual Learning for Image Recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, pages 770–778. IEEE Computer Society, (2016).arxiv.org/abs/1512.03385
  50. [50]Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In 9th International Conference on Learning Representations, ICLR 2021. OpenReview.net, (2021).
  51. [51]Liu, Z., Mao, H., Wu, C., Feichtenhofer, C., Darrell, T., and Xie, S. A ConvNet for the 2020s. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, pages 11966–11976. IEEE, (2022).
  52. [52]Woo, S., Debnath, S., Hu, R., Chen, X., Liu, Z., Kweon, I. S., and Xie, S. ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, pages 16133–16142. IEEE, (2023).arxiv.org/abs/2301.00808
  53. [53]Balestriero, R., Ibrahim, M., Sobal, V., Morcos, A., Shekhar, S., Goldstein, T., Bordes, F., Bardes, A., Mialon, G., Tian, Y., et al. (2023). A cookbook of self-supervised learning. arXiv preprint arXiv:2304.12210.arxiv.org/abs/2304.12210
  54. [54]Parks, D. R., Roederer, M., and Moore, W. A. (2006). A new "Logiclie" display method avoids deceptive effects of logarithmic scaling for low signals and compensated data. Cytometry Part A, 69(6):541–551.pubmed.ncbi.nlm.nih.gov/16604519
  55. [55]Moore, W. A. and Parks, D. R. (2012). Update for the logiclie data scale including operational code implementations. Cytometry Part A, 81(4):273–277.DOI
  56. [56]Spidlen, J., Moore, W., Force, I. D. S. T., and Brinkman, R. R. (2015). ISAC’s Gating-ML 2.0 data exchange standard for gating description. Cytometry Part A, 87(7):683–687.DOI
  57. [57]Ionita, M., McKeague, M. L., Painter, M. M., Mathew, D., Pattekar, A., Rezk, A., Shwetank, Maseda, D., Wherry, E. J., and Greenplate, A. R. (2025). Creating Robust Doublet Detection in Cytometry Data Based on Protein Expression Patterns. Cytometry Part A, 107(11):716–729.DOI
  58. [58]Cano, R. L. E. and Lopera, H. D. E. Introduction to T and B lymphocytes. In Autoimmunity: from bench to bedside [Internet]. El Rosario University Press, (2013).
  59. [59]Tian, Y., Chen, X., and Ganguli, S. Understanding self-supervised learning dynamics without contrastive pairs. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 10268–10278. PMLR, (2021).arxiv.org/abs/2102.06810
  60. [60]Wang, D., Wang, X., Wang, L., Li, M., Da, Q., Liu, X., Gao, X., Shen, J., He, J., Shen, T., et al. (2023). A Real-world Dataset and Benchmark For Foundation Model Adaptation in Medical Image Classification. Scientific Data, 10(1):574.
  61. [61]Baharoon, M., Qureshi, W., Ouyang, J., Xu, Y., Aljouie, A., and Peng, W. (2023). Evaluating General Purpose Vision Foundation Models for Medical Image Analysis: An Experimental Study of DINOv2 on Radiology Benchmarks. arXiv:2312.02366.arxiv.org/abs/2312.02366
  62. [62]Bagwell, C. B. (2005). Hyperlog-a flexible log-like transform for negative, zero, and positive valued data. Cytometry Part A, 64(1):34–42.pubmed.ncbi.nlm.nih.gov/15700280
  63. [63]Doan, M. URL8_Unannotated_Data. figshare, (2020). Version 1.
  64. [64]Macaulay, S. O., Maset, E., Boscutti, F., Cingano, P., Trevisan, F., Trotta, G., Vuerich, M., and Fusiello, A. (2026). Transferring RGB-Pretrained CNNs to Multispectral UAV Imagery for Salt Marsh Vegetation Classification. Remote Sensing, 18(4).DOI
  65. [65]Cong, Y., Khanna, S., Meng, C., Liu, P., Rozi, E., He, Y., Burke, M., Lobell, D. B., and Ermon, S. SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery. In Advances in Neural Information Processing Systems 35, NeurIPS 2022, (2022).arxiv.org/abs/2207.08051
  66. [66]de La Comble, A. and Prepin, K. Efficient transfer learning for multi-channel convolutional neural networks. In 2021 17th International Conference on Machine Vision and Applications (MVA), pages 1–6, (2021).
  67. [67]Huang, S.-C., Pareek, A., Jensen, M., Lungren, M. P., Yeung, S., and Chaudhari, A. S. (2023). Self-supervised learning for medical image classification: a systematic review and implementation guidelines. NPJ Digital Medicine, 6(1):74.DOI
  68. [68]Marks, M., Knott, M., Kondapaneli, N., Cole, E., Defraeye, T., Perez-Cruz, F., and Perona, P. (2025). A Closer Look at Benchmarking Self-supervised Pre-training with Image Classification. International Journal of Computer Vision, 133(8):5013–5025.
  69. [69]Healy, J. and McInnes, L. (2024). Uniform manifold approximation and projection. Nature Reviews Methods Primers, 4(1):82.DOI
  70. [70]Acevedo, A., Alférez, S., Merino, A., Puigví, L., and Rodellar, J. (2019). Recognition of peripheral blood cell images using convolutional neural networks. Computer Methods and Programs in Biomedicine, 180:105020.DOI
  71. [71]Acevedo, A., Merino, A., Alférez, S., Ángel Molina, Boldú, L., and Rodellar, J. (2020). A dataset of microscopic peripheral blood cell images for development of automatic recognition systems. Data in Brief, 30:105474.

Paper details

vlnc:W_tv4kvwqj
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 3

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

ChatGPT, Grammarly

View statement
Looking for related papers…
Contents