WorksComputer Science
DBHiCo: Dual-Branch Hierarchical Contrastive Learning for MRI Image Classification
Abstract
Automated Magnetic Resonance Imaging (MRI) image classification provides a crucial non-invasive approach for neurological diagnosis, yet it is fundamentally challenged by inherent acquisition noise and high inter-subject structural similarity, which obscure subtle pathological variations, thereby limiting the performance of deep neural networks. While contrastive learning (CL) offers a promising mitigation strategy, existing paradigms rely on rigid binary partitions that ignore negative sample heterogeneity, and they fail to capture the cross-view structural consistency vital for fine-grained anatomical alignment. To address these barriers, we propose Dual-Branch Hierarchical Contrastive Learning (DBHiCo), a novel framework explicitly designed to collaboratively enforce structural consistency and semantic discrimination. Specifically, DBHiCo leverages a natural anatomical multi-view pair (original and skull-stripped MRI scans), processing them through a dual-branch architecture to seamlessly integrate holistic structural robustness with localized intracranial details. Crucially, we introduce a hierarchical contrastive loss that discards conventional binary constraints in favor of a margin-ranked similarity hierarchy. This mechanism explicitly orders positive and negative pairs based on their structural and semantic difficulty, thereby amplifying the fine-grained semantic discriminability of the learned representations. Extensive experiments across three diverse neurological datasets demonstrate that DBHiCo achieves state-of-the-art performance, consistently outperforming existing MRI multi-view representation learning and advanced contrastive fine-tuning methods.
Introduction
Magnetic Resonance Imaging (MRI) serves as a cornerstone non-invasive tool for capturing internal structural and functional information, making it indispensable for identifying neurological abnormalities [1, 2, 3]. While computer-aided diagnosis (CAD) systems have profoundly alleviated the manual interpretation burden of MRI places on radiologists [4, 5, 6], empowering the efficient diagnosis of neurological disorders such as pediatric posterior fossa tumors [7, 8, 9], Alzheimer’s disease [10, 11, 12, 13], and Attention-Deficit Hyperactivity Disorder [14, 15, 16, 17], the core deep neural networks (DNNs) [18, 19, 20, 21] driving these systems face unique challenges: Unlike natural images, MRI acquisitions inevitably introduce system noise [12] that causes random intensity fluctuations—which DNNs may misinterpret as false diagnostic features—and exhibit high inter-subject similarity [22] that obscures subtle pathological variations. Consequently, direct transfer of large-scale pre-trained knowledge through standard supervised learning frequently induces overfitting. The network tends to memorize random intensity fluctuations rather than extracting robust, disease-relevant anatomical patterns [23, 24, 25, 26, 27].
To mitigate this bottleneck, recent approaches have increasingly adopted multi-view representation learning paradigms [28, 29, 30, 31]. By applying contrastive learning (CL) across multiple distinct views derived from a single MRI structure (e.g., skull-stripped volume)—such as through spatial masking or multi-level cropping [32, 33]—these methods effectively compel the network to extract robust features via pre-training. Despite their effectiveness, these methods overlook multi-view construction from an anatomical structural perspective. In standard neuroimaging pipelines, an exceptionally natural structural multi-view pair is inherently facilitated by the skull-stripping pre-processing step [34, 35, 36]. Notably, original MRI scans preserve holistic anatomical boundaries and soft-tissue continuity, whereas skull-stripped scans isolate intracranial parenchymal details [37, 38, 39]. Therefore, these two views provide highly complementary information, i.e., the anatomical structural integrity and the salience of tissue-level details, which are critical for accurate diagnosis.
However, existing paradigms fail to fully unlock the potential of such complementary cross-view data for downstream clinical tasks. Specifically, current multi-view pre-training methods heavily rely on the curation of massive MRI cohorts, and their self-supervised proxy objectives are often disconnected from specific diagnostic goals, leading to sub-optimal transferability.
To overcome this task disconnect, an intuitive alternative is to directly optimize representations on the downstream task via standard semantic supervision [32, 41, 42]. Nevertheless, these approaches focus predominantly on global semantic discrimination (Figure 1-a), thereby neglecting the cross-view structural consistency required to align fine-grained anatomical features. Furthermore, they typically employ a rigid binary partition—pulling same-class views together while pushing all other instances apart uniformly [40]. This simplistic strategy ignores the inherent heterogeneity among negative samples (e.g., varying degrees of intra-class vs. inter-class structural variations), treating them equally and thereby yielding a latent space that lacks the granularity necessary for precise semantic discrimination in complex brain structures (Figure 1-c).

Figure 1. Motivation of DBHiCo. (a)-(b) Comparison between the semantic supervised paradigms and ours. Intra-views supervision provides insufficient learning of structural relationships, whereas our method explicitly enforces cross-view and semantic contrastive learning. (c) Similarity analysis across views and classes. MRI images exhibit high similarity issue within the same view, while previous method (e.g., SuperCon [40]) disproportionately emphasize semantic discrimination but ignoring structural consistency learning.
To address these barriers, we propose the Dual-Branch Hierarchical Contrastive Learning (DBHiCo) method. Specifically, DBHiCo is explicitly designed to collaboratively enforce structural consistency and semantic discrimination during training (Figure 1-b). First, we introduce a dual-branch architecture that extracts and fuses features from paired original and skull-stripped MRI scans in parallel, integrating holistic structural robustness with localized semantic richness. Second, to rectify the limitations of binary contrastive targets, we propose a novel hierarchical contrastive loss (). In contrast to a uniform penalty, establishes a strict similarity ranking constraint across positive and varying-difficulty negative pairs. By explicitly modeling these hierarchical disparities via margin parameters driven by both structural (cross-view) and semantic (cross-class) conditions, the network is forced to capture subtle anatomical nuances.
Finally, by jointly optimizing this hierarchical contrastive objective with the downstream classification task, DBHiCo directly tailors the learned representations to clinical requirements.
We experimentally validate DBHiCo on three representative neurological disorders benchmark: our collected Tiantan dataset, public OASIS-1 [10] dataset, and ADHD-200 [14] competition dataset. DBHiCo achieves superior performance on various baseline model architecture, outperforming existing cross-view representation learning and supervised contrastive learning studies.
Our principal contributions can be summarized as follows.
We present DBHiCo, a novel multi-view contrastive learning framework tailored for brain MRI classification. By coupling dual-branch feature fusion with cross-view similarity constraints, DBHiCo effectively overcomes the inter-subject similarity bottlenecks inherent in neuroimaging analysis.
We propose a hierarchical contrastive loss function () that discards the prior binary contrast in favor of a margin-ranked similarity hierarchy. This explicitly orders positive and negative pairs based on structural and semantic difficulty, amplifying fine-grained semantic discrimination.
Extensive experiments demonstrate that DBHiCo achieves state-of-the-art performance, consistently outperforming existing specialized MRI multi-view representation learning and advanced contrastive fine-tuning methods.
Related works
Automated MRI image classification provides medical professionals with a non-invasive tool for diagnosing neurological disorders. Recently, CL has driven significant advancements in this domain through diverse representation optimization objectives. In this section, we first introduce existing multi-view representation learning methods tailored for MRI. The limitations inherent in these self-supervised pre-training paradigms motivate our exploration into applying CL directly to downstream clinical tasks. Furthermore, recognizing that conventional CL approaches often struggle to capture the intricate anatomical structures of the brain, we subsequently discuss the literature that inspires our proposed hierarchical contrastive design.
MRI Multi-View Representation Learning.
Numerous studies are built upon the insight that integrating multi-view MRI data enables a more comprehensive characterization of brain structures and pathological features [43]. Broadly, existing multi-view strategies can be categorized into two main paradigms. The first paradigm derives multiple views from distinct spatial regions of a single MRI volume. These methods typically employ techniques such as 3D volume masking (e.g., general [44], localized [28], and global-local [45]) or spatial cropping (e.g., random [29] and multi-level [30]) to formulate self-supervised pretext tasks. The second paradigm leverages different MRI modalities as distinct views, driving model pre-training through cross-modal reconstruction [46, 47, 48], multi-modal contrastive learning [49, 50, 31], or puzzle-solving objectives [33].
Despite their empirical success, these multi-view pre-training paradigms suffer from notable limitations: they heavily rely on the curation of exceptionally large-scale MRI pre-training cohorts, and their self-supervised proxy objectives are often disconnected from the specific goals of downstream clinical tasks, leading to sub-optimal transferability. In contrast, our DBHiCo effectively mitigates these bottlenecks. By introducing a novel cross-view strategy (Original vs. Skull-stripped) tailored to the intrinsic anatomical characteristics of MRI data, DBHiCo directly optimizes representations for downstream tasks. As validated in our experiments (see § 4.4.1), this targeted approach achieves superior performance in capturing both structural and semantic nuances without the burden of massive pre-training.
Contrastive Learning for Classification Tasks
CL aims to construct a feature space that encodes semantic similarity by pulling positive sample pairs closer while pushing negative pairs apart. As a representative self-supervised CL framework, SimCLR [32] leverages extensive data augmentations to introduce sufficient variations, enabling the network to learn robust and invariant representations. Building upon this foundation, SuperCon [40] explicitly incorporates supervisory signals into the contrastive objective, yielding highly discriminative features when class labels are available. This paradigm shift toward supervised CL has significantly enhanced representation learning on labeled data, empowering researchers to tailor custom loss designs for specific data characteristics, particularly in data-scarce and class-imbalanced scenarios [41, 42, 51, 52].
However, the predominant focus of these standard methods on global semantic discrimination limits their efficacy when applied to structure-rich MRI scans. Medical imaging requires a nuanced understanding of fine-grained anatomical details alongside global category semantics. To bridge this gap, our proposed DBHiCo introduces a novel hierarchical CL framework designed to achieve the integrated learning of structural consistency and semantic discrimination.
Methodology
This section provides an overview of our proposed DBHiCo method. Firstly, the overall framework of the DBHiCo method is introduced in § 3.1. Then, the process of constructing positive and negative pairs is presented in § 3.2. Finally, the hierarchical contrastive loss of the DBHiCo method is introduced in § 3.3.
Overall Framework
The overall framework of the proposed DBHiCo method is presented in Figure 2-a. First, we establish a dual-branch encoder architecture to parallelly extract cross-view representations from the input brain MRI samples. Specifically, let denote a labeled MRI dataset which includes a set of class labels , with samples per class. We perform basic pre-processing steps (see § 4.1) to obtain the view of original samples and corresponding skull-stripped samples . Each MRI image is represented as a 3D volume , formed by stacking 2D slices of frames. A typical encoder (e.g., CNN [19] or Transformer [18]) is used to transform each single view image frame-by-frame into a high-dimension feature map . Second, on the one side, the cross-view representations are concatenated as along the feature dimension , after which a classifier generates frame-level predictions . On the other side, we perform frame-averaging to obtain the contrast embedding , followed by the utilization of non-linear projectors (e.g., MLP) to map it into a lower-dimensional embedding . Finally, the objective function is formulated as:

Figure 2. An overview of DBHiCo. (a) The pipeline of proposed DBHiCo. A dual-branch pipeline processes original and skull-stripped MRIs with independent encoders, extracting features for classification () and contrastive learning (). (b) Hierarchical Contrastive Loss framework, which defines positive intra-subject pairs and multi-level negative pairs, enforcing structured similarity constraints to learn discriminative features.
where is the cross-entropy loss, and is our dedicated hierarchical contrast loss with weight . Further details of are presented in § 3.2 and § 3.3.
Constructing Positive and Negative Pairs
The construction strategy of positive and negative pairs is crucial for effectively leveraging CL to extract discriminative and robust representations from brain MRI samples. The most common strategy [32] is to treat different augmented views of the same image as a positive pair, while augmented views from different samples are negative pairs. To establish complementary cross-view relationships in brain MRI, we treat only the original and skull-stripped augmented views of the same image as positive pairs. Specifically, given the sample of the class in , the unique positive pair is defined as . The corresponding negative pairs are constructed as:
where ensures that negatives are drawn from different samples or classes. We assign difficulty levels to negative pairs according to three prior conditions:
Cross-view consistency priority. Within each sample, the Org and Stp counterparts should remain consistent, ensuring that intra-sample cross-view similarity is preserved.
Intra-view robustness. For samples of the same class, the intra-view pairs should exhibit higher similarity than cross-view pairs, thereby reinforcing structural consistency learning.
Discriminative constraint. Intra-class similarity should, overall, be higher than inter-class similarity, maintaining the class separability.
This prior design enables us to rank the difficulties. Taking the Stp samples as an example (see Figure 2-b), the ordering of its associated positive and negative pairs can be illustrated as follows:
where denotes the average over the specified image sets.
Hierarchical Contrastive Loss
Preliminary: Contrastive Loss. Given two sets of projected contrast embedding and of the two augmented views in the batch , the contrastive loss maximizes the similarity of positive pairs while minimizing that of negative pairs. Mathematically, the similarity can be calculated as:
where denotes the similarity function and is the temperature scaling factor that controls the concentration of the similarity distribution. Construct a row of logits with the sample as the anchor, where the positive sample is . Then, the standard contrastive (e.g., InfoNCE [53]) loss is:
which encourages the model to distinguish the positive sample (diagonal entry) from all others in the batch.
Hierarchical Contrastive Loss. Building upon the basic contrastive loss, the proposed hierarchical contrastive loss introduces a structured similarity constraint to model fine-grained semantic hierarchies in cross-view MRI samples. Let and denote the normalized feature embeddings of original and skull-stripped MRI views within a batch of size , respectively, and let be the corresponding class label vector, where and represent the labels of the and samples. Consider the image as the anchor, the imposes a strict similarity ordering across five hierarchical levels:
Self cross-view similarity. Similarity between the cross-view of the sample itself:
Intra-class and intra-view similarity. Average similarity between the of the sample and the others in the same class:
Intra-class and cross-view similarity. Average similarity between the of the sample and of the others in the same class:
Inter-class and intra-view similarity. Average similarity between the of the sample and of the of the others in different classes:
Inter-class and cross-view similarity. Average similarity between the of the sample and of the other in different classes:
The enforces a strict ordering according to (3) of these similarities with a margin . Violations of these constraints are penalized using the function. The single-directional loss is defined as the average of these penalty terms across all samples in the batch:
where
Here, and are indicator functions that equal 1 if there exist valid intra-class and inter-class samples for the sample, respectively, and 0 otherwise. This ensures the margin loss is computed only when sufficient samples are available to form the respective pairs. Finally, the symmetric is expressed as:
Experiments
This section first introduces the datasets and evaluation metrics (§ 4.1), and then describes the implementation details of the proposed DBHiCo (§ 4.2). Finally, we report the comparative experimental results against existing methods (§ 4.3), followed by a comprehensive analysis of our approach (§ 4.4).
Datasets and Evaluation
Dataset Collection and Pre-processing. In this study, we evaluate our method on three distinct types of brain disorders: Pediatric Posterior Fossa Tumors, Alzheimer’s Disease, and Attention-Deficit Hyperactivity Disorder.
Due to the scarcity of publicly available datasets for Pediatric Posterior Fossa Tumor classification [55, 56, 57, 58], we compiled a proprietary clinical dataset, referred to as the Tiantan dataset, sourced from the Department of Neurosurgery at Beijing Tiantan Hospital. This dataset comprises 407 cases of medulloblastoma, 101 cases of choroid plexus tumors, 177 cases of ependymoma, and 144 cases of glioma. This retrospective study was approved by the Institutional Review Board and the ethics committee of Beijing Tiantan Hospital, Capital Medical University (No. KY2022-067-01). Between December 2004 and September 2021, image acquisition was performed using three MRI scanners: Siemens Magnetom Verio, Siemens Magnetom TrioTim, and GE Discovery MR750. We ensured that each subject in the Tiantan dataset possessed three structural MRI modalities: T1-weighted (T1), contrast-enhanced T1-weighted (T1-E), and T2-weighted (T2) sequences. The acquisition parameters for the T1 sequence were: repetition time (TR) = 1900 2400 ms, echo time (TE) = 8.6 19.84 ms, slice thickness = 5 mm, and matrix sizes of (Magnetom Verio), (Magnetom TrioTim), or (Discovery MR750). The parameters for T1-E were: TR = 1900 2400 ms, TE = 8.6 19.84 ms, slice thickness = 5 mm, with identical matrix sizes to T1. For the T2 sequence, parameters were: TR = 4500 10738.38 ms, TE = 8.6 113.47 ms, slice thickness = 5 mm, and matrix sizes of (Magnetom Verio), (Magnetom TrioTim), or (Discovery MR750). Slice gaps were 1 mm on the Magnetom Verio and Discovery MR750, and 1.5 mm on the Magnetom TrioTim. Gd-DTPA (dose: 0.1 mmol/kg) was administered as the contrast agent for T1-E imaging.
For Alzheimer’s Disease classification, we utilized the widely adopted public dataset OASIS-1 [10]. It contains 416 subjects aged 18 to 96 years, including 100 clinically diagnosed with Alzheimer’s Disease. Each subject’s data includes 3 or 4 T1-weighted scans acquired during a single imaging session. For Attention-Deficit Hyperactivity Disorder, we evaluated our method on the public ADHD-200 [14] global competition dataset, originally aggregated from 8 independent imaging sites. We included 691 subjects with T1-weighted MRI scans and definitive diagnostic labels from 6 of these sites, of which 261 subjects were clinically diagnosed with ADHD.
Across all three datasets, our study focuses exclusively on cross-sectional structural MRI. For the Tiantan dataset, we initially applied the N4ITK [59] bias field correction to mitigate intensity inhomogeneities and standardize intensity distributions across similar tissue types. Subsequently, a unified skull-stripping procedure was applied to all subjects across the three datasets. Since skull stripping crucially impacts the downstream accuracy of brain tumor and lesion recognition, we selected the robust HD-BET [39] algorithm over other traditional methods [60, 61, 62, 63] due to its superior generalization capabilities across heterogeneous scanners and varying MRI modalities.
Evaluation metrics. Each dataset was partitioned into training, validation, and test sets. To ensure a robust evaluation, we performed five-fold cross-validation and reported the average test-set performance across all folds for each compared method. While Accuracy (ACC) and the Area Under the Receiver Operating Characteristic Curve (AUC) are standard metrics for disease classification, medical datasets often suffer from severe class imbalance. To address the inherent biases of ACC and AUC under such distributions, we additionally incorporated the macro-averaged F1-score (F1) as a comprehensive evaluation metric.
Implementation Details
As outlined in § 3.1, DBHiCo employs a standard backbone network as the feature encoder, initialized with pre-trained weights from large-scale natural image datasets (e.g., ILSVRC-2012 [64]). Originally, each MRI scan is a singlechannel grayscale 3D volume (multi-slice image). To seamlessly adapt these inputs for the 2D pre-trained encoder, we constructed pseudo-color 3-channel inputs based on the dataset: for the Tiantan dataset, the three distinct MRI modalities (T1, CET1, T2) of each subject were stacked along the channel dimension; for the OASIS-1 dataset, three separate T1-weighted scans from the same session were randomly selected and stacked; and for the ADHD-200 dataset, the single T1-weighted scan was replicated across all three channels. By default, we uniformly sampled 8 slices along the z-axis from each subject. This strategy guarantees adequate spatial coverage of the entire brain volume while minimizing computational overhead and data redundancy.
Hyperparameter setting. For network optimization, the model was trained using the AdamW optimizer with a batch size of 8, an initial learning rate of 0.001, and a weight decay of 0.05, regulated by a cosine annealing learning rate scheduler. The contrastive projector was configured as an MLP comprising two linear layers separated by a ReLU activation function. Its intermediate hidden dimension was set to 512, and the final output embedding dimension was set to 128. For the proposed hierarchical contrastive loss (), the temperature scaling factor was fixed at 1.0, and the margin was empirically set to 0.1.
Comparison methods. We rigorously compared our DB-HiCo with several established baselines, state-of-the-art specialized MRI classification networks, and advanced contrastive learning frameworks. First, we established standard supervised baselines using ViT [18] and ResNet [19], alongside a ViT Recipe [11] specifically tailored for the medical domain. Second, we compared our approach with state-of-the-art MRI multi-view representation learning methods—including DAE [28], VoCo [29], MiM [30], and BrainMVP [31]—as these also explicitly model cross-view consistency in MRI. For these specialized methods, we preserved the backbone architectures specified in their respective publications (e.g., Swin-UNETR [65], Swin-ViT [20], and Uniformer [66]), initializing them with pre-trained weights derived from the large-scale MRI/CT datasets constructed in their original studies. Third, we evaluated advanced contrastive learning paradigms, including SimCLR [32] and SupCon [40], as well as BCL [41] and Proco [42], which are specifically engineered to handle class imbalance. Finally, we included state-of-the-art medical contrastive approaches such as ECL [51], BPaCo [52], and CoCa [7]. Although several of these contrastive methods conceptually describe a dual-branch setup, they practically implement weight-sharing encoders; hence, we categorized them as single-branch frameworks to maintain comparative clarity. For all evaluated contrastive methods, we standardized the backbone using DeiT-Small [21] or ResNet-50 [19] initialized with ILSVRC-2012 weights. Importantly, to ensure a strictly fair comparison, all competing methods were trained utilizing both the original and the skull-stripped MRI scans.
Experiments on downstream tasks
Comparison on the Tiantan dataset
We first conduct comparative experiments on the Tiantan dataset, with the quantitative results summarized in Table 1. Notably, we observe that existing multi-view representation learning methods yield sub-optimal performance. For instance, the highest total ACC among them, achieved by DAE [28], is even lower than that of the standard DeiT-Small [21] baseline (79.96% vs. 81.88%). We attribute this underperformance to the discrepancy between their self-supervised pre-training tasks and the specific objectives of the downstream classification. Furthermore, their pre-training data (typically around 10K scans) are relatively limited compared to the million-scale ILSVRC-2012 dataset. Similarly, the heavier 3D ViT Recipe [11], despite its more complex network architecture and larger parameter count, is prone to overfitting on our dataset, achieving only a 77.11% total ACC.
| Method (Publications,Years) | Pre-training dataset | Network | Input Size | Medulloblastoma ACC(%) | Medulloblastoma F1(%) | Choroid ACC(%) | Choroid F1(%) | Ependymoma ACC(%) | Ependymoma F1(%) | Glioma ACC(%) | Glioma F1(%) | Total ACC(%) | Total F1(%) |
| MRI Multi-View Representation Learning | |||||||||||||
| DAE (MIDL 2024) [28] | DAE-10K [28] | Swin-Unetr | 90.40 | 89.04 | 55.33 | 63.01 | 69.99 | 67.14 | 79.80 | 79.96 | 79.96 | 74.79 | |
| VoCo (CVPR 2024) [29] | VoCo-1.6K [29] | Swin-Unetr | 89.69 | 87.67 | 71.14 | 72.91 | 64.94 | 68.61 | 76.28 | 74.93 | 79.86 | 76.03 | |
| MiM (TMI 2025) [30] | MiM-10K [30] | Swin-ViT | 86.76 | 86.07 | 75.19 | 71.38 | 61.51 | 65.35 | 77.71 | 77.31 | 78.42 | 75.03 | |
| BrainMVP (CVPR 2025) [31] | BrainMVP-16K [31] | Uniformer-small | 84.06 | 81.16 | 60.47 | 66.03 | 60.40 | 63.14 | 71.60 | 70.48 | 73.95 | 70.20 | |
| Baseline | |||||||||||||
| ViT (ICLR 2021) [18] | ILSVRC2012 | DeiT-Small | 90.93 | 88.43 | 71.28 | 76.02 | 74.44 | 73.84 | 74.95 | 78.43 | 81.88 | 79.18 | |
| ViT Recipe (MICCAI 2024) [11] | Brain-2K [11] | DeiT-Base-3D | 89.48 | 84.99 | 67.23 | 67.99 | 55.38 | 63.34 | 75.64 | 74.23 | 77.11 | 72.64 | |
| With General Contrastive fine-tuning | |||||||||||||
| SimCLR (ICML 2020) [32] | ILSVRC2012 | DeiT-Small | 89.47 | 89.43 | 70.33 | 77.22 | 80.22 | 78.00 | 79.83 | 77.95 | 83.43 | 80.65 | |
| SuperCon (NIPS 2020) [40] | ILSVRC2012 | DeiT-Small | 90.44 | 90.28 | 74.19 | 77.38 | 73.96 | 75.91 | 81.94 | 77.75 | 83.51 | 80.33 | |
| BCL (CVPR 2022) [41] | ILSVRC2012 | DeiT-Small | 88.47 | 90.53 | 82.14 | 81.30 | 79.04 | 79.30 | 83.34 | 78.54 | 84.71 | 82.42 | |
| Proco (TPAMI 2024) [42] | ILSVRC2012 | DeiT-Small | 87.72 | 89.04 | 74.28 | 77.81 | 79.13 | 75.37 | 79.82 | 79.07 | 82.89 | 80.33 | |
| With Medical Contrastive fine-tuning | |||||||||||||
| ECL (MICCAI 2023) [51] | ILSVRC2012 | DeiT-Small | 88.71 | 87.20 | 78.09 | 78.32 | 70.05 | 73.03 | 77.71 | 78.13 | 81.55 | 79.16 | |
| BPaCo (MICCAI 2024) [52] | ILSVRC2012 | DeiT-Small | 89.48 | 85.63 | 65.38 | 70.61 | 62.21 | 65.46 | 75.67 | 76.09 | 78.32 | 74.45 | |
| CoCa (KBS 2024) [7] | ILSVRC2012 | DeiT-Small | 89.71 | 90.10 | 78.14 | 80.19 | 77.26 | 76.28 | 80.46 | 79.58 | 84.08 | 81.54 | |
| HiCo (ours) | ILSVRC2012 | DeiT-Small | 90.92 | 90.68 | 73.23 | 78.37 | 81.89 | 78.87 | 79.82 | 80.74 | 84.89 | 82.17 | |
| DBHiCo (ours) | ILSVRC2012 | DeiT-Small | 91.92 | 92.22 | 82.19 | 83.23 | 77.93 | 78.08 | 84.68 | 83.29 | 86.51 | 84.21 |
Table 1. Experiment results on Tiantan dataset across four tumors. All methods are trained on both original and skull-stripped MRI scans to ensure fairness. The best result is highlighted in bold, while the second-best is marked with an underline.
When evaluating the re-implemented general and medical contrastive learning (CL) frameworks, BCL [41] and CoCa [7] emerge as the most competitive baselines, achieving the highest (84.71%) and second-highest (84.08%) total ACC, respectively. To ensure a fair structural comparison, we first evaluate a shared-encoder variant of our method, denoted as HiCo. Despite utilizing the same single-branch architecture as the baseline CL methods, HiCo achieves a superior 84.89% total ACC. Furthermore, our full dual-branch architecture, DBHiCo, significantly outperforms all competitors, surpassing BCL and CoCa by 1.80% and 2.43% in total ACC, respectively. As evident in Table 1, both HiCo and DBHiCo consistently deliver substantial improvements across nearly all per-class and overall metrics (ACC and F1-score). These strong results demonstrate that by jointly enforcing fine-grained structural and semantic constraints, our method enables a much more discriminative and robust understanding of MRI representations.
Comparison on the OASIS-I datasets
We further extend our evaluation to the OASIS-1 [10] dataset. As shown in Table 2, existing MRI multi-view representation learning methods plateau at a maximum of 81.02% ACC, 76.07% F1, and 87.64% AUC. Among the baseline models, we observe that ResNet-50 [19] consistently outperforms the ViT-based architectures across all three overall metrics. This superiority can likely be attributed to the limited scale of this downstream dataset, where CNNs typically exhibit stronger inductive biases [67] than Vision Transformers, making them less prone to overfitting. Consequently, we designate ResNet-50 as the standard backbone network for all subsequent contrastive learning comparisons on this dataset.
| Method (Publications, Years) | Pre-training dataset | Network | Input Size | Non Demented ACC(%) | Non Demented F1(%) | Demented ACC(%) | Demented F1(%) | Total ACC(%) | Total F1(%) | AUC(%) |
| MRI Multi-View Representation Learning | ||||||||||
| DAE (MIDL 2024) [28] | DAE-10K [28] | Swin-Unetr | 82.91 | 86.93 | 75.00 | 65.21 | 81.02 | 76.07 | 87.42 | |
| VoCo (CVPR 2024) [29] | VoCo-1.6K [29] | Swin-Unetr | 82.28 | 86.23 | 74.00 | 64.14 | 80.48 | 75.19 | 87.64 | |
| MiM (TMI 2025) [30] | MiM-10K [30] | Swin-ViT | 80.37 | 84.64 | 71.00 | 60.94 | 78.12 | 72.79 | 84.21 | |
| BrainMVP (CVPR 2025) [31] | BrainMVP-16K [31] | Uniformer-small | 76.56 | 82.95 | 76.99 | 61.45 | 76.68 | 72.21 | 84.66 | |
| Baseline | ||||||||||
| ResNet (CVPR 2016) [19] | ILSVRC2012 | ResNet-50 | 83.54 | 86.78 | 72.99 | 64.76 | 81.11 | 75.78 | 86.45 | |
| ViT (ICLR 2021) [18] | ILSVRC2012 | DeiT-Small | 87.34 | 87.31 | 60.00 | 60.15 | 80.56 | 73.67 | 85.30 | |
| ViT Recipe (MICCAI 2024) [11] | Brain-2K [11] | DeiT-Base-3D | 89.25 | 87.11 | 51.00 | 54.79 | 80.06 | 70.96 | 84.00 | |
| With General Contrastive fine-tuning | ||||||||||
| SimCLR (ICML 2020) [32] | ILSVRC2012 | ResNet-50 | 86.38 | 87.86 | 66.99 | 62.05 | 81.66 | 74.96 | 84.93 | |
| SuperCon (NIPS 2020) [40] | ILSVRC2012 | ResNet-50 | 81.01 | 85.96 | 77.00 | 65.28 | 79.99 | 75.62 | 87.76 | |
| BCL (CVPR 2022) [41] | ILSVRC2012 | ResNet-50 | 79.13 | 85.04 | 80.00 | 65.46 | 79.36 | 75.25 | 86.70 | |
| Proco (TPAMI 2024) [42] | ILSVRC2012 | ResNet-50 | 83.53 | 87.42 | 76.00 | 66.22 | 81.82 | 76.82 | 88.84 | |
| With Medical Contrastive fine-tuning | ||||||||||
| ECL (MICCAI 2023) [51] | ILSVRC2012 | ResNet-50 | 79.41 | 85.41 | 80.00 | 65.20 | 79.56 | 75.31 | 88.29 | |
| BPaCo (MICCAI 2024) [52] | ILSVRC2012 | ResNet-50 | 83.54 | 87.35 | 77.00 | 67.81 | 81.97 | 77.58 | 88.34 | |
| CoCa (KBS 2024) [7] | ILSVRC2012 | ResNet-50 | 94.01 | 89.05 | 45.99 | 55.25 | 82.46 | 72.15 | 88.49 | |
| HiCo (ours) | ILSVRC2012 | ResNet-50 | 85.75 | 88.54 | 74.99 | 67.70 | 83.17 | 78.12 | 90.21 | |
| DBHiCo (ours) | ILSVRC2012 | ResNet-50 | 89.88 | 90.49 | 72.00 | 69.70 | 85.55 | 80.10 | 91.13 |
Table 2. Experiment results on OASIS-1 dataset [10].
Notably, our proposed DBHiCo establishes a new state-of-the-art on this benchmark, surpassing all competing methods with an impressive 85.55% ACC, 80.10% F1, and 91.13% AUC. Furthermore, even our single-branch HiCo variant demonstrates significant superiority, yielding absolute improvements of at least 0.71% in ACC (against CoCa), 0.54% in F1 (against BPaCo), and 1.37% in AUC (against Proco) over the strongest baseline methods. Crucially, these striking improvements are attained without relying on any specialized mechanisms for handling class imbalance (unlike methods such as BCL or Proco), thereby underscoring the inherent robustness, scalability, and broad applicability of our hierarchical contrastive framework.
Comparison on the ADHD200 datasets
To rigorously evaluate the cross-site generalizability of our proposed method, we conducted experiments on the multi-center ADHD200 [14] dataset, which comprises data from six independent imaging sites (KKI, Neuro, NYU, OHSU, Peking, and Pitts). As presented in Table 3, we report the classification accuracy for each individual site, alongside the overall aggregate performance evaluated via three comprehensive metrics.
| Method (Publications, Years) | Pre-training dataset | Network | Input Size | KKI | Neuro | NYU | OHSU | Peking | Pitts | ACC (%) | F1 (%) | AUC (%) |
| MRI Multi-View Representation Learning | ||||||||||||
| DAE (MIDL 2024) [28] | DAE-10K [28] | Swin-Unetr | 74.57 | 63.51 | 56.40 | 62.47 | 59.05 | 100.0 | 66.33 | 63.00 | 71.32 | |
| VoCo (CVPR 2024) [29] | VoCo-1.6K [29] | Swin-Unetr | 68.99 | 71.51 | 59.08 | 61.99 | 64.92 | 100.0 | 67.91 | 64.93 | 71.42 | |
| MiM (TMI 2025) [30] | MiM-10K [30] | Swin-ViT | 66.90 | 58.83 | 53.64 | 58.19 | 68.64 | 98.88 | 65.12 | 62.37 | 69.21 | |
| BrainMVP (CVPR 2025) [31] | BrainMVP-16K [31] | Uniformer-small | 73.32 | 45.29 | 60.96 | 59.95 | 61.13 | 100.0 | 66.43 | 60.16 | 69.68 | |
| Baseline | ||||||||||||
| ViT (ICLR 2021) [18] | ILSVRC2012 | DeiT-Small | 68.81 | 56.91 | 55.39 | 62.37 | 65.41 | 100.0 | 65.97 | 62.03 | 72.21 | |
| ViT Recipe (MICCAI 2024) [11] | Brain-2K [11] | DeiT-Base-3D | 73.32 | 40.51 | 59.54 | 58.62 | 62.60 | 100.0 | 65.84 | 61.04 | 70.33 | |
| With General Contrastive fine-tuning | ||||||||||||
| SimCLR (ICML 2020) [32] | ILSVRC2012 | DeiT-Small | 73.24 | 57.55 | 57.50 | 58.72 | 61.33 | 100.0 | 65.90 | 64.63 | 72.59 | |
| SuperCon (NIPS 2020) [40] | ILSVRC2012 | DeiT-Small | 72.07 | 63.09 | 58.11 | 63.68 | 59.71 | 100.0 | 66.72 | 64.81 | 72.35 | |
| BCL (CVPR 2022) [41] | ILSVRC2012 | DeiT-Small | 66.91 | 64.05 | 56.97 | 65.28 | 66.96 | 100.0 | 67.34 | 65.66 | 71.85 | |
| Proco (TPAMI 2024) [42] | ILSVRC2012 | DeiT-Small | 72.01 | 53.87 | 54.27 | 61.68 | 63.31 | 100.0 | 65.03 | 61.48 | 65.96 | |
| With Medical Contrastive fine-tuning | ||||||||||||
| ECL (MICCAI 2023) [51] | ILSVRC2012 | DeiT-Small | 70.82 | 58.05 | 59.02 | 59.50 | 61.98 | 100.0 | 66.14 | 64.07 | 70.05 | |
| BPaCo (MICCAI 2024) [52] | ILSVRC2012 | DeiT-Small | 62.28 | 47.37 | 56.83 | 60.26 | 54.68 | 100.0 | 62.53 | 61.46 | 70.00 | |
| CoCa (KBS 2024) [7] | ILSVRC2012 | DeiT-Small | 72.26 | 62.72 | 62.19 | 59.92 | 59.62 | 100.0 | 67.01 | 64.60 | 73.40 | |
| HiCo (ours) | ILSVRC2012 | DeiT-Small | 68.61 | 80.18 | 58.27 | 68.23 | 62.55 | 100.0 | 68.73 | 66.93 | 73.86 | |
| DBHiCo (ours) | ILSVRC2012 | DeiT-Small | 73.32 | 78.73 | 60.73 | 64.85 | 63.19 | 100.0 | 69.81 | 66.78 | 74.43 |
Table 3. Experiment results on ADHD200 dataset [14] across six independent imaging sites. Note: KKI: Kennedy Krieger Institute, Neuro: NeuroIMAGE, NYU: New York University, OHSU: Oregon Health & Science University, Peking: Peking University, Pitts: University of Pittsburgh.
Notably, both our single-branch HiCo and dual-branch DBHiCo achieve superior overall performance. Specifically, they yield absolute improvements of at least 0.82% (68.73% vs. 67.91%) and 1.90% (69.81% vs. 67.91%) in average ACC, respectively, compared to the strongest baseline method (VoCo [29]). Furthermore, the site-specific accuracies of our approaches consistently rank as the best or second-best across these highly heterogeneous subsets. It is worth noting that the accuracy on the Pitts subset saturates at 100.0% for the majority of evaluated methods; this anomaly is attributable to the extreme class imbalance within this specific cohort, which consists exclusively of the majority class. In summary, these multi-site evaluation results demonstrate that our method exhibits exceptional robustness and generalizability, effectively mitigating potential domain shifts caused by varying scanning parameters across different institutions.
Analysis of our proposed method
Ablation study
Contrastive views. As detailed in § 3.2, our DBHiCo leverages original and skull-stripped MRI scans as complementary views for contrastive pair construction. To validate this design, we evaluated several alternative view-construction strategies on the T1iantan dataset (Table 4), including cross-modality contrasting (T1-E mod T2 mod), varying intensities of data augmentation transformations (Strong trans Weak trans), and the incorporation of both whole-brain structures and localized tumor tissues (Brain Tumor). Notably, isolating specific MRI modalities to form contrastive views is the only strategy that degrades performance compared to the non-contrastive baseline, which we attribute to the fact that the joint utilization of all three modalities is indispensable for accurate tumor diagnosis [31]. Ultimately, our proposed ‘Original Skull-stripped’ paradigm emerges as the optimal strategy, delivering the most substantial performance gains across four out of the five comprehensive evaluation metrics and affirming the value of jointly modeling global anatomical context alongside fine-grained brain tissue features.
| Contrastive Types | ACC(%) | PRE(%) | REC(%) | F1(%) | AUC(%) |
| w/o Dual-Branch & Contrast | 81.88 | 81.23 | 77.90 | 79.18 | 93.81 |
| T1E modT2 mod | 81.55 | 80.09 | 77.18 | 77.63 | 94.56 |
| Strong transWeak trans | 82.38 | 78.63 | 79.11 | 79.40 | 94.60 |
| BrainTumor | 84.27 | 83.05 | 81.27 | 81.92 | 96.10 |
| OriginalSkull-stripped | 86.51 | 84.69 | 84.18 | 84.21 | 96.09 |
Table 4. Evaluation of model with different contrastive views.
Method components. Table 5 assesses the individual and synergistic contributions of DBHiCo’s core components: the dual-branch encoder architecture (Dual-Branch), the hierarchical contrastive loss (Hierarchical), and the contrastive projector (Projector). The results demonstrate progressive performance gains through cumulative integration, culminating in peak performance across all three datasets when the complete configuration is utilized. Notably, the Hierarchical loss exhibits the most substantial independent impact, yielding a 2.48% absolute accuracy improvement on the ADHD200 [14] dataset (68.45% vs. 65.97%) even in a single-branch setup. The inclusion of the Projector further refines these representations by mapping
| Components | Tiantan ACC(%) | Tiantan F1(%) | OASIS-I ACC(%) | OASIS-I F1(%) | ADHD200 ACC(%) | ADHD200 F1(%) | ||
| Dual-Branch | Hierarchical | Projector | 82.27 | 79.18 | 80.56 | 73.73 | 65.97 | 62.03 |
| ✗ | ✓ | ✗ | 83.27 | 80.91 | 83.98 | 77.07 | 68.45 | 66.66 |
| ✗ | ✓ | ✓ | 84.89 | 82.17 | 83.17 | 78.12 | 68.73 | 66.93 |
| ✓ | ✗ | ✗ | 82.16 | 78.50 | 81.67 | 75.35 | 65.99 | 63.91 |
| ✓ | ✓ | ✗ | 85.15 | 83.01 | 83.49 | 79.42 | 68.02 | 66.32 |
| ✓ | ✓ | ✓ | 86.51 | 84.21 | 85.32 | 79.86 | 69.81 | 66.78 |
Table 5. Evaluation of our DBHiCo’s components.
features into a more discriminative latent space, providing consistent incremental gains. Furthermore, while the standalone Dual-Branch architecture offers limited benefits over the baseline, it demonstrates a powerful synergistic effect when coupled with the Hierarchical loss and Projector, ultimately driving profound F1-score enhancements of 5.71% (84.21% vs. 78.50%) on Tiantan and 4.51% (79.86% vs. 75.35%) on OASIS-I [10] compared to the isolated dual-branch configuration.
Loss function. Table 6 demonstrates the empirical necessity of explicitly modeling distinct negative difficulty levels through our proposed hierarchical contrastive loss. When upgrading the standard contrastive loss (detailed in § 3.3) to our hierarchical variant, both the single-branch HiCo and dual-branch DBHiCo architectures exhibit consistent and substantial performance gains across all three evaluated datasets. Most notably, integrating this hierarchical objective into the DBHiCo framework yields absolute accuracy improvements of 2.91%, 1.99%, and 1.69% on the Tiantan, OASIS-I, and ADHD200 benchmarks, respectively, firmly validating that a calibrated negative sampling strategy is crucial for learning highly discriminative medical image representations.
| Method | Loss function | Tiantan ACC(%) | Tiantan F1(%) | OASIS-I ACC(%) | OASIS-I F1(%) | ADHD200 ACC(%) | ADHD200 F1(%) |
| HiCo | Contrastive loss [53] | 83.72 | 81.13 | 82.85 | 74.88 | 66.57 | 64.01 |
| Hierarchical Contrastive loss | 84.89 | 82.17 | 83.17 | 78.12 | 68.73 | 66.93 | |
| DBHiCo | Contrastive loss [53] | 83.60 | 80.64 | 83.33 | 76.89 | 68.12 | 63.72 |
| Hierarchical Contrastive loss | 86.51 | 84.21 | 85.32 | 79.86 | 69.81 | 66.78 |
Table 6. Evaluation of whether using hierarchical difficulty.
Loss terms of . Next, we systematically evaluate the individual contributions of the four difficulty-aware contrastive constraints ( to ) formulated from our prior conditions (§ 3.2). By isolating each hierarchical level, Table 7 demonstrates that imposing constraints on the most challenging negative pairs generally yields the most substantial performance benefits; for instance, the most stringent condition () consistently outperforms the most relaxed condition () across all datasets (e.g., 84.09% vs. 82.40% accuracy on Tiantan). Although the individual effectiveness of these terms progressively diminishes as the negative pair difficulty decreases, integrating all four levels is crucial, as their synergistic combination achieves the absolute peak performance across every metric and benchmark.
| Loss | Tiantan ACC(%) | Tiantan F1(%) | OASIS-I ACC(%) | OASIS-I F1(%) | ADHD200 ACC(%) | ADHD200 F1(%) | ||||
| ✓ | ✗ | ✗ | ✗ | 84.09 | 81.26 | 83.57 | 77.65 | 66.76 | 64.87 | |
| ✗ | ✓ | ✗ | ✗ | 83.72 | 81.08 | 82.53 | 77.29 | 65.75 | 63.96 | |
| ✗ | ✗ | ✓ | ✗ | 84.68 | 82.13 | 81.82 | 76.32 | 65.85 | 63.58 | |
| ✗ | ✗ | ✗ | ✓ | 82.40 | 79.53 | 81.98 | 77.74 | 66.23 | 62.98 | |
| ✓ | ✓ | ✓ | ✓ | 86.51 | 84.21 | 85.32 | 79.86 | 69.81 | 66.78 |
Table 7. Evaluation of the loss terms .
Hyper-parameter analysis
Weights of . To evaluate the impact of the hyperparameter weighting , we visualize the performance variations across all datasets in Figure 4. The empirical results demonstrate that the evaluation metrics consistently peak when is set to 1.0. Generally, increasing the value up to this optimal threshold yields steady overall performance improvements. This positive trend is particularly pronounced on the more challenging ADHD200 [14] benchmark, firmly underscoring the critical role of our proposed in simultaneously reinforcing structural consistency and fine-grained semantic discriminability within MRI representations.

Figure 4. Evaluation of the loss weight for . (a) AVG; (b) Tiantan-Tumor Dataset; (c) OASIS-I Dataset; (d) ADHD200 Dataset.
Margin-parameter adjustment strategy. As detailed in § 3.3, the margin parameter is critical for computing . Table 8 evaluates two adjustment dimensions: Uniform (applying a fixed across all hierarchical levels versus assigning distinct values of 0.25, 0.2, 0.15, and 0.1 based on negative difficulty) and Decay (maintaining a static margin throughout training versus applying an epoch-wise decay rate of 0.99). Empirical results indicate that a static, uniform margin strategy optimally balances simplicity and effectiveness on the Tiantan and OASIS-I [10] datasets, achieving peak accuracies of 86.51% and 85.32%, respectively. Conversely, for the more challenging and heterogeneous ADHD200 [14] benchmark, employing distinct, non-decaying margin values yielded the best performance (69.81% accuracy), demonstrating the flexibility of our methods in tailoring margin constraints to suit varying downstream dataset complexities.
| Margin | Uniform | Decay | Tiantan ACC(%) | Tiantan F1(%) | OASIS-I ACC(%) | OASIS-I F1(%) | ADHD200 ACC(%) | ADHD200 F1(%) |
| ✓ | ✗ | 86.51 | 84.21 | 85.32 | 79.86 | 68.26 | 65.58 | |
| ✓ | ✓ | 85.28 | 83.08 | 83.65 | 77.84 | 66.72 | 64.25 | |
| ✗ | ✓ | 85.53 | 82.93 | 82.94 | 76.86 | 69.81 | 66.78 | |
| ✗ | ✗ | 85.05 | 82.70 | 84.37 | 79.61 | 68.17 | 66.00 |
Table 8. Evaluation of our DBHiCo’s margin-parameter adjustment strategy.
Interpretability
Grad-CAM visualization. We analyze Grad-CAM [54] attention maps across four tumor types of the Tiantan dataset from the perspective of structural-semantic complementarity. As shown in Figure 3, SimCLR mainly focuses on global representation learning, but its prone to being affected by the background, resulting in blurred attention map boundaries and distorted shapes. For example, in the attention map of Medulloblastoma, the focused area fails to align with the elliptical contour of the cerebellar vermis. In contrast, our HiCo enforces the model to learn fine-grained structural details through hierarchical CL, while DBHiCo further enhances the precise understanding of anatomical structures and pathological tissues.

Figure 3. Grad-CAM [54] visualizations across four tumor types on the Tiantan dataset. While the baseline SimCLR is prone to background interference and exhibits blurred boundaries, our proposed HiCo effectively captures fine-grained structural details. Furthermore, DBHiCo achieves the most precise localization of anatomical structures and pathological tissues.
T-SNE visualization. Furthermore, we analyze T-SNE [68] visualizations by evaluating class separation and centroid distance across OASIS-I [10] and ADHD200 [14] datasets as shown in Figure 5. Qualitatively, compared with SimCLR, our DBHiCo shows significantly better discrimination on both of these two binary clusters, indicating that the feature representation of DBHiCo significantly amplifies the inter-class differences. Qualitatively, DBHiCo achieves lower center distances (e.g., on the Demented of OASIS-I [10], distance for DBHiCo vs. SimCLR: 7.11 vs. 10.82), proving that DBHiCo improves the intra-class compactness and thereby achieves more accurate automatic diagnosis.

Figure 5. Visualization of T-SNE [68] results. (a)-(b) Feature distributions for SimCLR on OASIS-I and ADHD200, exhibiting noticeable class overlap. (c)-(d) Feature distributions for DBHiCo on the respective datasets, showing enhanced inter-class separability and tighter intra-class compactness.
Conclusion
This paper presents DBHiCo, a novel multi-view representation learning framework designed to overcome the inherent noise and high inter-subject similarity challenges in automated MRI classification. Unlike existing contrastive paradigms that rely on rigid binary partitions, DBHiCo collaboratively enforces structural consistency and semantic discrimination through a dedicated dual-branch architecture, seamlessly integrating holistic anatomical boundaries with localized intracranial details derived from paired original and skull-stripped scans. To effectively align these fine-grained features, we introduce a hierarchical contrastive loss () that replaces conventional uniform penalties with a margin-ranked similarity ordering, explicitly calibrating positive and negative pairs based on their cross-view and cross-class difficulty. Extensive experiments across three diverse neurological datasets demonstrate that DBHiCo not only establishes state-of-the-art diagnostic performance but also significantly enhances inter-class separability in the latent space.
Acknowledgements
This work was supported by the Natural Science Foundation of China under Grant 62571507.
References
- [1]G. Katti, S. A. Ara, A. Shireen, Magnetic resonance imaging (mri)—a review, International journal of dental clinics 3 (1) (2011) 65–70.
- [2]M. Mousavian, J. Chen, Z. Traylor, S. Greening, Depression detection from smri and rs-fmri images using machine learning, Journal of Intelligent Information Systems 57 (2) (2021) 395–418.DOI
- [3]S. H. Hojjati, A. Ebrahimzadeh, A. Khazaee, A. Babajani-Feremi, A. D. N. Initiative, et al., Predicting conversion from mci to ad by integrating rs-fmri and structural mri, Computers in biology and medicine 102 (2018) 30–39.
- [4]S. Bouhafra, H. El Bahi, Deep learning approaches for brain tumor detection and classification using mri images (2020 to 2024): a systematic review, Journal of Imaging Informatics in Medicine 38 (3) (2025) 1403–1433.DOI
- [5]R. Alexander, S. Waite, M. A. Bruno, E. A. Krupinski, L. Berlin, S. Macknik, S. Martinez-Conde, Mandating limits on workload, duty, and speed in radiology, Radiology 304 (2) (2022) 274–282.DOI
- [6]T. C. Kwee, R. M. Kwee, Workload of diagnostic radiologists in the foreseeable future based on recent scientific advances: growth expectations and role of artificial intelligence, Insights into imaging 12 (1) (2021) 88.DOI
- [7]C. Sun, Z. Yan, Y. Zhang, X. Tian, J. Gong, Expert-level diagnosis of pediatric posterior fossa tumors via consistency calibration, Knowledge-Based Systems 296 (2024) 111919.DOI
- [8]L. Reddy, M. Elangovan, M. Vamsikrishna, C. Ravindra, Brain tumor detection and classification using deep learning models on mri scans., EAI Endorsed Transactions on Pervasive Health & Technology 10 (1) (2024).
- [9]G. Singh, A. Chhabra, A. Mittal, Evaluating deep learning algorithms for mri-based brain tumor classification, in: 2024 International Conference on Emerging Innovations and Advanced Computing (INNOCOMP), IEEE, 2024, pp. 428–434.
- [10]D. S. Marcus, T. H. Wang, J. Parker, J. G. Csernansky, J. C. Morris, R. L. Buckner, Open access series of imaging studies (oasis): cross-sectional mri data in young, middle aged, nondemented, and demented older adults, Journal of cognitive neuroscience 19 (9) (2007) 1498–1507.
- [11]K. Kunanbayev, V. Shen, D.-S. Kim, Training vit with limited data for alzheimer’s disease classification: An empirical study, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2024, pp. 334–343.DOI
- [12]H. Liu, L. Shi, Y. Gao, Y. Shi, J. Xu, Y. Zhang, Q. Wang, Simcmc: A simple compact multi-view contrastive framework for self-supervised early alzheimer’s disease diagnosis, IEEE Transactions on Instrumentation and Measurement (2025).
- [13]W. Cui, Y. Leng, Y. Peng, C. Bai, L. Li, X. Jiang, G. Yuan, J. Zheng, A novel dynamic neural network for heterogeneity-aware structural brain network exploration and alzheimer’s disease diagnosis, IEEE Transactions on Neural Networks and Learning Systems (2025).DOI
- [14]A.-. consortium, The adhd-200 consortium: a model to advance the translational potential of neuroimaging in clinical neuroscience, Frontiers in systems neuroscience 6 (2012) 62.
- [15]D. Kuang, L. He, Classification on adhd with deep learning, in: 2014 International Conference on Cloud Computing and Big Data, IEEE, 2014, pp. 27–32.
- [16]M. N. I. Qureshi, B. Lee, Classification of adhd subgroup with recursive feature elimination for structural brain mri, in: 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE, 2016, pp. 5929–5932.
- [17]L. Shi, L. Shi, Z. Cui, C. Lin, R. Zhang, J. Zhang, Y. Zhu, W. Shi, J. Wang, Y. Wang, et al., Transformer-based structural connectivity networks for adhd-related connectivity alterations, Communications Medicine 5 (1) (2025) 296.DOI
- [18]A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al., An image is worth 16x16 words: Transformers for image recognition at scale, arXiv preprint arXiv:2010.11929 (2020).arxiv.org/abs/2010.11929
- [19]K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
- [20]Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, B. Guo, Swin transformer: Hierarchical vision transformer using shifted windows, in: Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 10012–10022.arxiv.org/abs/2103.14030
- [21]H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, H. Jégou, Training data-efficient image transformers & distillation through attention, in: International conference on machine learning, PMLR, 2021, pp. 10347–10357.
- [22]I. Sebenius, L. Dorfschmidt, J. Seidlitz, A. Alexander-Bloch, S. E. Morgan, E. Bullmore, Structural mri of brain similarity networks, Nature Reviews Neuroscience 26 (1) (2025) 42–59.
- [23]R. Khan, S. Akbar, A. Mehmood, F. Shahid, K. Munir, N. Ilyas, M. Asif, Z. Zheng, A transfer learning approach for multiclass classification of alzheimer’s disease using mri images, Frontiers in Neuroscience 16 (2023) 1050777.DOI
- [24]C. Srinivas, N. P. KS, M. Zakariah, Y. A. Alothaibi, K. Shaukat, B. Partibane, H. Awal, Deep transfer learning approaches in performance analysis of brain tumor classification using mri images, Journal of Healthcare Engineering 2022 (1) (2022) 3264367.
- [25]A. K. Sharma, A. Nandal, A. Dhaka, L. Zhou, A. Alhud-haif, F. Alenezi, K. Polat, Brain tumor classification using the modified resnet50 model based on transfer learning, Biomedical Signal Processing and Control 86 (2023) 105299.DOI
- [26]I. S. Rajput, A. Gupta, V. Jain, S. Tyagi, A transfer learning-based brain tumor classification using magnetic resonance images, Multimedia Tools and Applications 83 (7) (2024) 20487–20506.
- [27]A. Mehmood, S. Yang, Z. Feng, M. Wang, A. S. Ahmad, R. Khan, M. Maqsood, M. Yaqub, A transfer learning approach for early diagnosis of alzheimer’s disease on mri images, Neuroscience 460 (2021) 43–52.
- [28]J. M. J. Valanarasu, Y. Tang, D. Yang, Z. Xu, C. Zhao, W. Li, V. M. Patel, B. Landman, D. Xu, Y. He, et al., Disruptive autoencoders: Leveraging low-level features for 3d medical image pre-training, arXiv preprint arXiv:2307.16896 (2023).arxiv.org/abs/2307.16896
- [29]L. Wu, J. Zhuang, H. Chen, Voco: A simple-yet-effective volume contrastive learning framework for 3d medical image analysis, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 22873–22882.arxiv.org/abs/2402.17300
- [30]J. Zhuang, L. Wu, Q. Wang, P. Fei, V. Vardhanabhuti, L. Luo, H. Chen, Mim: Mask in mask self-supervised pre-training for 3d medical image analysis, IEEE Transactions on Medical Imaging (2025).
- [31]S. Rui, L. Chen, Z. Tang, L. Wang, M. Liu, S. Zhang, X. Wang, Multi-modal vision pre-training for medical image analysis, in: Proceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 5164–5174.
- [32]T. Chen, S. Kornblith, M. Norouzi, G. Hinton, A simple framework for contrastive learning of visual representations, in: International conference on machine learning, PmLR, 2020, pp. 1597–1607.arxiv.org/abs/2002.05709
- [33]A. Taleb, C. Lippert, T. Klein, M. Nabi, Self-supervised learning for medical images by solving multimodal jigsaw puzzles, IEEE Transact Med Imaging 12729 (2017) 661–673.
- [34]A. S. Ridha, A. T. Hashim, Improves brain skull stripping using attention residual recurrent 3d u-net, in: International Conference on Data Analytics & Management, Springer, 2024, pp. 289–315.DOI
- [35]J. Acosta-Cabronero, G. B. Williams, J. M. Pereira, G. Pengas, P. J. Nestor, The impact of skull-stripping and radio-frequency bias correction on grey-matter segmentation for voxel-based morphometry, Neuroimage 39 (4) (2008) 1654–1665.
- [36]A. Lakshmi, T. Arivoli, R. Vinupriyadharshini, Noise and skull removal of brain magnetic resonance image using curvelet transform and mathematical morphology, in: 2014 International Conference on Electronics and Communication Systems (ICECS), IEEE, 2014, pp. 1–4.DOI
- [37]P. Kalavathi, V. S. Prasath, Methods on skull stripping of mri head scan images—a review, Journal of digital imaging 29 (3) (2016) 365–379.DOI
- [38]J. DiGregorio, G. Arezza, A. Gibicar, A. R. Moody, P. N. Tyrrell, A. Khademi, Intracranial volume segmentation for neurodegenerative populations using multicentre flair mri, Neuroimage: Reports 1 (1) (2021) 100006.DOI
- [39]F. Isensee, M. Schell, I. Pflueger, G. Brugnara, D. Bonekamp, U. Neuberger, A. Wick, H.-P. Schlemmer, S. Heiland, M. Wick, et al., Automated brain extraction of multisequence mri using artificial neural networks, Human brain mapping 40 (17) (2019) 4952–4964.DOI
- [40]P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y. Tian, P. Isola, A. Maschinot, C. Liu, D. Krishnan, Supervised contrastive learning, Advances in neural information processing systems 33 (2020) 18661–18673.arxiv.org/abs/2004.11362
- [41]J. Zhu, Z. Wang, J. Chen, Y.-P. P. Chen, Y.-G. Jiang, Balanced contrastive learning for long-tailed visual recognition, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 6908–6917.
- [42]C. Du, Y. Wang, S. Song, G. Huang, Probabilistic contrastive learning for long-tailed visual recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence 46 (9) (2024) 5890–5904.
- [43]A. S. Talai, J. Sedlacik, K. Boelmans, N. D. Forkert, Utility of multi-modal mri for differentiating of parkinson’s disease and progressive supranuclear palsy using machine learning, Frontiers in Neurology 12 (2021) 648548.DOI
- [44]Z. Chen, D. Agarwal, K. Aggarwal, W. Safta, M. M. Balan, K. Brown, Masked image modeling advances 3d medical image analysis, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023, pp. 1970–1980.
- [45]J. Zhuang, L. Luo, Q. Wang, M. Wu, L. Luo, H. Chen, Advancing volumetric medical image segmentation via global-local masked autoencoders, IEEE Transactions on Medical Imaging (2025).DOI
- [46]Q. Wang, Z. Wen, J. Shi, Q. Wang, D. Shen, S. Ying, Spatial and modal optimal transport for fast cross-modal mri reconstruction, IEEE Transactions on Medical Imaging 43 (11) (2024) 3924–3935.
- [47]H. Zhang, Q. Wang, J. Shi, S. Ying, Z. Wen, Deep unfolding network with spatial alignment for multi-modal mri reconstruction, Medical Image Analysis 99 (2025) 103331.
- [48]Z. Zeng, Z. Peng, X. Yang, W. Shen, Missing as masking: Arbitrary cross-modal feature reconstruction for incomplete multimodal brain tumor segmentation, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2024, pp. 424–433.DOI
- [49]T. Liu, W. Liu, L. Yu, L. Wan, T. Han, L. Zhu, Joint prediction of meningioma grade and brain invasion via task-aware contrastive learning, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2022, pp. 355–365.arxiv.org/abs/2209.01517
- [50]Z. Li, C. Huang, S. Xie, Multimodality-assisted semi-supervised brain tumor segmentation in nondominant modality based on consistency learning, IEEE Transactions on Instrumentation and Measurement 73 (2024) 1–11.DOI
- [51]Y. Zhang, J. Chen, K. Wang, F. Xie, Ecl: Class-enhancement contrastive learning for long-tailed skin lesion classification, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2023, pp. 244–254.arxiv.org/abs/2307.04136
- [52]Z. Cai, T. Wei, L. Lin, H. Chen, X. Tang, Bpaco: Balanced parametric contrastive learning for long-tailed medical image classification, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2024, pp. 383–393.DOI
- [53]A. v. d. Oord, Y. Li, O. Vinyals, Representation learning with contrastive predictive coding, arXiv preprint arXiv:1807.03748 (2018).arxiv.org/abs/1807.03748
- [54]R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, D. Batra, Grad-cam: Visual explanations from deep networks via gradient-based localization, in: Proceedings of the IEEE international conference on computer vision, 2017, pp. 618–626.
- [55]J. Jeong, L. Wang, B. Ji, Y. Lei, A. Ali, T. Liu, W. J. Curran, H. Mao, X. Yang, Machine-learning based classification of glioblastoma using delta-radiomic features derived from dynamic susceptibility contrast enhanced magnetic resonance images: Introduction, Quantitative imaging in medicine and surgery 9 (7) (2019) 1201.DOI
- [56]T. Nakamoto, W. Takahashi, A. Haga, S. Takahashi, S. Kiryu, K. Nawa, T. Ohta, S. Ozaki, Y. Nozawa, S. Tanaka, et al., Prediction of malignant glioma grades using contrast-enhanced t1-weighted and t2-weighted magnetic resonance images based on a radiomic analysis, Scientific reports 9 (1) (2019) 19411.DOI
- [57]S. A. A. Ismael, A. Mohammed, H. Hefny, An enhanced deep learning approach for brain cancer mri images classification using residual networks, Artificial intelligence in medicine 102 (2020) 101779.
- [58]J. Quon, W. Bala, L. Chen, J. Wright, L. Kim, M. Han, K. Shpanskaya, E. Lee, E. Tong, M. Iv, et al., Deep learning for pediatric posterior fossa tumor detection and classification: a multi-institutional study, American Journal of Neuroradiology 41 (9) (2020) 1718–1725.DOI
- [59]S. Song, Y. Zheng, Y. He, A review of methods for bias correction in medical images, Biomedical Engineering Review 1 (1) (2017).
- [60]S. M. Smith, Fast robust automated brain extraction, Human brain mapping 17 (3) (2002) 143–155.
- [61]D. W. Shattuck, S. R. Sandor-Leahy, K. A. Schaper, D. A. Rottenberg, R. M. Leahy, Magnetic resonance image tissue classification using a partial volume model, NeuroImage 13 (5) (2001) 856–876.
- [62]S. F. Eskildsen, P. Coupé, V. Fonov, J. V. Manjón, K. K. Leung, N. Guizard, S. N. Wassef, L. R. Østergaard, D. L. Collins, A. D. N. Initiative, et al., Beast: brain extraction based on nonlocal segmentation technique, NeuroImage 59 (3) (2012) 2362–2373.DOI
- [63]S. Roy, J. A. Butman, D. L. Pham, A. D. N. Initiative, et al., Robust skull stripping using multiple mr image contrasts insensitive to pathology, Neuroimage 146 (2017) 132–147.DOI
- [64]O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al., Imagenet large scale visual recognition challenge, International journal of computer vision 115 (3) (2015) 211–252.
- [65]A. Hatamizadeh, V. Nath, Y. Tang, D. Yang, H. R. Roth, D. Xu, Swin unetr: Swin transformers for semantic segmentation of brain tumors in mri images, in: International MICCAI brainlesion workshop, Springer, 2021, pp. 272–284.arxiv.org/abs/2201.01266
- [66]K. Li, Y. Wang, P. Gao, G. Song, Y. Liu, H. Li, Y. Qiao, Uniformer: Unified transformer for efficient spatiotemporal representation learning, arXiv preprint arXiv:2201.04676 (2022).arxiv.org/abs/2201.04676
- [67]S. d’Ascoli, H. Touvron, M. L. Leavitt, A. S. Morcos, G. Biroli, L. Sagun, Convit: Improving vision transformers with soft convolutional inductive biases, in: International conference on machine learning, PMLR, 2021, pp. 2286–2296.DOI
- [68]L. v. d. Maaten, G. Hinton, Visualizing data using t-sne, Journal of machine learning research 9 (Nov) (2008) 2579–2605.