betaFeedback

WorksComputer Science

Ref-MIL: Plug-and-Play Posterior Refinement for Multiple Instance Learning in QSM-Based Brain Disease Classification

Abstract

Quantitative susceptibility mapping (QSM) is a novel quantitative MRI contrast for studying tissue susceptibility changes in brain disorders, including Parkinson's disease (PD) and multiple sclerosis (MS). However, small cohorts and the lack of local annotations make developing deep learning models for QSM-based diagnosis challenging. In this work, we propose PoRef-MIL, a weakly supervised multiple instance learning (MIL) framework combining iterative slice relabeling and network retraining with a plug-and-play posterior refinement module (PRM). During training, a 2D dual-branch diagnosis network and a subject-level classifier were alternately trained for five rounds. The classifier was used to update slice pseudo-labels for retraining the diagnosis network. The PRM was applied during inference to refine slice probabilities through bidirectional posterior inference and calculate entropy-derived confidence from the refined probabilities, which was then used in sliding-window scoring to obtain subject-level predictions. PoRef-MIL was evaluated on an in-house PD cohort of 287 participants and a public MS cohort of 150 participants. Compared with eight classification methods, PoRef-MIL achieved the highest mean areas under the receiver operating characteristic curve of 0.942 and 0.872, and accuracies of 0.926 and 0.833, for PD and MS classification, respectively. Ablation experiments were conducted to validate the effectiveness of iterative relabeling and PRM, and the plug-and-play feature of the proposed PRM was also evaluated using four state-of-the-art MIL models. Overall, PoRef-MIL offers a promising approach to QSM-based disease classification in small cohorts without manual slice annotations.

Introduction

Quantitative susceptibility mapping (QSM) is a novel MRI post-processing technique capable of calculating the spatial distribution of tissue magnetic susceptibility from phase measurements [46, 27, 31], which is sensitive to changes in paramagnetic iron and diamagnetic myelin and provides quantitative information to investigate tissue variations in a range of neurological disorders [2, 37, 5]. For instance, in Parkinson’s disease (PD), previous QSM studies have reported increased susceptibility in the substantia nigra, together with susceptibility alterations in other subcortical and cortical regions, highlighting the regional distribution of disease-related abnormalities [24, 1, 15, 16], while in multiple sclerosis (MS), lesion susceptibility is reported to be influenced by both myelin loss and iron accumulation [23, 48, 8, 44]. QSM has also demonstrated great potential in studying inflammatory diseases and characterizing differences in lesion tissue properties and their evolution [22, 33, 11].

Existing studies have developed several approaches to extracting disease-related patterns from 3D QSM images, including hybrid feature extraction [49, 53], radiomics-based classification [21] (Li et al. 2019), and multimodal machine learning that combines susceptibility and metabolic information [41] (Yu et al. 2026). Some deep neural networks have also been adopted for PD classification [45] and for identifying chronic active MS lesions by combining lesion-level convolutional and radiomic features [53]. In addition to disease classification, QSM-based graph neural networks have been investigated for assessing postural abnormalities, rigidity, and arising-from-chair abnormalities in PD patients [39, 38 b]. Despite these promising applications, deep learning-based disease classification using QSM remains relatively underexplored.

Both limited training data and the granularity of available annotations hinder broader development of deep learning-based QSM disease classification methods. In medical imaging, deep models often need to be developed using relatively small patient cohorts, which provide only a small number of examples for inter-subject variability and disease-related imaging patterns [6, 20]. In addition, subject-level diagnostic labels do not specify which slices or regions contain disease-related abnormalities [28, 6, 47]. Although these diagnostic labels can supervise subject-level classification, they do not provide the local targets required for fully supervised slice-level learning, and therefore cannot provide slice- or region-level evidence. Simply assigning the subject’s diagnosis to every slice will neglect the spatially heterogeneous distribution of abnormalities within a brain volume [6, 47, 20]. These limitations motivate weakly supervised approaches that can learn disease-relevant local patterns from the subject-level labels.

Multiple instance learning (MIL) provides a suitable framework by representing each subject as a bag of image instances and aggregating local image information for subject-level classification without requiring explicit instance-level annotations [4, 6, 43, 20]. This paradigm has been widely adopted for gigapixel whole-slide image (WSI) classification in computational pathology [35, 52, 3, 50] (Lu et al. 2021). It has also been applied to brain MRI classification through different strategies for representing and aggregating local image information [28, 55, 54]. Previous studies have explored landmark-based instance construction [28], dual-attention aggregation [55], multi-instance knowledge distillation [14], anatomy-guided instance selection [19], and slice- or patch-based aggregation for subject- or examination-level diagnosis [54]. There are also some methods investigating attention-based pooling, inter-instance correlation modeling and feature distillation to improve instance representation and bag-level prediction [18, 35, 52]. In QSM analysis, MIL-based graph networks have also incorporated intervention and counterfactual strategies to reduce the influence of irrelevant or spuriously relevant features [39, 38].

However, several challenges remain when applying MIL to QSM-based brain disease classification. For example, accurate subject-level classification does not necessarily imply reliable instance-level predictions [43, 47, 3], which is particularly relevant to 3D QSM images, where disease-related susceptibility changes may be concentrated in specific regions rather than distributed uniformly throughout the brain [1, 16, 33]. For interpreting QSM-based predictions, explicit slice-level labels accompanied by prediction probabilities and uncertainty information would allow the distribution of model-identified abnormalities to be examined in greater detail. Furthermore, reducing annotation requirements in MIL does not totally eliminate the limitations imposed by small training cohorts [28, 47], which may result in a decrease in performance due to insufficient inter-subject diversity. Most previous MIL methods chose to improve classification performance through novel designs in more efficient feature learning [28, 52], attention mechanisms [18] (Lu et al. 2021), or instance aggregation [35] within the trained model.

In this study, we propose PoRef-MIL, a multiple instance learning framework with a plug-and-play Posterior Refinement Module (PRM) for QSM-based brain disease classification. PoRef-MIL treats each 3D QSM volume as a sequence of consecutive 2D slices and uses a 2D dual-branch diagnosis network to predict slice-level abnormality probabilities. The iterative training strategy alternates between training a subject-level classifier to generate slice pseudo-labels (relabeling procedure) and retraining the diagnosis network using these updated labels. The PRM refines the ordered slice-level predictions through bidirectional posterior inference, followed by sliding-window scoring weighted by entropy-derived confidence to obtain final subject-level predictions. PoRef-MIL was evaluated on an in-house PD QSM dataset and a publicly available MS QSM dataset (Fiscone et al., 2024) through comparisons with eight state-of-the-art methods. Slice-level gradient-weighted class activation mapping (Grad-CAM) heatmaps [34] were generated to examine the spatial regions contributing to model predictions, together with slice abnormality probability distributions, refined slice states, and confidence curves. Ablation experiments were conducted to assess the effectiveness of iterative relabeling and the PRM. The module was further applied to four state-of-the-art MIL-based classifiers [55, 3, 50, 35], with the pretrained backbones kept frozen, to demonstrate its plug-and-play capability. Experiments combining QSM with complementary MRI contrasts, including T1-weighted (T1w) and fluid-attenuated inversion recovery (FLAIR) images, were also conducted to evaluate the framework’s applicability to multimodal classification. Finally, age conditioning using feature-wise linear modulation (FiLM) [30] was also evaluated to examine the influence of incorporating age information on classification performance.

Method

Overview of the Proposed PoRef-MIL Framework

The proposed PoRef-MIL framework performs subject-level brain disease classification from 3D QSM volumes under weak supervision, using only subject-level diagnostic labels. It is also able to produce slice-level abnormality probability and confidence profiles, and generates slice-level pseudo-labels during training. Following many previous MIL formulations [18, 28, 4, 35, 50, 3], each subject is represented as a bag of image instances in this work, corresponding to consecutive 2D slices extracted from a 3D QSM volume.

Specifically, the 2D dual-branch diagnosis network (DBD-Net) combines a vision transformer (ViT) branch [10] with a convolutional neural network (CNN) branch to extract image features and predict an abnormality probability for each slice (Figure 1(a)). DBD-Net is trained using an iterative strategy that alternates between slice relabeling (i.e., training a subject-level classifier to generate slice pseudo-labels) and network retraining (i.e., retraining the slice-level DBD-Net using these updated labels), as detailed in § 2.2.

Figure 1. Overview of the 2D dual-branch diagnosis network and its iterative training in PoRef-MIL. (a) In the 2D diagnosis network, each 2D QSM slice is processed by four feature extraction blocks (FEBs), followed by max pooling and slice-level classification. (b) Each FEB block fuses the features from the transformer-based and convolutional branches through cross-attention. (c) Latent features extracted from all 2D slices within a 3D volume are processed by self-attention along the slice direction, and slice scores are then aggregated using attention weights to compute a bag-level focal loss under subject-level supervision. Pseudo-labels derived from these scores are used to retrain the diagnosis network and update the FEB for the next relabeling round. This cycle is repeated for KK rounds.

During inference, a plug-and-play posterior refinement module (PRM) refines the ordered slice-level abnormality probabilities produced by the pretrained DBD-Net. Entropy-derived confidence is then calculated from the refined probabilities and incorporated into a sliding-window scoring process to obtain the final subject-level prediction, as illustrated in Figure 2 and detailed in § 2.3.

Figure 2. Posterior Refinement Module (PRM) in the proposed PoRef-MIL inference framework. The 3D QSM volumes are treated as a consecutive series of 2D slices, which are fed into the pretrained 2D dual-branch diagnosis network to obtain initial slice probabilities. The slice probabilities and states are then refined using a bidirectional posterior refinement module. After the refinement, entropy-derived confidence is calculated from these probabilities and used in sliding-window scoring to determine the final subject-level predictions.

The 2D Dual-Branch Diagnosis Network and Its Iterative Training

As illustrated in Figure 1(a), the proposed DBD-Net consists of four sequential feature extraction blocks (FEBs) and an MLP-based slice classifier. The features extracted by the FEBs are reduced to a 256-dimensional vector via max pooling and then passed to an MLP classifier with successive hidden-layer widths of 512, 1024, 512, 256, 128, and 2, followed by a single output unit for slice-level abnormality prediction. The structure of each FEB is detailed in Figure 1(b), where each FEB is composed of a transformer branch for modeling relationships among image tokens and a convolutional branch for extracting local spatial features. In the transformer branch, tokenized features are processed by two consecutive attention feed-forward blocks following the transformer architecture (Vaswani et al., [42]; Dosovitskiy et al. [10]), each containing layer normalization, multi-head self-attention, and a two-layer feed-forward network that expands the embedding dimension from 256 to 512 and projects it back to 256. In parallel, the convolutional branch initially applies four convolutional stages, each consisting of a 3×33 \times3 convolution, batch normalization, ReLU activation, and max pooling. The resulting latent features are then processed by a 1×11 \times1 convolutional layer to adjust the number of channels to 128. Next, a feature bridge consisting of layer normalization, reshaping, and a 1×11 \times1 convolution is introduced to transfer intermediate transformer features to the convolutional branch, via channel-wise concatenation. The combined features are then fed into two consecutive residual convolutional blocks, each comprising a 1×11 \times1 convolution and a 3×33 \times3 convolution. Finally, features from the convolutional and transformer branches are fused using cross-attention to produce the FEB output.

In this work, the proposed DBD-Net was trained using an iterative strategy that alternates between slice relabeling and network retraining to obtain slice-level supervision using only subject-level diagnostic labels, as illustrated in Figure 1(c).

Initially, the diagnostic label of each training subject was assigned to all slices within the corresponding QSM volume for an initial warm-up training. These initial slice labels were used to train DBD-Net by minimizing a binary focal loss (Lin et al., [26]) calculated for individual slices, as in traditional supervised learning.

After the initial warm-up training, the weights of the four pretrained FEBs were transferred to a slice relabeling module. Specifically, for each 3D QSM volume (data size: H×W×DH \times W \times D), features extracted from all DD slices were arranged in their original order and processed by self-attention along the slice direction to incorporate information across all slices. The resulting latent feature representation of each slice was then passed through max pooling (along the second dimension) and an MLP to produce a raw score sis_i, where i=1,2,…,Di = 1, 2, \ldots, D denotes the slice index within the 3D volume. This relabeling classifier used the same layer configuration as the DBD-Net slice classifier, but its parameters were learned separately. The same raw scores were passed through the sigmoid and softmax operations to obtain the slice abnormality probabilities and corresponding attention weights, respectively:

pi=σ(si)=11+exp⁡(−si),ai=exp⁡(si)∑j=1Dexp⁡(sj),(1)p_i = \sigma(s_i) = \frac{1}{1+\exp(-s_i)}, \qquad a_i = \frac{\exp(s_i)}{\sum_{j=1}^{D}\exp(s_j)}, \tag*{(1)}

where pi∈[0,1]p_i \in[0,1] denotes the slice abnormality probability for slice ii, with 0 representing a healthy slice and 1 denoting a potentially abnormal slice, and aia_i denotes its attention weight. The softmax operation normalizes the scores across the slices from the same subject, such that ∑i=1Dai=1\sum_{i=1}^{D} a_i = 1.

Similar to previous attention-weighted aggregations (Ilse et al., [18]), the slice abnormality probabilities pip_i were combined to obtain a subject-level probability (PsubjectP_{\mathrm{subject}}):

Psubject=∑i=1Daipi,(2)P_{\mathrm{subject}} = \sum_{i=1}^{D} a_i p_i, \tag*{(2)}

and this subject-level probability was then used to calculate the subject-level focal loss, based on the original subject-level diagnostic label (Y∈{0,1}Y \in\{0,1\}, with 0 and 1 denoting healthy and diseased subjects, respectively), as shown below:

Lsubject=−αt(1−Pt)γlog⁡(Pt),(3)\mathcal{L}_{\mathrm{subject}} = -\alpha_t(1-P_t)^\gamma\log(P_t), \tag*{(3)}

where Pt=YPsubject+(1−Y)(1−Psubject)P_t = YP_{\mathrm{subject}} + (1-Y)(1-P_{\mathrm{subject}}), αt=αY+(1−α)(1−Y)\alpha_t = \alpha Y +(1-\alpha)(1-Y) is the class weight, and γ\gamma is the focusing parameter. In this work, α\alpha and γ\gamma were empirically set as 0.5 and 2, respectively. This loss was back-propagated to optimize the relabeling classifier (i.e., subject-level classifier in Figure 1(c)) for up to 300 epochs, while the pretrained FEB weights remained frozen during relabeling.

After this subject-level classifier was trained, the proposed relabeling module in Figure 1(c) was subsequently used to predict slice abnormality probabilities (P={pi}, i=1,2,…,DP = \{p_i\},\ i = 1, 2, \ldots, D), and new slice pseudo-labels (Y~={y~i}\widetilde{Y} = \{\widetilde{y}_i\}, where ii is the slice index) were generated correspondingly based on the following Eq. (4), allowing the slice-level retraining and network refinement without manual slice annotations:

y~i={1,pi≥0.5,0,pi<0.5(4)\widetilde{y}_i = \begin{cases} 1, & p_i \ge0.5, \\ 0, & p_i < 0.5 \end{cases} \tag*{(4)}

Then, each pseudo-label paired with its corresponding original QSM slice was used for next-round DBD-Net re-training, and this relabeling module was used only during network training procedures. Overall, this network training and slice relabeling framework was repeated for KK rounds, and in each round, both the proposed DBD-Net (Figure 1(a)) and the subject-level classifier (Figure 1(c)) were reinitialized and trained from scratch.

Posterior Refinement Module and Inference in PoRef-MIL

As shown in Figure 2, during inference, each 3D QSM volume (data size: H×W×DH \times W \times D) was divided into DD consecutive slices, which were fed into the pretrained DBD-Net to obtain the initial slice abnormality probabilities (P={pi}P = \{p_i\}, where i=1,2,…,Di = 1, 2, \ldots, D). Rather than directly assigning a healthy or abnormal status to each slice by thresholding its initial probability, the proposed PRM refined these probabilities by incorporating information from both preceding and following slices.

In this work, each slice was assumed to be associated with two possible states, with 0 and 1 denoting healthy and abnormal, respectively. The slice abnormality probabilities predicted by DBD-Net were used to construct a slice observation evidence matrix (E=[ei(s)]s,iE = [e_i(s)]_{s,i}, where s∈{0,1}s \in\{0,1\} and i=1,2,…,Di = 1, 2, \ldots, D) for these two states:

ei(0)=1−pi,ei(1)=pi,(5)e_i(0) = 1-p_i,\qquad e_i(1) = p_i, \tag*{(5)}

where ei(0)e_i(0) and ei(1)e_i(1) represent the evidence supporting the healthy and abnormal states of slice ii, respectively.

Then, a symmetric 2×22 \times2 transition matrix AA was introduced to model the state dependence between consecutive slices, whose element ArsA_{rs} denoted the probability that the current slice is in state ss given that the preceding slice is in state rr, with ∑s=01Ars=1\sum_{s=0}^{1} A_{rs}=1 for r∈{0,1}r \in\{0,1\}. For example, A00A_{00} and A01A_{01} represent the probabilities that the current slice is healthy or abnormal, respectively, given that the preceding slice is healthy. In this work, the transition matrix was empirically set as

A=[0.90.10.10.9],(6)A = \begin{bmatrix} 0.9 & 0.1 \\ 0.1 & 0.9 \end{bmatrix}, \tag*{(6)}

favoring the same state in consecutive slices while still allowing transitions between healthy and abnormal states.

Next, forward and backward recursions for a two-state hidden Markov model (HMM) with fixed transition probabilities [32] were performed along the entire slice sequence to refine the initial slice abnormality probabilities using information from both directions. The forward quantity αi(s)\alpha_i(s) represents the accumulated support for state ss at slice ii based on the observation evidence from slices 11 to ii, while the backward quantity βi(s)\beta_i(s) denotes the support provided by the subsequent slices, from slice DD to i+1i+1, conditional on slice ii being in state ss. Specifically, αi(s)\alpha_i(s) was calculated as follows:

αi(s)=ei(s)∑r=01αi−1(r)Ars∑u=01ei(u)∑r=01αi−1(r)Aru,(7)\alpha_i(s) = \frac{e_i(s)\sum_{r=0}^{1}\alpha_{i-1}(r)A_{rs}}{\sum_{u=0}^{1}e_i(u)\sum_{r=0}^{1}\alpha_{i-1}(r)A_{ru}}, \tag*{(7)}

where s∈{0,1}s \in\{0,1\}, i=1,2…,Di = 1,2\ldots,D. The numerator part combines evidence propagated from the previous slice with the current slice evidence. The denominator sums these results over the two possible states, ensuring that αi(0)+αi(1)=1\alpha_i(0) + \alpha_i(1) = 1. rr denotes the preceding state, and ss denotes the current state.

Similarly, the backward quantity βi(s)\beta_i(s) was calculated from the end of the slice sequence using the corresponding recursion:

βi(s)=∑t=01Astei+1(t)βi+1(t)∑u=01∑t=01Autei+1(t)βi+1(t),(8)\beta_i(s) = \frac{\sum_{t=0}^{1}A_{st}e_{i+1}(t)\beta_{i+1}(t)}{\sum_{u=0}^{1}\sum_{t=0}^{1}A_{ut}e_{i+1}(t)\beta_{i+1}(t)}, \tag*{(8)}

where i=D−1,D−2,…,1i = D-1,D-2,\ldots,1, and tt denotes the state of the next slice. Similarly, the numerator combines the support propagated from the next slice, while the denominator ensures that βi(0)+βi(1)=1\beta_i(0) + \beta_i(1) = 1.

The forward and backward recursions were initialized as

α0(s)=0.5,βD(s)=0.5,s∈{0,1},(9)\alpha_0(s) = 0.5,\quad\beta_D(s) = 0.5,\quad s \in\{0,1\}, \tag*{(9)}

where α0(s)\alpha_0(s) specifies equal initial support for the two possible states (healthy or abnormal) before the first slice. Since no slices follow slice DD, the backward boundary βD(s)\beta_D(s) assigns equal weights to both states, representing the absence of further observation evidence.

After the bidirectional recursions, the refined abnormality probability qiq_i of slice ii was calculated by combining the forward and backward support:

qi=αi(1)βi(1)∑u=01αi(u)βi(u),(10)q_i = \frac{\alpha_i(1)\beta_i(1)}{\sum_{u=0}^{1}\alpha_i(u)\beta_i(u)}, \tag*{(10)}

where 1−qi1-q_i denotes the corresponding probability of the healthy state; the numerator combines the support for the abnormal state from both directions, while the denominator normalizes the combined support over the healthy and abnormal states. Therefore, qiq_i incorporates observation evidence from the entire slice sequence through the state transitions specified by AA.

Following posterior probability refinement, a binary Shannon entropy HiH_i and the corresponding confidence CiC_i were also calculated for each slice:

{Hi=−qilog⁡2(qi)−(1−qi)log⁡2(1−qi),Ci=1−Hi(11)\begin{cases} H_i = -q_i\log_2(q_i) - (1-q_i)\log_2(1-q_i),\\ C_i = 1-H_i \tag*{(11)} \end{cases}

where Hi,Ci∈[0,1]H_i,C_i \in[0,1], with 0log⁡200\log_2 0 defined as 00. When qiq_i approaches 0.50.5, the two states have similar probabilities, resulting in high entropy and low confidence. Conversely, probabilities approaching either 00 or 11 will yield low entropy and high confidence. Overall, qiq_i describes the predicted abnormality of a slice, whereas CiC_i describes the confidence of this binary prediction.

Finally, to obtain a subject-level score, a sliding window of length LL (empirically set to 10 in this work) was traversed along the whole probability sequence with a stride of 1. For a window starting at slice jj, the refined probabilities were averaged using the corresponding confidence values as weights:

Ssubject=max⁡j∈JWj=max⁡j∈J∑i=jj+L−1Ciqi∑i=jj+L−1Ci,(12)S_{\mathrm{subject}}=\max_{j\in\mathcal{J}}W_j=\max_{j\in\mathcal{J}}\frac{\sum_{i=j}^{j+L-1}C_iq_i}{\sum_{i=j}^{j+L-1}C_i}, \tag*{(12)}

where J={1,…,D−L+1}\mathcal{J}=\{1,\ldots,D-L+1\} denotes the set of available window starting positions. The normalization by the confidence sum allowed slices with higher confidence to contribute more to the decision. The final subject-level classification (Y^subject\hat{Y}_{subject}) was obtained by the following simple thresholding:

Y^subject={1(Diseased),Ssubject>0.50(Healthy),Ssubject≤0.5(13)\hat{Y}_{subject}= \begin{cases} 1 & \text{(Diseased)},\quad S_{\mathrm{subject}}>0.5\\ 0 & \text{(Healthy)},\quad S_{\mathrm{subject}}\leq0.5 \end{cases} \tag*{(13)}

Experiments

Dataset Acquisition and Partitioning

The proposed PoRef-MIL was evaluated separately on an in-house PD dataset and a publicly available MS dataset Fiscone et al. (2024). The PD dataset was collected at the Third Xiangya Hospital, Central South University, and included a total of 287 participants, comprising 156 patients with PD and 131 healthy controls (HCs). The public MS dataset included 150 participants, comprising 100 patients with MS and 50 HCs. The demographic characteristics of both datasets are summarized in Table 1. These datasets were used for two separate binary classification tasks, distinguishing patients with PD or MS from the corresponding HCs (i.e., HC vs. PD and HC vs. MS, respectively).

Institutional ethics board approvals have been obtained for the in-house MRI brain data used in this work. For the PD dataset, all QSM scans were acquired on a Philips 3T scanner (Ingenia Elition X; Philips Medical Systems, Best, the Netherlands) with a 32-channel receiver array head coil and the following sequence and parameters: 3D multi-echo GRE with 10 unipolar echoes, FOV =224×224×128 mm3=224\times224\times128\ \mathrm{mm}^3, flip angle =20∘=20^\circ, BW =855=855 Hz/pixel, TE1_1 / Δ\Delta TE / TR =4.2=4.2 / 4.1 / 43.4 ms, spatial resolution =0.8=0.8 mm ×\times 0.8 mm ×\times 1 mm, CS-SENSE acceleration factor =3=3, and scan time =4.6=4.6 min. The recently developed single-step iQSM+ method Gao et al. [13] was used for QSM reconstruction from phase images, after brain mask segmentation using BET Smith [36] based on the magnitude images. For each subject, the whole-brain mean susceptibility was subtracted from the reconstructed QSM map, setting the mean susceptibility within the brain to zero. In addition, sagittal T1-weighted (T1w) structural images were also acquired using a three-dimensional turbo field echo (3D-TFE) sequence with the following parameters: TR/TE =8.3/3.8=8.3/3.8 ms, flip angle =8∘=8^\circ, turbo factor =240=240, FOV =240×240×170 mm3=240\times240\times170\ \mathrm{mm}^3, voxel size =0.94=0.94 mm ×\times 0.94 mm ×\times 1 mm, acquisition matrix =256×256×170=256\times256\times170, compressed sensing acceleration factor =3=3, and acquisition time =3=3 min 5 s. These T1w images were used together with QSM images in the multimodal classification experiments for the HC vs. PD task.

The public MS dataset also included T1w and fluid-attenuated inversion recovery (FLAIR) images in addition to the QSM images. Details of the acquisition protocols are described in the original publication Fiscone et al. [12]. QSM images were used for the main classification experiments, while T1w and FLAIR images were combined with QSM in the additional multimodal MS classification experiments.

For each dataset, a fixed test set was held out before model training. The PD test set included 31 patients with PD and 26 HCs, while the MS test set included 20 patients with MS and 10 HCs. The remaining participants formed the development set for each dataset. All partitions were defined at the subject level, with all slices from the same subject assigned to the same subset.

Five-fold cross-validation was performed within each development set during network training. In each fold, four subsets were used for training and the remaining subset was used for validation and model selection (early stopping). The complete iterative training procedure was performed separately for each fold, yielding five models for each dataset. All five models were then evaluated individually on the fixed test set, which was excluded from model training and selection.

Network Training Details

PoRef-MIL was implemented in PyTorch 2.9.1 and trained on an NVIDIA RTX A6000 GPU. The iterative training procedure described in § 2.2 was performed. Specifically, DBD-Net was first trained using the subject-level diagnostic labels assigned to individual slices in the initial warm-up training, with a batch size of 8 subjects (using all slices of each subject). Then, slice relabeling was conducted and a subject-level classifier was trained using one complete subject per batch, comprising 128 axial slices for the HC vs. PD task and 96 axial slices for the HC vs. MS task. After subject-level classifier training, new slice pseudo-labels were generated, enabling the retraining of the slice-level DBD-Net with the original QSM slices and these updated labels. These two procedures alternated with each other for K=5K=5 rounds in this work. Both the slice-level DBD-Net and the subject-level classifier (during the relabeling procedure) were optimized using the Adam optimizer with a weight decay of 1×10−51\times10^{-5} and binary focal loss

Note: The metrics are presented as mean ±\pm standard deviation across five models trained using five-fold cross-validation within the development sets. All five models were evaluated on the same fixed test set for each dataset. Bold and underlined values indicate the highest and second-highest means, respectively. tt-tests were conducted to compare each method with the proposed PoRef-MIL, with * and ** indicating P<0.05P < 0.05 and P<0.01P < 0.01, respectively.

[26], with α=0.5\alpha= 0.5 and γ=2\gamma= 2. The initial learning rates for the five specific training rounds were set to 1×10−41 \times10^{-4}, 5×10−55 \times10^{-5}, 3×10−53 \times10^{-5}, 1×10−51 \times10^{-5}, and 5×10−65 \times10^{-6}, respectively. Each training stage was conducted for a maximum of 300 epochs, with early stopping applied. The ReduceL-

ROnPlateau scheduler was used to monitor subject-level validation performance using the area under the receiver operating characteristic curve (AUC), with a learning rate reduction factor of 0.5, a patience of 10 epochs, and a minimum learning rate of 1×10−71 \times10^{-7}. A new checkpoint was saved when the validation AUC exceeded the previously recorded best value by more than 1×10−41 \times10^{-4}. Training stopped after 50 consecutive epochs without such an improvement. The complete training of all five models took approximately 24 hours in total, including both DBD-Net training and the intervening relabeling-classifier training.

Performance Evaluation and Metrics

The proposed PoRef-MIL was compared with four subject-level classification methods, including an attention-gated squeeze-and-excitation ResNeXt50 model used for PD diagnosis (AG-SE-ResNeXt50) [45], CAAE+CRAT [7], a multi-template meta-information regularized network (MMRN) [17], a dense Transformer foundation model with a mixture of experts (DenseFormer-MoE) [9], and four MIL methods, i.e., dual attention multi-instance deep learning (DA-MIDL) [55], AttriMIL [3], PMIL [50], and Transformer-based correlated multiple instance learning (TransMIL) [35], using AUC, accuracy (ACC), sensitivity (SEN), specificity (SPE), and F1 score (with patients treated as the positive class and HCs as the negative class). Each metric was calculated separately for the five trained models of each task, and the results were reported as mean ±\pm standard deviation (SD). Receiver operating characteristic (ROC) curves were also generated by pooling the subject-level prediction scores and corresponding diagnostic labels from the five models evaluated on the same fixed test set for each task.

Ablation experiments were also conducted to evaluate the contribution of the proposed PRM based on both the in-house PD and public MS datasets. The PRM was also applied to DA-MIDL, AttriMIL, PMIL, and TransMIL to evaluate its plug-and-play capability and its contribution to classification performance across different models. Furthermore, the effectiveness of the iterative relabeling strategy was examined by varying the iterative training rounds from K=0K = 0 to K=5K = 5, with K=0K = 0 representing a model trained without the slice relabeling.

All PoRef-MIL results in this work were obtained using models trained on axial slices, except for the additional orthogonal qualitative heatmaps generated using two models trained separately on coronal slices from the PD cohort and sagittal slices from the MS cohort. Grad-CAM heatmaps [34] were illustrated for representative PD and MS subjects, together with slice abnormality probability distributions, the slice states after PRM, and the entropy-derived confidence curves. The numbers of model-predicted abnormal slices were also compared between the HC and patient groups in both datasets using t-tests.

Additional experiments were also conducted to evaluate the applicability of PoRef-MIL to different MRI modalities. For the PD task, two modalities (T1w and QSM) were evaluated, while for the MS task, T1w, QSM, and FLAIR images were evaluated. Finally, the influence of incorporating age information on classification performance was investigated by comparing models with and without feature-wise linear modulation (FiLM)-based age conditioning [30].

Results

Performance Comparisons with State-of-the-Art Methods

Table 2 reports the classification performance comparisons of the proposed PoRef-MIL with several state-of-the-art methods on the in-house PD and public MS test datasets. Overall, PoRef-MIL achieved the highest mean AUC, ACC, SPE, and F1 score on both classification tasks. Specifically, on the HC vs. PD dataset, PoRef-MIL achieved the highest mean AUC of 0.942, ACC of 0.926, and F1 score of 0.932, which were 0.015, 0.028, and 0.027 significantly (P<0.01P < 0.01) higher than those of TransMIL (the second-best results), respectively. In addition, PoRef-MIL achieved the highest mean AUC of 0.872±0.0340.872 \pm0.034, ACC of 0.833±0.0530.833 \pm0.053, and F1 score of 0.871±0.0420.871 \pm0.042 among all compared methods for the HC vs. MS task, exceeding those of TransMIL by 0.021, 0.020, and 0.015, respectively.

MethodAUCACCSENSPEF1 Score
PD dataset
AG-SE-ResNeXt50 (Wang et al., [45])0.893 ±\pm 0.002{sup}**`0.846 ±\pm 0.015{sup}**`0.858 ±\pm 0.018{sup}**`0.831 ±\pm 0.021{sup}**`0.858 ±\pm 0.014{sup}**`
MMRN (Han et al., [17])0.899 ±\pm 0.002{sup}**`0.860 ±\pm 0.012{sup}**`0.832 ±\pm 0.014{sup}**`0.892 ±\pm 0.032{sup}*`0.866 ±\pm 0.010{sup}**`
CAAE + CRAT (Cobbinah et al., [7])0.903 ±\pm 0.003{sup}**`0.870 ±\pm 0.020{sup}**`0.858 ±\pm 0.018{sup}**`0.885 ±\pm 0.027{sup}*`0.878 ±\pm 0.019{sup}**`
DenseFormer-MoE (Ding et al., [9])0.906 ±\pm 0.001{sup}**`0.856 ±\pm 0.015{sup}**`0.813 ±\pm 0.014{sup}**`0.908 ±\pm 0.0210.860 ±\pm 0.014{sup}**`
PMIL (Yan et al., [50])0.890 ±\pm 0.013{sup}**`0.853 ±\pm 0.010{sup}**`0.865 ±\pm 0.014{sup}**`0.838 ±\pm 0.017{sup}**`0.865 ±\pm 0.009{sup}**`
DA-MIDL (Zhu et al., [55])0.914 ±\pm 0.002{sup}**`0.867 ±\pm 0.010{sup}**`0.865 ±\pm 0.014{sup}**`0.869 ±\pm 0.021{sup}**`0.876 ±\pm 0.009{sup}**`
AttriMIL (Cai et al., [3])0.912 ±\pm 0.002{sup}**`0.874 ±\pm 0.008{sup}**`0.884 ±\pm 0.018{sup}**`0.862 ±\pm 0.021{sup}**`0.884 ±\pm 0.007{sup}**`
TransMIL (Shao et al., [35])0.927 ±\pm 0.001{sup}**`0.898 ±\pm 0.008{sup}**`0.890 ±\pm 0.018{sup}*`0.908 ±\pm 0.0210.905 ±\pm 0.008{sup}**`
PoRef-MIL (proposed)0.942 ±\pm 0.0020.926 ±\pm 0.0080.923 ±\pm 0.0180.931 ±\pm 0.0170.932 ±\pm 0.008
MS dataset (Fiscone et al., [12])
AG-SE-ResNeXt50 (Wang et al., [45])0.792 ±\pm 0.051{sup}*`0.753 ±\pm 0.0610.780 ±\pm 0.0570.700 ±\pm 0.0710.808 ±\pm 0.049
MMRN (Han et al., [17])0.838 ±\pm 0.0290.800 ±\pm 0.0530.820 ±\pm 0.0570.760 ±\pm 0.0550.845 ±\pm 0.043
CAAE + CRAT (Cobbinah et al., [7])0.812 ±\pm 0.029{sup}*`0.767 ±\pm 0.033{sup}*`0.780 ±\pm 0.027{sup}*`0.740 ±\pm 0.0550.817 ±\pm 0.026{sup}*`
DenseFormer-MoE (Ding et al., [9])0.783 ±\pm 0.034{sup}**`0.740 ±\pm 0.049{sup}*`0.760 ±\pm 0.042{sup}*`0.700 ±\pm 0.0710.796 ±\pm 0.040{sup}*`
PMIL (Yan et al., [50])0.790 ±\pm 0.021{sup}**`0.740 ±\pm 0.028{sup}**`0.830 ±\pm 0.0270.560 ±\pm 0.055{sup}**`0.810 ±\pm 0.021{sup}*`
DA-MIDL (Zhu et al., [55])0.802 ±\pm 0.033{sup}*`0.773 ±\pm 0.0430.790 ±\pm 0.0420.740 ±\pm 0.0550.823 ±\pm 0.035
AttriMIL (Cai et al., [3])0.820 ±\pm 0.010{sup}*`0.780 ±\pm 0.0180.870 ±\pm 0.0270.600 ±\pm 0.000{sup}**`0.840 ±\pm 0.015
TransMIL (Shao et al., [35])0.851 ±\pm 0.0410.813 ±\pm 0.0560.830 ±\pm 0.0450.780 ±\pm 0.0840.856 ±\pm 0.043
PoRef-MIL (proposed)0.872 ±\pm 0.0340.833 ±\pm 0.0530.850 ±\pm 0.0500.800 ±\pm 0.0710.871 ±\pm 0.042

Table 2. Comparison of the Proposed PoRef-MIL with Eight State-of-the-Art Methods on PD and MS Datasets

Figure 3 compares the ROC curves of PoRef-MIL with AG-SE-ResNeXt50, CAAE + CRAT, MMRN, DenseFormer-MoE, DA-MIDL, AttriMIL, PMIL, and TransMIL. PoRef-MIL generally showed higher true positive rates in the low false positive rate region for both classification tasks. In addition, the confusion matrices for the proposed PoRef-MIL are also reported, with subject counts presented as mean ±\pm SD across the five trained models. Overall, PoRef-MIL correctly classified on average 28.6 of the 31 patients with PD and 24.2 of the 26 HCs in the PD test set, and 17.0 of the 20 patients with MS and 8.0 of the 10 HCs in the MS test set.

ROC curves for PD and MS classification with confusion matrices

Figure 3. Comparison of receiver operating characteristic (ROC) curves of the proposed PoRef-MIL with the other methods on PD (a) and MS (b) test sets, with corresponding confusion matrices of PoRef-MIL results (mean ±\pm standard deviation across the five trained models) shown on the right.

KK: number of slice relabeling rounds; K=0K = 0 denotes models trained using slice-level loss without slice relabeling.

a^{a} Conventional MIL models trained end-to-end using subject-level loss; KK is not applicable.

Ablation Studies of the Posterior Refinement Module and Iterative Relabeling

Table 3 compares the classification performance of the complete PoRef-MIL with two prediction configurations without PRM on both datasets. On the PD dataset, incorporating the PRM achieved the best mean AUC, ACC, and F1 score of 0.942, 0.926, and 0.932, respectively, while the corresponding values were 0.872, 0.833, and 0.871 on the MS dataset.

Model ConfigurationsPD datasetMS dataset (Fiscone et al., 2024)
AUCACCF1 ScoreAUCACCF1 Score
AttriMIL w/o PRM0.912 ±\pm 0.0020.874 ±\pm 0.0080.884 ±\pm 0.0070.820 ±\pm 0.0100.780 ±\pm 0.0180.840 ±\pm 0.015
AttriMIL + PRM0.920 ±\pm 0.0030.888 ±\pm 0.0100.898 ±\pm 0.0080.844 ±\pm 0.0120.807 ±\pm 0.0280.859 ±\pm 0.021
DA-MIDL w/o PRM0.914 ±\pm 0.0020.867 ±\pm 0.0100.876 ±\pm 0.0090.802 ±\pm 0.0330.773 ±\pm 0.0430.823 ±\pm 0.035
DA-MIDL + PRM0.924 ±\pm 0.0030.891 ±\pm 0.0150.899 ±\pm 0.0140.834 ±\pm 0.0150.807 ±\pm 0.0280.850 ±\pm 0.022
PMIL w/o PRM0.890 ±\pm 0.0130.853 ±\pm 0.0100.865 ±\pm 0.0090.790 ±\pm 0.0210.740 ±\pm 0.0280.810 ±\pm 0.021
PMIL + PRM0.912 ±\pm 0.0030.884 ±\pm 0.0160.893 ±\pm 0.0150.830 ±\pm 0.0130.813 ±\pm 0.0300.857 ±\pm 0.023
TransMIL w/o PRM0.927 ±\pm 0.0010.898 ±\pm 0.0080.905 ±\pm 0.0080.851 ±\pm 0.0410.813 ±\pm 0.0560.856 ±\pm 0.043
TransMIL + PRM0.934 ±\pm 0.0050.909 ±\pm 0.0150.916 ±\pm 0.0130.868 ±\pm 0.0290.827 ±\pm 0.0430.868 ±\pm 0.034
PoRef-MIL w/o PRM{sup}1`0.925 ±\pm 0.0050.898 ±\pm 0.0150.906 ±\pm 0.0140.851 ±\pm 0.0330.800 ±\pm 0.0330.845 ±\pm 0.029
PoRef-MIL w/o PRM{sup}2`0.938 ±\pm 0.0060.909 ±\pm 0.0150.918 ±\pm 0.0130.865 ±\pm 0.0320.820 ±\pm 0.0380.863 ±\pm 0.030
PoRef-MIL (proposed)0.942 ±\pm 0.0020.926 ±\pm 0.0080.932 ±\pm 0.0080.872 ±\pm 0.0340.833 ±\pm 0.0530.871 ±\pm 0.042

Table 3. Plug-and-play Effectiveness of the Proposed Posterior Refinement Module (PRM) Across the Five MIL Models

In addition, the plug-and-play feature of PRM was also validated using the other four MIL models (keeping the pretrained network backbones frozen). For example, as shown in Table 3, the TransMIL model combined with the plug-and-play PRM block achieved average increases of 0.012 in AUC, 0.0125 in ACC, and 0.0115 in F1 score across the two datasets compared with the original TransMIL model.

The influences of iterative relabeling and training rounds (i.e., KK) are investigated in Table 4. The mean AUC, ACC, and F1 score consistently increased with more relabeling and retraining rounds (from K=1K = 1 to K=5K = 5, where K=0K = 0 denotes a model trained without the slice relabeling) on both datasets. Overall, all models trained with iterative relabeling (K≥1K \ge1) demonstrated dramatically better AUC and ACC results than the initial model (K=0K = 0). The first relabeling round increased the mean AUC from 0.667 to 0.913 on the PD dataset. As KK increased from 1 to 5, AUC, ACC, and F1 score gradually increased from 0.913, 0.877, and 0.888 to 0.942, 0.926, and

KKAUCACCF1 Score
PD dataset
aa0.912 ± 0.0020.895 ± 0.0120.903 ± 0.012
00.667 ± 0.0270.639 ± 0.0160.673 ± 0.012
10.913 ± 0.0060.877 ± 0.0180.888 ± 0.015
20.924 ± 0.0050.898 ± 0.0150.906 ± 0.014
30.932 ± 0.0050.909 ± 0.0150.916 ± 0.013
40.938 ± 0.0040.916 ± 0.0080.922 ± 0.008
50.942 ± 0.0020.926 ± 0.0080.932 ± 0.008
MS dataset
aa0.838 ± 0.0030.793 ± 0.0150.839 ± 0.011
00.623 ± 0.0330.713 ± 0.0510.816 ± 0.031
10.810 ± 0.0320.760 ± 0.0490.814 ± 0.039
20.831 ± 0.0320.787 ± 0.0450.835 ± 0.036
30.854 ± 0.0330.813 ± 0.0450.857 ± 0.035
40.866 ± 0.0320.827 ± 0.0430.867 ± 0.035
50.872 ± 0.0340.833 ± 0.0530.871 ± 0.042

Table 4. Classification Performance with Conventional MIL and Different Iterative Slice Relabeling Rounds

Bold and underlined values indicate the highest and second-highest means, respectively.

0.9320.932, respectively, on the HC vs. PD dataset. In addition, models trained with different numbers of relabeling rounds were compared with a conventional end-to-end MIL model trained using subject-level loss without iterative slice relabeling. PoRef-MIL with K≥3K \ge3 achieved higher mean AUC, ACC, and F1 scores than conventional MIL on both datasets.

Slice-Level Predictions and Model Interpretability

Figure 4 illustrates the Grad-CAM heatmaps and slice-level prediction distributions for two representative PD and MS subjects. For the PD subject, stronger contributions were observed in deep gray matter regions on axial slices 58 and 70 (zoomed-in regions in Figure 4(a)) and coronal slice 119 in Figure 4(e). For the MS subject, several regions with stronger responses overlapped with the lesions visible in the QSM images, as indicated by the yellow arrows in the enlarged images of axial slices 67 and 70 (Figure 4(c)) and sagittal slice 143 in Figure 4(e).

Grad-CAM maps and slice-level predictions for representative PD and MS subjects

Figure 4. Grad-CAM maps and slice-level predictions for representative PD (a, b) and MS (c, d) subjects. (a) and (c) show Grad-CAM maps overlaid on the original QSM slices, with warmer colors indicating stronger contributions. (b) and (d) show the initial slice-level abnormality probability distributions, with red and blue dots indicating slices predicted as abnormal and healthy, respectively. The red and blue dashed step lines represent the corresponding slice states after PRM, with shaded intervals marking slices 45 to 77 in the PD subject and slices 63 to 76 in the MS subject. Purple dashed curves indicate entropy-derived confidence calculated from the refined slice-level probabilities. (e) shows additional coronal and sagittal Grad-CAM heatmaps for a PD subject and an MS subject, respectively, generated using separately trained models.

The initial slice abnormality probabilities (red and blue dots) predicted by the DBD-Net, slice states after PRM (red and blue dashed step lines), and entropy-derived confidence (purple lines) are also plotted in Figure 4(b, Figure 4d). After the PRM, slices 45 to 77 in the PD subject and slices 63 to 76 in the MS subject were classified as abnormal, while the remaining slices were classified as healthy. It is also noticed that some slices outside these intervals, although assigned high initial abnormality probabilities by the pretrained network, were still assigned healthy states after refinement. The confidence derived from the refined posterior probabilities also varied across the slice sequence. Overall, high confidence was observed for both predicted healthy and abnormal slices, with local decreases around several transitions between the two states.

Figure 5 compares the numbers of model-predicted abnormal slices between the HC and patient groups in both test sets. In both datasets, the patient groups showed significantly more predicted abnormal slices than the corresponding HC groups (P<0.001P < 0.001). Patients with PD had 31.68 ± 3.17 predicted abnormal slices per subject, compared with 9.46 ± 3.69 in the HC group. For the MS dataset, the corresponding numbers were 17.85 ± 3.83 in patients with MS and 6.70 ± 2.65 in HCs. In both datasets, the mean proportion of predicted abnormal slices was below 25% in both the HC and patient groups, indicating that most slices were classified as healthy.

Figure 5. Comparison of the numbers of model-predicted abnormal slices between healthy controls (HCs) and patients in the (a) PD and (b) MS test sets. Percentages indicate the mean proportion of predicted abnormal slices among all analyzed slices per subject (128 slices for PD and 96 slices for MS).

Multimodal Classification and Age Conditioning

The classification performance of PoRef-MIL using single- and multi-modal MRI images is reported in Table 5. In the PD dataset, combining T1w and QSM images achieved the highest mean AUC, ACC, and F1 score of 0.964, 0.944, and 0.949, respectively, compared with 0.942, 0.926, and 0.932 using QSM alone. For the HC vs. MS task, T1w, QSM, and FLAIR images together also achieved the best mean AUC, ACC, and F1 score of 0.887, 0.860, and 0.894, respectively.

InputsAUCACCF1 Score
PD dataset
T1w0.908 ± 0.0120.898 ± 0.0150.906 ± 0.014
QSM0.942 ± 0.0020.926 ± 0.0080.932 ± 0.008
T1w+QSM0.964 ± 0.0040.944 ± 0.0150.949 ± 0.013
MS dataset
T1w0.844 ± 0.0300.800 ± 0.0530.848 ± 0.040
QSM0.872 ± 0.0340.833 ± 0.0530.871 ± 0.042
FLAIR0.877 ± 0.0280.840 ± 0.0490.877 ± 0.038
T1w+QSM+FLAIR0.887 ± 0.0280.860 ± 0.0430.894 ± 0.032

Table 5. Single-modality and multimodal classification with PoRef-MIL

Given the higher mean age of HCs than that of patients in the MS cohort, FiLM-based age conditioning was also evaluated, with the results reported in Table 6. Compared with the reference model using QSM images without additional age input, incorporating age information resulted in slightly lower mean AUC, ACC, and F1 scores on both datasets. For example, the mean AUC decreased from 0.942 to 0.931 on the PD dataset and from 0.872 to 0.860 on the MS dataset.

Age InputAUCACCF1 Score
PD dataset
w/ FiLM0.931 ± 0.0060.916 ± 0.0080.921 ± 0.008
w/o Age0.942 ± 0.0020.926 ± 0.0080.932 ± 0.008
MS dataset
w/ FiLM0.860 ± 0.0320.813 ± 0.0610.854 ± 0.049
w/o Age0.872 ± 0.0340.833 ± 0.0530.871 ± 0.042

Table 6. Evaluation of FiLM-based Age Conditioning in PoRef-MIL.

Discussion and Conclusion

In this work, we proposed PoRef-MIL, a weakly supervised framework for QSM-based brain disease classification using only subject-level diagnostic labels. The framework combines iterative slice relabeling during network training with a plug-and-play posterior refinement module (PRM) during inference. Comprehensive experiments were conducted on an in-house PD dataset and a publicly available MS dataset (Fiscone et al., 2024) to evaluate the proposed framework, including comparisons with several state-of-the-art subject-level classification and MIL methods. The results suggest that PoRef-MIL is a promising approach to disease classification in relatively small QSM cohorts without requiring manual slice-level annotations.

PoRef-MIL reformulates each 3D QSM volume as a sequence of consecutive 2D slices, allowing individual slices within the same subject to receive different predictions. The iterative relabeling strategy updates the slice-level supervision during network training, while the PRM incorporates information across the ordered slice sequence to refine predictions during inference. The ablation studies supported the contributions of both iterative relabeling and the PRM to the final classification performance. Both cohorts used in this work contained fewer than 300 participants, and the public MS dataset Fiscone et al., 2024 included only 150 participants. The performance obtained in these cohorts suggests that the proposed framework can learn disease-related imaging patterns from relatively small QSM datasets without requiring manual slice annotations.

According to the results in Figure 5, slices predicted by PoRef-MIL as healthy predominated in both the HC and patient groups, although patients had significantly more slices predicted as abnormal than HCs. This distribution is consistent with the motivation for allowing different slices within the same subject to have different training labels. When the diagnostic label of a patient is directly assigned to every slice, slices containing little disease-related information also receive the disease label, which might be problematic. By generating slice pseudo-labels, the iterative relabeling procedure may reduce the mismatch between inherited diagnostic labels and local image patterns. The lower AUC and ACC obtained without relabeling (K=0K = 0), together with the improvements after introducing relabeling, also support the value of updating slice-level supervision during network training.

Previous MIL studies have tried to improve classification performance through different strategies, including learned attention pooling (Ilse et al., [18]), clustering-constrained feature learning (Lu et al., 2021), modeling of inter-instance correlations (Shao et al., [35]), and instance attribute constraints (Cai et al., [3]). These developments improve how image information is represented and combined within the trained model. However, predictions from a trained network may still benefit from additional refinement before determining the final subject-level diagnosis, particularly when training cohorts are small and local annotations are unavailable. In this work, the network outputs were therefore treated as initial estimates of slice abnormality, and the PRM incorporates evidence from preceding and following slices to refine these estimates, while entropy-derived confidence determines their contributions to subsequent subject-level scoring. This additional inference procedure complements network training by using the information from the entire slice sequence to reassess local predictions before producing the final classification.

The PRM also has a plug-and-play property because it operates on ordered slice-level probabilities and does not depend on the internal feature representation of DBD-Net. For classifiers that already provide these probabilities, the PRM can be directly applied without further optimization of the network weights. For methods that only provide subject-level predictions, a slice-level classification head can be trained while keeping the pretrained backbone fixed before applying the PRM. In this work, the PRM was applied to DA-MIDL (Zhu et al., [55]), AttriMIL (Cai et al., [3]), PMIL (Liu et al., 2024), and TransMIL (Shao et al., [35]), with improvements in classification performance across these models.

In addition, PoRef-MIL provides slice abnormality probabilities, refined slice states, and entropy-derived confidence curves alongside the subject-level prediction. These outputs allow the distribution of model-predicted abnormalities and the associated confidence to be examined across the whole volume. In the representative MS examples, regions with stronger Grad-CAM responses overlapped well with lesions visible in the QSM images, providing qualitative support for the correspondence between model responses and local imaging abnormalities. The multimodal experiments further showed that complementary MRI contrasts could be incorporated through input-channel concatenation while retaining the same overall training and inference strategy.

However, there are still several limitations in this work. First, the alternating training of DBD-Net and the relabeling classifier increases the overall training time, while the number of relabeling rounds was also selected empirically, and K=5K = 5 may not be optimal for other datasets. Second, the age difference between HCs and patients in the MS cohort remains a potential source of confounding. Although FiLM-based age conditioning was evaluated, this experiment does not establish that the classification results were totally independent of age-related imaging differences. Third, the lack of slice-level reference annotations prevented direct assessment of the accuracy of the generated pseudo-labels and refined slice predictions. Finally, the evaluation was limited to two relatively small cohorts. Future studies involving different populations and acquisition settings are needed to better evaluate the generalizability of the proposed framework.

Data and Code Availability Statements

Data are available on request due to privacy/ethical restrictions. Source codes and trained networks are available at: https://github.com/YangGaoUQ/PoRef-MIL-QSM.

Acknowledgments

YG acknowledges that this work was supported by the National Natural Science Foundation of China under Grant No. 62301616, and the Natural Science Foundation of Hunan under Grant No. 2024JJ6530. HS acknowledges support from the Australian Research Council (DE210101297). GBP acknowledges support from Canadian Institutes of Health Research (FDN-143290).

References

References

  1. [1]Acosta-Cabronero, J., Cardenas-Blanco, A., Betts, M.J., Butryn, M., Valdes-Herrera, J.P., Galazky, I., Nestor, P.J., 2017. The whole-brain pattern of magnetic susceptibility perturbations in Parkinson’s disease. Brain 140, 118–131. doi:10.1093/brain/aww278.
  2. [2]Ayton, S., Fazlollahi, A., Bourgeat, P., Raniga, P., Ng, A., Lim, Y.Y., Diouf, I., Farquharson, S., Fripp, J., Ames, D., et al., 2017. Cerebral quantitative susceptibility mapping predicts amyloid-β-related cognitive decline. Brain 140, 2112–2119. doi:10.1093/brain/awx137.DOI
  3. [3]Cai, L., Huang, S., Zhang, Y., Lu, J., Zhang, Y., 2025. AttriMIL: Revisiting attention-based multiple instance learning for whole-slide pathological image classification from a perspective of instance attributes. Medical Image Analysis 103, 103631. doi:10.1016/j.media.2025.103631.DOI
  4. [4]Carbonneau, M.A., Cheplygina, V., Granger, E., Gagnon, G., 2018. Multiple instance learning: A survey of problem characteristics and applications. Pattern Recognition 77, 329–353. doi:10.1016/j.patcog.2017.10.009.DOI
  5. [5]Chen, L., Soldan, A., Oishi, K., Faria, A., Zhu, Y., Albert, M., van Zijl, P.C., Li, X., 2021. Quantitative susceptibility mapping of brain iron and β-amyloid in mri and pet relating to cognitive performance in cognitively normal older adults. Radiology 298, 353–362. doi:10.1148/radiol.2020201603.DOI
  6. [6]Cheplygina, V., de Bruijne, M., Pluim, J.P.W., 2019. Not-so-supervised: A survey of semi-supervised, multi-instance, and transfer learning in medical image analysis. Medical Image Analysis 54, 280–296. doi:10.1016/j.media.2019.03.009.DOI
  7. [7]Cobbinah, B.M., Sorg, C., Yang, Q., Ternblom, A., Zheng, C., Han, W., Che, L., Shao, J., 2022. Reducing variations in multi-center Alzheimer’s disease classification with convolutional adversarial autoencoder. Medical Image Analysis 82, 102585. doi:10.1016/j.media.2022.102585.DOI
  8. [8]Deh, K., Ponath, G.D., Molvi, Z., Parel, G.C.T., Gillen, K.M., Zhang, S., Nguyen, T.D., Spincemaille, P., Ma, Y., Gupta, A., Gauthier, S.A., Pitt, D., Wang, Y., 2018. Magnetic susceptibility increases as diamagnetic molecules breakdown: Myelin digestion during multiple sclerosis lesion formation contributes to increase on QSM. Journal of Magnetic Resonance Imaging 48, 1281–1287. doi:10.1002/jmri.25997.DOI
  9. [9]Ding, R., Lu, H., Liu, M., 2025. DenseFormer-MoE: A Dense Transformer Foundation Model with Mixture of Experts for Multi-Task Brain Image Analysis. IEEE Transactions on Medical Imaging 44, 4037–4048. doi:10.1109/TMI.2025.3551514.DOI
  10. [10]Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N., 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, in: International Conference on Learning Representations.
  11. [11]Elkady, A.M., Elliott, C., Fetco, D., Araujo, D., Karimaghloo, Z., Ganzetti, M., Clayton, D., Craveiro, L., Kazlauskaite, A., Narayanan, S., Arnold, D.L., Rudko, D.A., 2025. Longitudinal Multiparametric Quantitative MRI Evaluation of Acute and Chronic Multiple Sclerosis Paramagnetic Rim Lesions. Journal of Magnetic Resonance Imaging 61, 1812–1828. doi:10.1002/jmri.29583.DOI
  12. [12]Fiscon, C., Signoril, G., Manners, D.N., Motta, L., Venturi, G., Panzera, I., Zaccagna, F., Rundo, L., Lugaresi, A., Lodi, R., Tonon, C., Castelli, M., 2024. Multiparametric MRI dataset for susceptibility-based radiomic feature extraction and analysis. Scientific Data 11, 575. doi:10.1038/s41597-024-03418-6.
  13. [13]Gao, Y., Xiong, Z., Shan, S., Liu, Y., Rong, P., Li, M., Wilman, A.H., Pike, G.B., Liu, F., Sun, H., 2024. Plug-and-play latent feature editing for orientation-adaptive quantitative susceptibility mapping neural networks. Medical Image Analysis 94, 103160. doi:10.1016/j.media.2024.103160.DOI
  14. [14]Guan, H., Wang, C., Tao, D., 2021. MRI-based Alzheimer’s disease prediction via distilling the knowledge in multi-modal data. NeuroImage 244, 118586. doi:10.1016/j.neuroimage.2021.118586.DOI
  15. [15]Guan, X., Guo, T., Zhou, C., Wu, J., Zeng, Q., Li, K., Luo, X., Bai, X., Wu, H., Gao, T., et al., 2022. Altered brain iron depositions from aging to Parkinson’s disease and Alzheimer’s disease: A quantitative susceptibility mapping study. NeuroImage 264, 119683. doi:10.1016/j.neuroimage.2022.119683.DOI
  16. [16]Guan, X., Lancione, M., Ayton, S., Dusek, P., Langkammer, C., Zhang, M., 2024. Neuroimaging of Parkinson’s disease by quantitative susceptibility mapping. NeuroImage 289, 120547. doi:10.1016/j.neuroimage.2024.120547.DOI
  17. [17]Han, K., Li, G., Fang, Z., Yang, F., 2023. Multi-template meta-information regularized network for alzheimer’s disease diagnosis using structural mri. IEEE Transactions on Medical Imaging 43, 1664–1676. doi:10.1109/TMI.2023.3344384.DOI
  18. [18]Ilse, M., Tomczak, J., Welling, M., 2018. Attention-based Deep Multiple Instance Learning, in: Proceedings of the 35th International Conference on Machine Learning, PMLR. pp. 2127–2136.
  19. [19]Jiang, H., Miao, C., 2024. Anatomy-Aware Gating Network for Explainable Alzheimer’s Disease Diagnosis, in: Medical Image Computing and Computer Assisted Intervention – MICCAI 2024, Springer Nature Switzerland. pp. 90–100. doi:10.1007/978-3-031-72086-4_9.DOI
  20. [20]Jin, C., Guo, Z., Lin, Y., Luo, L., Chen, H., 2026. Learning with less supervision: A survey of label-efficient learning for medical image analysis. Medical Image Analysis 111, 104062. doi:10.1016/j.media.2026.104062.DOI
  21. [21]Kang, J.J., Chen, Y., Xu, G.D., Bao, S.L., Wang, J., Ge, M., Shen, L.H., Jia Z.Z., 2022. Combining quantitative susceptibility mapping to radiomics in diagnosing Parkinson’s disease and assessing cognitive impairment. European Radiology 32, 6992–7003. doi:10.1007/s00330-022-08790-8.DOI
  22. [22]Kaunzner, U.W., Kang, Y., Zhang, S., Morris, E., Yao, Y., Pandya, S., Hurtado Rua, S.M., Park, C., Gillen, K.M., Nguyen, T.D., Wang, Y., Pitt, D., Gauthier, S.A., 2019. Quantitative susceptibility mapping identifies inflammation in a subset of chronic multiple sclerosis lesions. Brain 142, 133–145. doi:10.1093/brain/awy296.DOI
  23. [23]Langkammer, C., Liu, T., Khalil, M., Enzinger, C., Jehna, M., Fuchs, S., Fazekas, F., Wang, Y., Ropele, S., 2013. Quantitative Susceptibility Mapping in Multiple Sclerosis. Radiology 267, 551–559. doi:10.1148/radiol.12120707.DOI
  24. [24]Langkammer, C., Pirpamer, L., Seiler, S., Deistung, A., Schweser, F., Frenthal, S., Homayoon, N., Katschmig-Winter, P., Koegl-Wallner, M., Pendl, T., et al., 2016. Quantitative susceptibility mapping in parkinson’s disease. PloS one 11, e0162460. doi:10.1371/journal.pone.0162460.DOI
  25. [26]Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollar, P., 2017. Focal Loss for Dense Object Detection, in: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 2980–2988. doi:10.1109/ICCV.2017.324.DOI
  26. [27]Liu, C., Wei, H., Gong, N.J., Cronin, M., Dibb, R., Decker, K., 2015. Quantitative susceptibility mapping: contrast mechanisms and clinical applications. Tomography 1, 3–17. doi:10.18383/j.tom.2015.00136.DOI
  27. [28]Liu, M., Zhang, J., Adeli, E., Shen, D., 2018. Landmark-based deep multi-instance learning for brain disease diagnosis. Medical Image Analysis 43, 157–168. doi:10.1016/j.media.2017.10.005.DOI
  28. [30]Perez, E., Strub, F., de Vries, H., Dumoulin, V., Courville, A., 2018. FiLM: Visual Reasoning with a General Conditioning Layer, in: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 3942–3951. doi:10.1609/aaai.v32i1.11671.DOI
  29. [31]QSM Consensus Organization Committee, Bilgic, B., Costagli, M., Chan, K.S., Duyn, J., Langkammer, C., Lee, J., Li, X., Liu, C., Marques, J.P., et al., 2024. Recommended implementation of quantitative susceptibility mapping for clinical research in the brain: A consensus of the ISMRM electro-magnetic tissue properties study group. Magnetic Resonance in Medicine 91, 1834–1862. doi:10.1002/mrm.30006.
  30. [32]Rabiner, L.R., 1989. A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 77, 257–286. doi:10.1109/5.18626.DOI
  31. [33]Rahmanzadeh, R., Galbusera, R., Lu, P.J., Bahn, E., Weigel, M., Barakovic, M., Franz, J., Nguyen, T.D., Spincemaille, P., Schiavi, S., et al., 2022. A New Advanced MRI Biomarker for Remyelinated Lesions in Multiple Sclerosis. Annals of Neurology 92, 486–502. doi:10.1002/ana.26441.DOI
  32. [34]Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D., 2017. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization, in: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 618–626. doi:10.1109/ICCV.2017.74.DOI
  33. [35]Shao, Z., Bian, H., Chen, Y., Wang, Y., Zhang, J., Ji, X., Zhang, Y., 2021. TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification, in: Advances in Neural Information Processing Systems, Curran Associates, Inc.. pp. 2136–2147.arxiv.org/abs/2106.00908
  34. [36]Smith, S.M., 2002. Fast robust automated brain extraction. Human brain mapping 17, 143–155. doi:10.1002/hbm.10062.DOI
  35. [37]Sun, H., Klahr, A.C., Kate, M., Gioia, L.C., Emery, D.J., Butcher, K.S., Wilman, A.H., 2018. Quantitative susceptibility mapping for following intracranial hemorrhage. Radiology 288, 830–839. doi:10.1148/radiol.2018171918.DOI
  36. [38]Tang, X., Guo, R., Zhang, C., Qian, X., 2024a. A causal counterfactual graph neural network for arising-from-chair abnormality detection in parkinsonians. Medical Image Analysis 97, 103266. doi:10.1016/j.media.2024.103266.DOI
  37. [39]Tang, X., Guo, R., Zhang, C., Zhuang, X., Qian, X., 2023. A causality-driven graph convolutional network for postural abnormality diagnosis in parkinsonians. IEEE Transactions on Medical Imaging 42, 3752–3763. doi:10.1109/TMI.2023.3305378.DOI
  38. [41]Tian, Y., Zhang, Y., Cui, Y., Nan, D., Liu, J., Wang, Q., Duan, J., Lian, J., Tian, S., Lin, L., Liu, H., Liu, P., 2026. Quantitative susceptibility map ping and MRS-based multimodal machine learning for early Parkinson’s disease classification. npj Parkinson’s Disease 12, 134. doi:10.1038/s41531-026-01302-1.
  39. [42]Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polozsukhin, I., 2017. Attention is All You Need, in: Advances in Neural Information Processing Systems.arxiv.org/abs/1706.03762
  40. [43]van der Velden, B.H.M., Kuijf, H.J., Gilhuijs, K.G.A., Viergever, M.A., 2022. Explainable artificial intelligence (XAI) in deep learning-based medical image analysis. Medical Image Analysis 79, 102470. doi:10.1016/j.media.2022.102470.
  41. [44]Voon, C.C., Wiltgen, T., Wiestler, B., Schlaeger, S., Mühlau, M., 2024. Quantitative susceptibility mapping in multiple sclerosis: A systematic review and meta-analysis. NeuroImage: Clinical 42, 103598. doi:10.1016/j.nicl.2024.103598.DOI
  42. [45]Wang, Y., He, N., Zhang, C., Zhang, Y., Wang, C., Huang, P., Jin, Z., Li, Y., Cheng, Z., Liu, Y., et al., 2023. An automatic interpretable deep learning pipeline for accurate Parkinson’s disease diagnosis using quantitative susceptibility mapping and T1-weighted images. Human Brain Mapping 44, 4426–4438. doi:10.1002/hbm.26399.DOI
  43. [46]Wang, Y., Liu, T., 2015. Quantitative susceptibility mapping (QSM): decoding mri data for a tissue magnetic biomarker. Magnetic resonance in medicine 73, 82–101. doi:10.1002/mrm.25358.DOI
  44. [47]Weißer, C., Netzer, N., Görtz, M., Schütz, V., Hielscher, T., Schwab, C., Hohenfellner, M., Schlemmer, H.P., Maier-Hein, K.H., Bonekamp, D., 2024. Weakly Supervised MRI Slice-Level Deep Learning Classification of Prostate Cancer Approximates Full Voxel- and Slice-Level Annotation: Effect of Increasing Training Set Size. Journal of Magnetic Resonance Imaging 59, 1409–1422. doi:10.1002/jmri.28891.
  45. [48]Wisnieff, C., Ramanan, S., Olesik, J., Gauthier, S., Wang, Y., Pitt, D., 2015. Quantitative susceptibility mapping (QSM) of white matter multiple sclerosis lesions: Interpreting positive susceptibility and the presence of iron. Magnetic Resonance in Medicine 74, 564–570. doi:10.1002/mrm.25420.
  46. [49]Xiao, B., He, N., Wang, Q., Cheng, Z., Jiao, Y., Haacke, E.M., Yan, F., Shi, F., 2019. Quantitative susceptibility mapping based hybrid feature extraction for diagnosis of Parkinson’s disease. NeuroImage: Clinical 24, 102070. doi:10.1016/j.nicl.2019.102070.DOI
  47. [50]Yan, R., Sun, Q., Jin, C., Liu, Y., He, Y., Guan, T., Chen, H., 2024. Shapley values-enabled progressive pseudo bag augmentation for whole-slide image classification. IEEE transactions on medical imaging 44, 588–597. doi:10.1109/TMI.2024.3453386.DOI
  48. [52]Zhang, H., Meng, Y., Zhao, Y., Qiao, Y., Yao, Y., Yang, X., Coupland, S.E., Zheng, Y., 2022a. DTFD-MIL: Double-tier feature distillation multiple instance learning for histopathology whole slide image classification, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 18802–18812. doi:10.1109/CVPR52688.2022.01824.DOI
  49. [53]Zhang, H., Nguyen, T.D., Zhang, J., Marcille, M., Spincemaille, P., Wang, Y., Gauthier, S.A., Sweeney, E.M., 2022b. QSMRim-Net: Imbalance-aware learning for identification of chronic active multiple sclerosis lesions on quantitative susceptibility maps. NeuroImage: Clinical 34, 102979. doi:10.1016/j.nicl.2022.102979.DOI
  50. [54]Zhang, W., Yang, C., Cao, Z., Li, Z., Zhuo, L., Tan, Y., He, Y., Yao, L., Zhou, Q., Gong, Q., Sweeney, J.A., Shi, F., Lui, S., 2023. Detecting individuals with severe mental illness using artificial intelligence applied to magnetic resonance imaging. eBioMedicine 90, 104541. doi:10.1016/j.ebiom.2023.104541.DOI
  51. [55]Zhu, W., Sun, L., Huang, J., Han, L., Zhang, D., 2021. Dual Attention Multi-Instance Deep Learning for Alzheimer’s Disease Diagnosis With Structural MRI. IEEE Transactions on Medical Imaging 40, 2354–2366. doi:10.1109/tmi.2021.3077079.DOI

Paper details

vlnc:W_k5jezu2e
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC0-1.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents