betaFeedback

WorksComputer Science

Generative Active Learning for Label-Efficient Pan-Organ Ultrasound Diagnosis

Abstract

Medical ultrasound (US) enables real-time, non-invasive diagnosis, yet building US-based AI systems is limited by data scarcity, costly expert annotation, and limited cross-center generalizability. Privacy constraints and instance-level reporting hinder the collection of diverse abnormal cases and frame-level labels, while repeated retraining for new centers increases computational cost. We introduce EvoDiff, a generative active learning framework that integrates diffusion modelling with information-theoretic objectives, evolutionary search, and curriculum-based sampling to select informative samples across heterogeneous US datasets. EvoDiff formulates sample selection as a generate-to-retrieve process, in which diffusion-based optimization in feature space guided by information gain produces synthetic probes, and only the real unlabeled samples retrieved by these probes are annotated and used for downstream training. Across six benchmarks covering fifteen datasets and 16,696 images, we evaluate EvoDiff on convolutional network and transformer-based foundation model. EvoDiff approaches fully supervised performance with substantially fewer labels, outperforms active learning baselines, and improves foundation model generalization under cross-domain external validation on unseen cohorts. A reader study with six radiologists indicates the potential to support clinical annotation workflows, with a model-based estimated 24% to 84% reduction in annotation workload at matched downstream performance.

Introduction

Medical ultrasound (US) is one of the most widely adopted imaging modalities in routine clinical care, valued for its non-invasive, real-time, and cost-effective nature across diverse organ systems [1, 2, 3]. Deep learning has recently accelerated progress in US analysis [4, 5, 6, 7], achieving strong performance for appendicitis assessment [8], breast [9, 10] and ovarian tumor [11] characterization, liver [12] and thyroid [13] diagnosis, and a variety of other diagnostic tasks [14]. However, translating these advances into reliable clinical deployment remains challenging. Daily US workflows rely on instance-level reporting, face strict privacy constraints, and produce highly redundant frame sequences with pronounced class imbalance, all of which hinder the collection of diverse and well-annotated datasets [15, 16]. Frame-level annotation further requires experienced radiologists, making large-scale labels slow and costly to obtain even at high-volume centers. Besides, models trained at a single institution typically require substantial retraining or fine-tuning before they can be safely deployed at new centers, which places an additional computational burden on resource-constrained hospitals. These bottlenecks exacerbate data scarcity, inflate expert annotation costs, and ultimately limit cross-center generalizability of US-based AI systems[17, 18]. These factors lead to pervasive data scarcity and inconsistent label quality, both of which complicate the development of robust deep learning models. While recent advances in CNNs, Transformers, and foundation models have improved representation learning, they continue to rely on abundant high-quality labels. Naïvely expanding datasets by aggregating scans from heterogeneous anatomies, devices, and acquisition protocols risks introducing label noise, incorporating uninformative samples, and degrading generalization due to negative transfer effects[19, 20]. This motivates label-efficient strategies capable of systematically prioritizing informative samples, reducing expert workload, and remaining robust to heterogeneous US data distributions and downstream model architectures[21].

Active learning (AL) offers a principled way to address annotation bottlenecks by selectively querying the most informative samples for expert review[22, 23, 24]. By prioritizing high-value data points, AL can substantially reduce the amount of labeled data needed to achieve strong diagnostic performance. However, its effectiveness in medical US remains limited by anatomical complexity, inter-scanner variability, and the multi-organ nature of real-world cohorts[25, 26]. Existing AL methods are typically developed and evaluated for a single downstream classification task and a single model family, most often CNN classifiers, and perform poorly when scaled to heterogeneous pan-organ settings or when paired with modern Transformer-based foundation models. Conceptually, AL strategies fall into two broad paradigms[27, 28]. Exploitation-oriented methods estimate sample informativeness from model prediction uncertainties and preferentially select highly uncertain examples[29]. Although intuitive, this reliance on posterior-based uncertainty can skew the training distribution, overlook sample learnability, and produce biased or unstable selections when applied to heterogeneous US cohorts. Exploration-oriented methods aim to maximize representativeness, for example via clustering or distance-based sampling[30, 31], yet tend to over-select redundant central samples while missing clinically important outliers. Hybrid strategies attempt to balance uncertainty and representativeness, yet they remain constrained by posterior frameworks in which the quality of selection is only revealed after full model retraining and evaluation[32]. These limitations are magnified in multi-organ US, where speckle noise, variable image quality, and distinct anatomical characteristics create high-dimensional, multi-source feature spaces that challenge conventional AL heuristics[33, 34, 35].

Recent progress in generative models offers a complementary path for AL. Diffusion models have demonstrated strong capability in capturing complex medical image distributions and generating anatomically coherent samples[36, 37], suggesting that generative priors could guide AL in high-dimensional feature spaces. In principle, AL methods could leverage such priors to design information-theoretic sampling strategies rather than relying on ad hoc posterior heuristics. However, existing generative AL approaches, for example, those based on generative adversarial networks or Bayesian generative models[38, 39], provide limited theoretical guidance on how to guarantee information gain (IG) for downstream classification tasks and have rarely been explored in pan-organ or foundation model-based US settings.

These observations reveal a critical gap. Current AL frameworks for medical US are predominantly task-specific, region-specific, and model-specific. They do not fully exploit modern generative model capabilities, or integrate seamlessly with foundation models that provide shared, transferable US representations across anatomies and scanners[40, 41]. There is a need for a pan-organ AL framework that can operate in heterogeneous, multi-organ US cohorts, leverage generative priors to explore high-dimensional feature spaces, and prioritizing samples using principled information-theoretic criteria. Such a framework should be compatible with both conventional CNNs and Transformer-based foundation models, while offering improved interpretability by making explicit the information value of selected samples.

In this work, we introduce EvoDiff, a pan-organ AL framework that integrates Evolutionary optimization with information gain (IG) guidance during Diffusion sampling. Instead of querying samples from a fixed posterior-based strategy, EvoDiff reformulates sample selection as a diffusion-based feature space optimization problem and establishes a novel theoretical connection between diffusion sampling trajectories and evolutionary search for AL. We estimate IG via entropy differentials along DDIM sampling paths, guiding generation toward informative regions without updating the parameters of the pretrained diffusion model or guidance classifier, and use evolutionary perturbations to escape local optima. EvoDiff further induces a natural curriculum learning strategy in which generated samples progress from certain to uncertain cases, yielding smoother learning dynamics for downstream classifiers and improved efficiency in limited-data settings.

We conducted comprehensive validation across six representative US benchmarks encompassing fifteen datasets with 16,696 images. To demonstrate broad applicability across architectural paradigms, we evaluated EvoDiff with two distinct downstream models, namely a conventional convolutional neural network ResNet-50[42] and a Transformer-based foundation model USFM[43]. These experiments cover six diagnostic tasks, including appendix diagnosis, breast tumor recognition and diagnosis, follow-up cardiovascular event prediction, fatty liver diagnosis, ovarian tumor assessment, and thyroid nodule diagnosis, covering eight internal cohorts and seven independent external cohorts. EvoDiff consistently identifies informative samples in complex, imbalanced multi-organ US feature spaces, approaches fully supervised performance while using substantially fewer labels, and outperforms state-of-the-art AL methods. Moreover, EvoDiff enhances the cross-domain generalization of the foundation model under external validation on unseen datasets. To assess EvoDiff’s clinical effectiveness, we conducted a reader study

Results

In the following subsections, we first describe the multi-organ, multi-task dataset and formalize the problem. We then introduce the IG-guided EvoDiff framework and outline its pipeline. We show that EvoDiff can yield consistent performance improvements on CNN-based and Foundation-based downstream models with distinct architectures. Subsequently, we broaden the sample selection strategy from an IG evaluation to explicitly optimizing a foundation model’s generalization on external validation cohorts. Finally, a comprehensive reader study is conducted in a real-world clinical setting to demonstrate that EvoDiff substantially reduces the annotation time estimated by the Human Annotation Effort (HAE) metric while improving interpretability via intuitive visualizations.

Cross-organ Multi-task Ultrasound Dataset

Our goal is to facilitate informative data selection for pan-organ US image diagnostic tasks by integrating diffusion models with semi-supervised active learning (SSAL). In this study, we constructed an extensive US imaging dataset from 12 public repositories and three private cohorts, comprising 16,696 images across 6 major organs. As shown in Table 1, the extensive downstream diagnostic tasks included: pediatric appendicitis diagnosis[44], breast tumor recognition and malignancy classification[45, 46, 47, 48, 49], carotid artery US for cardiovascular event prediction[50], fatty liver assessment[51], ovarian health and pathology evaluation[52], and thyroid nodule diagnosis[53, 54]. The ovary dataset (MMOTU) originally comprised seven distinct ovarian disease categories and one normal category. We manually combined these seven ovarian disease categories into a single abnormality class, thereby developing a binary classification task that distinguishes between normal and abnormal samples. Eight datasets were partitioned using a fixed 8:1:1 ratio and merged to construct the internal training set, internal test set, and a limited labeled subset for model initialization followed by supervised fine-tuning. The 8:1:1 split is fixed throughout the study. At each cumulative annotation budget the expanded labeled training set is subjected to five-fold cross-validation, in which four folds are used for training and one fold for validation and model selection, and the five resulting fold-specific models are evaluated on the same fixed internal test set. Strict case-level partitioning was ensured for seven of the eight public internal datasets. BUSI was the only dataset for which patient-level partitioning could not be verified, because its public release contains 780 images from 600 women without patient identifiers. For BUSI we performed image-level partitioning and additionally applied image-similarity matching to keep highly similar images within the same split wherever possible, and we note that strict patient-level separation for this dataset cannot be formally verified. The internal validation cohorts were employed to assess EvoDiff’s capability in selecting informative samples that enhance the performance of two downstream architectures, e.g., ResNet-50 and USFM. Additionally, seven independent datasets were used in their entirety as external validation cohorts to rigorously evaluate the cross-domain generalization ability of the USFM model fine-tuned with EvoDiff-selected samples. Four of these are public repositories and three are private cohorts collected at Zhejiang Cancer Hospital. The three private external cohorts were assembled retrospectively under ethics approval by the Institutional Review Board of Zhejiang Cancer Hospital (No. 2026ZNY004-1). Notably, the pan-organ US dataset introduces additional challenges arising from substantial inter-organ heterogeneity. The cohort exhibits pronounced long-tailed class imbalance and marked structural variability across anatomical regions, disease phenotypes, imaging devices, and patient populations. For example, thyroid US images constitute 34.31% of the internal training set, whereas appendix samples account for only 4.66%, illustrating the extreme disparity in sample availability. This heterogeneity, coupled with imbalanced data distribution, poses stringent challenges for developing unified and robust representations that generalize across organ systems.

OrganDatasetChallengeNormalBenignAbnormalTotalCountryCollection PeriodUltrasound EquipmentPatient CountAge Range (Mean±SD)aDataset Typeb
AppendixAppendix[44]Appendicitis diagnosis174–300474Germany2016–2021Toshiba Xario, Aplio XG machines5790–18Internal
PrivAppendixAppendicitis diagnosis21–2445China––––External
BreastBUSI[45]Breast tumor recognition133437210780Egypt2018GE Logiq E9–25–75Internal
BUS-BRA[46]Breast tumor diagnosis–1,2686071,875Brazil–GE Logiq 5 & 7, Aplio 300, U-Systems1,64216–89Internal
UDIAT[47]Breast tumor diagnosis–10954163Spain2012Siemens C512163–Internal
ALN-Ultra[48]Breast tumor diagnosis–121136257China2019–2021–25730–85External
BUS_UCBreast tumor diagnosis–358453811China–SonoSkills, Hitachi Medical Systems––External
QAMEBI[49]Breast tumor diagnosis–109123232Iran–AirPlorer Ultimate232above 18External
PrivBreastBreast tumor diagnosis–5941100China––––External
CarotidCUBS[50]Cardiovascular event prediction1,124–2541,378Toronto–Epiq 7G, MyLab25, ATL HDI500, FAMUS––Internal
LiverFatty-Liver[51]Fatty liver diagnosis170–380550Poland–GE Vivid E955– (40.1±9.1)Internal
PrivLiverFatty liver diagnosis24–4569China––––External
OvaryMMOTU[52]Ovarian tumor assessment267–1,2021,469China–Mindray Resona8294–Internal
ThyroidTN3K[53]Thyroid nodule diagnosis–2,2831,2103,493–2016–2020–2,421–Internal
TN5000[54]Thyroid nodule diagnosis–1,4263,5745,000China–GE Logiq E9 & S7––External
Total (Internal)1,8684,0974,21710,182––––––
Total (External)452,0734,3966,514––––––
Grand Total1,9136,1708,61316,696––––––

Table 1. Overview of 6 clinical applications and dataset distributions across multi-organ US datasets. Fifteen datasets comprising 16,696 images across six anatomical regions were used, with 10,182 images for model training and internal validation, and 6,514 images from ALN-Ultra, BUS_UC, QAMEBI, TN5000, and the three private cohorts for external validation.

Problem Formulation

Consider a heterogeneous pan-organ US dataset spanning KK anatomical regions, denoted as G={σk}k=1K\mathcal{G} = \{\sigma_k\}_{k=1}^{K}, collected from MM independent medical centers. The full dataset is represented as D={Di}i=1M\mathcal{D} = \{\mathcal{D}_i\}_{i=1}^{M}, where each Di\mathcal{D}_i corresponds to data from center ii. The distribution of imaging sources is typically uneven across centers and organs, introducing inter-center variations in acquisition protocols and scanner characteristics, where each organ category may have data collected from either multiple centers or a single center. Each center-specific dataset is defined as Di={(xji,yji)}j=1Ni\mathcal{D}_i = \{(x_j^i,y_j^i)\}_{j=1}^{N_i}, where xji∈Rh×wx_j^i \in\mathbb{R}^{h \times w} represents a US image and yjiy_j^i is the corresponding diagnostic label, which varies according to the clinical task associated with each organ.

According to our AL formulation, the dataset from each center is partitioned in a fixed 8:1:1 ratio and subsequently integrated to construct a unified pan-organ US learning benchmark. This partitioning serves three purposes.

  1. an unlabeled candidate pool DiU\mathcal{D}_i^{U} (80%), containing only organ-level metadata and used as the candidate pool from which EvoDiff selects informative samples for annotation;

(2) a labeled initialization set DiL\mathcal{D}^{L}_{i} (10%), comprising fully annotated samples with both organ and diagnostic labels for warm-start model training;

(3) a test set DiT\mathcal{D}^{T}_{i} (10%), reserved for internal evaluation of downstream performance in each AL iteration.

The 8:1:1 partition is fixed for the whole study, and all reported performance is obtained by five-fold cross-validation within the expanded labeled training set at each cumulative annotation budget, with every fold-specific model evaluated on the same fixed test set DiT\mathcal{D}^{T}_{i}. We implement K=6K = 6 anatomical regions collected from M=8M = 8 independent medical centers. Specifically, breast US images are contributed by three centers, while each of the remaining five organ cohorts originates from a single institution.

Unlike prior AL studies that typically target homogeneous, single-organ datasets[55] or multi-modal settings where separate models are trained per dataset[56], our task presents unique challenges to existing AL methods. These challenges arise from a high-dimensional, complex representation space in our pan-organ US dataset, characterized by class boundary ambiguity and feature space entanglement across different anatomical structures. The dataset requires AL frameworks to learn simultaneously across multiple organs and diagnostic tasks in the presence of extreme class imbalance, inter-class overlap, and subtle feature variations. This multi-domain heterogeneity introduces distribution shifts that exacerbate the representation collapse problem in limited annotation scenarios. CNN-based AL strategies implicitly assume homogeneous representation spaces and uniform data distributions, failing to generalize under severe cross-domain heterogeneity. Their inability to learn discriminative representations from limited labeled data becomes particularly pronounced when facing intra-class variability across anatomical domains. Consequently, our problem formulation necessitates a more effective sample selection strategy that can identify structurally informative examples while progressively organizing a coherent embedding space suitable for pan-organ US analysis.

Overview of EvoDiff

To address these challenges in heterogeneous pan-organ US learning, we propose EvoDiff, a generative AL framework that couples diffusion modeling with information-theoretic sampling to enable label-efficient learning from multi-domain US data. As shown in Figure 1 and Figure 7, a diffusion model and a guidance classifier are first jointly pretrained on abundant organ-identified but pathology-unlabeled US images and a limited labeled subset, through organ-conditioned coarse pretraining followed by pathology-level fine-tuning, to learn anatomical representations. Second, during generation, we introduce an IG-guided sampling strategy that computes entropy-based information gradients along the DDIM denoising trajectory, steering sample synthesis toward regions of maximal epistemic value while the parameters of the pretrained diffusion model and guidance classifier remain frozen. Meanwhile, Local Optimization Evolutionary Algorithm (LOEA) refines intermediate diffusion latents by maximizing entropy under category consistency constraints, yielding highly informative synthetic candidates that better explore diagnostically relevant regions of the data manifold. Finally, EvoDiff performs iterative AL by ranking generated samples via cosine similarity against the unlabeled dataset, selecting the top informative US images for expert annotation, and scaling the learning difficulty of downstream models through curriculum learning subset selection from certain to uncertain cases. The enriched labeled pool is then used to finetune downstream diagnostic models (e.g., ResNet-50 or USFM), achieving label-efficient generalization across 6 clinical tasks spanning 6 organs. Together, EvoDiff establishes a principled integration of generative modeling, information theory, and evolutionary optimization to reduce annotation cost in real-world medical imaging.

Overview of the EvoDiff pipeline

Figure 1. Overview of the EvoDiff pipeline. a) Organ-conditioned coarse pretraining of the diffusion model and the guidance classifier on multi-organ, organ-identified but pathology-unlabeled ultrasound images, followed by pathology-level fine-tuning on the limited labeled subset. A flame symbol indicates that the parameters of a network are updated in that stage, and a snowflake symbol indicates that the network is frozen. b) Information-gain (IG)-guided sampling combined with the Local Optimization Evolutionary Algorithm (LOEA), which performs gradient-free evolutionary refinement along the frozen diffusion trajectory and uses entropy-based IG signals to steer latents toward informative regions. The triangular maps show the predictive-entropy landscape of the guidance classifier before and after LOEA refinement, coloured from dark blue for low entropy to yellow for high entropy. As the guidance strength increases, generation moves from certain samples, drawn as green disks, to uncertain samples, drawn as orange disks, as indicated by the green-to-orange arrow. The generated probes and the unlabeled candidate pool then undergo feature extraction and cosine-similarity ranking, and the expert oracle, shown in the red dashed box, annotates the top α\alpha% most informative real samples. The generated probes are never annotated and are never used as downstream training samples. c) Newly annotated data are merged with the existing labeled data to fine-tune the downstream models, a ResNet-50 and the ultrasound foundation model (USFM), and each subsequent active learning (AL) iteration increases the uncertainty of the training set. After all AL iterations, the downstream models are validated on 15 datasets, 8 internal and 7 external, spanning 6 diagnostic tasks.

Technical pipeline and implementation details of the EvoDiff framework

Figure 7. Technical pipeline and implementation details of the EvoDiff framework. a) Pre-training phase. Organ-conditioned coarse pretraining of the diffusion model and the guidance classifier uses the 80% pathology-unlabeled pool, followed by pathology-level fine-tuning on the 10% pathologically labeled subset, with the remaining 10% reserved as the test set for final validation. A flame symbol indicates that a network is updated in that step, and a snowflake symbol indicates that it is frozen. b) Sampling phase. Information gain-guided sampling with the frozen diffusion model generates synthetic sample sets under guidance strengths (GS) from 1 to 9, forming the certain-to-uncertain curriculum sets depicted by the green-to-orange cylinders and arrow. c) Selecting phase. Features of the synthetic probes and of the unlabeled candidate pool are extracted by a ResNet-50 pretrained on natural images, cosine similarity between the two feature sets is computed within each organ-level category, and the top α%\alpha\% most similar real images, outlined in red in the ranking columns, are selected for expert annotation. d) Active learning (AL) iteration phase. The newly annotated samples are merged into the training set to fine-tune the downstream models, a ResNet-50 and the ultrasound foundation model (USFM), which are then validated on the frozen test set, and successive cycles progressively increase the curriculum difficulty, as shown by the green-to-orange arrow.

Comparative Performance Analysis on CNN-based Downstream Model

We benchmarked EvoDiff against 6 representative baselines: (1) FSL (Fully Supervised Learning), which uses the full labeled dataset and defines an empirical upper bound on performance; (2) Random sampling, which uniformly selects samples without prioritization; (3) VAAL[57], a variational adversarial AL method that aligns labeled and unlabeled distributions via latent-space discrimination; (4) Core-Set[58], which selects samples that minimize the maximum distance to the labeled set in feature space; (5) Max-Entropy[59], which queries samples with high predictive uncertainty; and (6) BALD[60], which measures mutual information between predictions and model parameters under Bayesian inference. All methods were evaluated under identical annotation budgets of 5%, 10%, 20%, 30%, 40%, 50%, and 100% of labeled data. For consistency, all methods employed ResNet-50 as the downstream architecture, initialized with ImageNet pre-trained weights and further fine-tuned on 10% of the labeled dataset to ensure a fair comparison.

Experiments were conducted across eight heterogeneous US datasets spanning 6 anatomical organs: Appendix[44], BUSI[45], BUS-BRA[46], UDIAT[47], CUBS[50], Fatty-Liver[51], MMOTU[52], and TN3K[53]. As summarized in Table 1, this benchmark exhibits substantial inter-organ heterogeneity and class imbalance. For instance, the Appendix dataset comprises only 4.66% of the total images, while several organ datasets show pronounced intra-class imbalance between benign and malignant cases. This design enables a rigorous and realistic evaluation of EvoDiff under clinically relevant data distributions. Across the 48 evaluated dataset and budget settings, EvoDiff ranked first in accuracy in 43 and second in the remaining five, where the largest gap from the best-performing method was 0.88 percentage points (Table 2). To further examine performance trends over iterative AL rounds, we reported the area under the ROC curve (AUC) across AL iterations in Figure 2. EvoDiff not only achieves the highest AUC across all datasets but also exhibits a consistently steeper improvement trajectory compared to uncertainty-based and representation-based baselines. Notably, EvoDiff demonstrates stable convergence behavior with lower variance across AL rounds, indicating enhanced robustness in sample acquisition.

Performance comparison of six active learning methods across annotation rounds on eight ultrasound image datasets

Figure 2. Performance comparison of six active learning methods across annotation rounds on eight ultrasound image datasets. Results are reported as the area under the receiver operating characteristic curve (AUC, %) of the downstream ResNet-50 classifier. Round 0 represents the baseline performance of the initialized ResNet-50, and rounds 1 to 6 correspond to cumulative annotation budgets of 5%, 10%, 20%, 30%, 40%, and 50% of the unlabeled candidate pool. a) Appendicitis diagnosis on Appendix. b) Breast tumor diagnosis on BUSI. c) Breast tumor diagnosis on BUS-BRA. d) Breast tumor diagnosis on UDIAT. e) Cardiovascular event prediction on CUBS. f) Fatty liver diagnosis on Fatty-Liver. g) Ovarian tumor assessment on MMOTU. h) Thyroid nodule diagnosis on TN3K. In every panel, the blue dashed line denotes Random, the orange dashed line denotes VAAL, the green dashed line denotes Core-Set, the red dotted line denotes Max-Entropy, the purple dash-dotted line denotes BALD, and the brown dashed line denotes EvoDiff, each with circular markers at the evaluated rounds. The grey horizontal dashed line marks the fully supervised upper bound trained on the complete labeled set. The shaded band around each curve, drawn in the same colour as the curve, indicates the s.d. (standard deviation) across the five cross-validation folds.

Comparison of classification accuracy between EvoDiff and other AL methods on the test sets. Values are reported as mean ±\pm s.d. across the five fold-specific models obtained by five-fold cross-validation. One ResNet-50 was trained independently for each dataset, with no parameter sharing across datasets or organs. Strategy-versus-Random statistical comparisons, including the two-sided paired tt-tests and the Benjamini–Hochberg multiplicity correction applied across the five strategy-versus-Random comparisons, are reported separately in the Supplementary Information. (Bold represents the best result, <u>Underline</u> represents the second best result).

OrganDatasetMethod5%10%20%30%40%50%100%
AppendixAppendix[44]FSL (upper bound)------0.6404±0.01040.6404\pm0.0104
Random0.4783±0.05830.4783\pm0.05830.4870±0.11630.4870\pm0.11630.4913±0.06680.4913\pm0.06680.5391±0.04220.5391\pm0.04220.6000±0.03790.6000\pm0.03790.6087±0.05500.6087\pm0.0550-
VAAL[57]0.4635±0.09670.4635\pm0.09670.4891±0.07630.4891\pm0.07630.4936±0.08690.4936\pm0.08690.5279±0.04480.5279\pm0.04480.5435±0.09670.5435\pm0.09670.5832±0.13830.5832\pm0.1383-
Core-Set[58]0.4845±0.06920.4845\pm0.06920.5096±0.01490.5096\pm0.01490.4858±0.09180.4858\pm0.09180.5392±0.05730.5392\pm0.05730.5502±0.02700.5502\pm0.02700.5745±0.08540.5745\pm0.0854-
Max-Entropy[59]0.5016±0.07080.5016\pm0.07080.5277±0.06450.5277\pm0.06450.5556±0.07940.5556\pm0.07940.5804±0.09400.5804\pm0.09400.5932±0.08090.5932\pm0.08090.6217±0.09270.6217\pm0.0927-
BALD[60]0.5286±0.10770.5286\pm0.10770.5145±0.06920.5145\pm0.06920.5769±0.08700.5769\pm0.08700.5932±0.13830.5932\pm0.13830.5896±0.08490.5896\pm0.08490.6292±0.07730.6292\pm0.0773-
EvoDiff (Ours)0.5435±0.00180.5435\pm0.00180.5652±0.02170.5652\pm0.02170.5764±0.05430.5764\pm0.05430.5978±0.01090.5978\pm0.01090.6196±0.01090.6196\pm0.01090.6373±0.05430.6373\pm0.0543-
BUSI[45]FSL (upper bound)------0.9424±0.02610.9424\pm0.0261
Random0.6670±0.05340.6670\pm0.05340.7515±0.03400.7515\pm0.03400.7758±0.02940.7758\pm0.02940.8273±0.04850.8273\pm0.04850.8545±0.03120.8545\pm0.03120.8739±0.03210.8739\pm0.0321-
VAAL[57]0.6596±0.17550.6596\pm0.17550.6822±0.08910.6822\pm0.08910.7282±0.01460.7282\pm0.01460.7892±0.07970.7892\pm0.07970.8462±0.09690.8462\pm0.09690.8315±0.16740.8315\pm0.1674-
Core-Set[58]0.6555±0.02010.6555\pm0.02010.7599±0.09550.7599\pm0.09550.8044±0.08390.8044\pm0.08390.8326±0.03780.8326\pm0.03780.8675±0.14370.8675\pm0.14370.8798±0.07430.8798\pm0.0743-
Max-Entropy[59]0.6779±0.01210.6779\pm0.01210.7110±0.03620.7110\pm0.03620.7567±0.04470.7567\pm0.04470.7941±0.09650.7941\pm0.09650.8706±0.07410.8706\pm0.07410.8731±0.06760.8731\pm0.0676-
BALD[60]0.6919±0.09110.6919\pm0.09110.7449±0.08600.7449\pm0.08600.7592±0.03630.7592\pm0.03630.8232±0.08710.8232\pm0.08710.8531±0.06290.8531\pm0.06290.8792±0.08290.8792\pm0.0829-
EvoDiff (Ours)0.7287±0.01090.7287\pm0.01090.8075±0.03230.8075\pm0.03230.8318±0.04680.8318\pm0.04680.8460±0.02660.8460\pm0.02660.8825±0.03960.8825\pm0.03960.9010±0.03190.9010\pm0.0319-
BreastBUS-BRA[46]FSL (upper bound)------0.9558±0.01230.9558\pm0.0123
Random0.6695±0.07870.6695\pm0.07870.7347±0.07390.7347\pm0.07390.7663±0.04370.7663\pm0.04370.8084±0.03440.8084\pm0.03440.8495±0.02970.8495\pm0.02970.8632±0.03540.8632\pm0.0354-
VAAL[57]0.6744±0.02520.6744\pm0.02520.6905±0.01010.6905\pm0.01010.7118±0.03750.7118\pm0.03750.7555±0.02840.7555\pm0.02840.7946±0.02930.7946\pm0.02930.8238±0.06950.8238\pm0.0695-
Core-Set[58]0.6746±0.07360.6746\pm0.07360.6959±0.02910.6959\pm0.02910.7444±0.09070.7444\pm0.09070.7715±0.07930.7715\pm0.07930.8290±0.11430.8290\pm0.11430.8578±0.02910.8578\pm0.0291-
Max-Entropy[59]0.7088±0.03170.7088\pm0.03170.7317±0.07270.7317\pm0.07270.7455±0.03440.7455\pm0.03440.7859±0.04610.7859\pm0.04610.8385±0.02260.8385\pm0.02260.8528±0.02260.8528\pm0.0226-
BALD[60]0.7041±0.03290.7041\pm0.03290.7364±0.02700.7364\pm0.02700.7807±0.01180.7807\pm0.01180.7939±0.06860.7939\pm0.06860.8351±0.09790.8351\pm0.09790.8681±0.02630.8681\pm0.0263-
EvoDiff (Ours)0.7261±0.05190.7261\pm0.05190.7826±0.06190.7826\pm0.06190.8327±0.02760.8327\pm0.02760.8517±0.03650.8517\pm0.03650.8891±0.04610.8891\pm0.04610.9136±0.03510.9136\pm0.0351-
UDIAT[47]FSL (upper bound)------0.9222±0.02720.9222\pm0.0272
Random0.6556±0.07370.6556\pm0.07370.6889±0.05670.6889\pm0.05670.7333±0.09560.7333\pm0.09560.8111±0.06670.8111\pm0.06670.8333±0.06090.8333\pm0.06090.8556±0.07540.8556\pm0.0754-
VAAL[57]0.6513±0.09690.6513\pm0.09690.6772±0.09000.6772\pm0.09000.7355±0.05870.7355\pm0.05870.7601±0.10680.7601\pm0.10680.7856±0.09760.7856\pm0.09760.8318±0.06910.8318\pm0.0691-
Core-Set[58]0.6750±0.02360.6750\pm0.02360.7186±0.02780.7186\pm0.02780.7537±0.01190.7537\pm0.01190.7776±0.05310.7776\pm0.05310.7810±0.04080.7810\pm0.04080.8409±0.03020.8409\pm0.0302-
Max-Entropy[59]0.6870±0.06640.6870\pm0.06640.6856±0.01750.6856\pm0.01750.7579±0.01530.7579\pm0.01530.7786±0.03230.7786\pm0.03230.8116±0.06570.8116\pm0.06570.8303±0.02310.8303\pm0.0231-
BALD[60]0.6714±0.02780.6714\pm0.02780.7176±0.04150.7176\pm0.04150.7930±0.07220.7930\pm0.07220.7511±0.07620.7511\pm0.07620.7731±0.08310.7731\pm0.08310.8334±0.08820.8334\pm0.0882-
EvoDiff (Ours)0.7061±0.07610.7061\pm0.07610.7340±0.01290.7340\pm0.01290.7851±0.07610.7851\pm0.07610.8454±0.07180.8454\pm0.07180.8684±0.06490.8684\pm0.06490.9074±0.05430.9074\pm0.0543-
CarotidCUBS[50]FSL (upper bound)------0.8906±0.04510.8906\pm0.0451
Random0.6705±0.03230.6705\pm0.03230.7122±0.04570.7122\pm0.04570.7410±0.05610.7410\pm0.05610.7899±0.04110.7899\pm0.04110.8029±0.02930.8029\pm0.02930.8072±0.05370.8072\pm0.0537-
VAAL[57]0.7048±0.04510.7048\pm0.04510.7257±0.05280.7257\pm0.05280.7113±0.01510.7113\pm0.01510.7657±0.01410.7657\pm0.01410.7852±0.09550.7852\pm0.09550.7730±0.04110.7730\pm0.0411-
Core-Set[58]0.7241±0.03060.7241\pm0.03060.7317±0.02040.7317\pm0.02040.7413±0.05580.7413\pm0.05580.7799±0.04960.7799\pm0.04960.8060±0.01250.8060\pm0.01250.8247±0.02710.8247\pm0.0271-
Max-Entropy[59]0.7212±0.05210.7212\pm0.05210.7462±0.03210.7462\pm0.03210.7916±0.03920.7916\pm0.03920.8186±0.02080.8186\pm0.02080.8362±0.09240.8362\pm0.09240.8288±0.06710.8288\pm0.0671-
BALD[60]0.7308±0.06320.7308\pm0.06320.7403±0.04160.7403\pm0.04160.7756±0.07970.7756\pm0.07970.8023±0.08320.8023\pm0.08320.8262±0.08020.8262\pm0.08020.8392±0.03890.8392\pm0.0389-
EvoDiff (Ours)0.7518±0.03960.7518\pm0.03960.7951±0.00450.7951\pm0.00450.8218±0.01840.8218\pm0.01840.8442±0.02540.8442\pm0.02540.8593±0.02470.8593\pm0.02470.8770±0.04500.8770\pm0.0450-
LiverFatty-Liver[51]FSL (upper bound)------0.9741±0.03610.9741\pm0.0361
Random0.8251±0.06320.8251\pm0.06320.8469±0.07660.8469\pm0.07660.8753±0.08100.8753\pm0.08100.9157±0.06880.9157\pm0.06880.9163±0.06410.9163\pm0.06410.9321±0.03890.9321\pm0.0389-
VAAL[57]0.8042±0.07130.8042\pm0.07130.8169±0.04910.8169\pm0.04910.8417±0.09220.8417\pm0.09220.8763±0.05890.8763\pm0.05890.8901±0.08140.8901\pm0.08140.9360±0.07620.9360\pm0.0762-
Core-Set[58]0.8145±0.04490.8145\pm0.04490.8294±0.03600.8294\pm0.03600.8490±0.07970.8490\pm0.07970.8870±0.08310.8870\pm0.08310.8510±0.04600.8510\pm0.04600.8973±0.05310.8973\pm0.0531-
Max-Entropy[59]0.8258±0.05210.8258\pm0.05210.8566±0.07200.8566\pm0.07200.8934±0.09130.8934\pm0.09130.9011±0.04420.9011\pm0.04420.9481±0.07700.9481\pm0.07700.9450±0.07490.9450\pm0.0749-
BALD[60]0.8445±0.04290.8445\pm0.04290.8641±0.04550.8641\pm0.04550.8987±0.04210.8987\pm0.04210.9371±0.08700.9371\pm0.08700.9572±0.08500.9572\pm0.08500.9620±0.04750.9620\pm0.0475-
EvoDiff (Ours)0.8563±0.02080.8563\pm0.02080.8815±0.03360.8815\pm0.03360.9240±0.04910.9240\pm0.04910.9552±0.07670.9552\pm0.07670.9608±0.02320.9608\pm0.02320.9724±0.03890.9724\pm0.0389-
OvaryMMOTU[52]FSL (upper bound)------0.9410±0.02120.9410\pm0.0212
Random0.7256±0.04260.7256\pm0.04260.7564±0.04810.7564\pm0.04810.7987±0.01190.7987\pm0.01190.8359±0.03590.8359\pm0.03590.8679±0.03230.8679\pm0.03230.8885±0.01880.8885\pm0.0188-
VAAL[57]0.7273±0.07320.7273\pm0.07320.7393±0.08660.7393\pm0.08660.7877±0.08440.7877\pm0.08440.8195±0.03990.8195\pm0.03990.8442±0.09250.8442\pm0.09250.8704±0.07370.8704\pm0.0737-
Core-Set[58]0.7219±0.08640.7219\pm0.08640.7630±0.03530.7630\pm0.03530.8168±0.08500.8168\pm0.08500.8583±0.05700.8583\pm0.05700.8687±0.06580.8687\pm0.06580.8900±0.08030.8900\pm0.0803-
Max-Entropy[59]0.7530±0.04940.7530\pm0.04940.7747±0.06210.7747\pm0.06210.8372±0.06930.8372\pm0.06930.8770±0.07570.8770\pm0.07570.8867±0.09820.8867\pm0.09820.9015±0.08880.9015\pm0.0888-
BALD[60]0.7450±0.03360.7450\pm0.03360.7630±0.08610.7630\pm0.08610.8248±0.06420.8248\pm0.06420.8475±0.06840.8475\pm0.06840.8550±0.02120.8550\pm0.02120.8931±0.08310.8931\pm0.0831-
EvoDiff (Ours)0.7513±0.05580.7513\pm0.05580.8095±0.04960.8095\pm0.04960.8320±0.01920.8320\pm0.01920.8682±0.02900.8682\pm0.02900.9057±0.05280.9057\pm0.05280.9222±0.04750.9222\pm0.0475-
ThyroidTN3K[53]FSL (upper bound)------0.8937±0.01570.8937\pm0.0157
Random0.6029±0.02610.6029\pm0.02610.6263±0.02090.6263\pm0.02090.7109±0.01190.7109\pm0.01190.7377±0.03140.7377\pm0.03140.7617±0.00670.7617\pm0.00670.7749±0.00380.7749\pm0.0038-
VAAL[57]0.5980±0.07580.5980\pm0.07580.6018±0.21200.6018\pm0.21200.6372±0.08400.6372\pm0.08400.7075±0.03230.7075\pm0.03230.7573±0.04830.7573\pm0.04830.7952±0.06940.7952\pm0.0694-
Core-Set[58]0.6406±0.03130.6406\pm0.03130.6625±0.20690.6625\pm0.20690.6896±0.04840.6896\pm0.04840.7509±0.03220.7509\pm0.03220.7670±0.05440.7670\pm0.05440.8082±0.05760.8082\pm0.0576-
Max-Entropy[59]0.6527±0.02060.6527\pm0.02060.7023±0.05900.7023\pm0.05900.7436±0.05090.7436\pm0.05090.7682±0.07260.7682\pm0.07260.7729±0.05820.7729\pm0.05820.7853±0.05140.7853\pm0.0514-
BALD[60]0.6334±0.03810.6334\pm0.03810.6721±0.08390.6721\pm0.08390.7569±0.05500.7569\pm0.05500.7877±0.05140.7877\pm0.05140.7816±0.07100.7816\pm0.07100.8218±0.04680.8218\pm0.0468-
EvoDiff (Ours)0.6730±0.00110.6730\pm0.00110.7048±0.04510.7048\pm0.04510.7793±0.01590.7793\pm0.01590.8126±0.03820.8126\pm0.03820.8386±0.02080.8386\pm0.02080.8656±0.05970.8656\pm0.0597-

Table 2.

AL algorithms face great challenge at low cumulative annotation budgets (5%, 10%), where the precise identification of informative samples from extreme labeling budgets becomes critical for downstream classifier optimization. Under the low-label conditions, EvoDiff consistently outperformed established methods across multiple datasets (Table 2 and Figure 2). At 5% annotation budget, EvoDiff achieved substantial performance improvements, with the most pronounced gain observed on the Appendix dataset (13.63%↑13.63\%\uparrow over random sampling). Consistent gains were also observed on other datasets, ranging from 3.42%↑3.42\%\uparrow to 12.13%↑12.13\%\uparrow, demonstrating EvoDiff’s ability to acquire high-value samples even from scarce data. At 10% annotation, EvoDiff maintained strong advantages. For example, on the BUSI dataset, EvoDiff achieved 0.8075 accuracy, significantly exceeding Core-Set (0.7599, 6.26%↑6.26\%\uparrow) and BALD (0.7449, 8.40%↑8.40\%\uparrow). Similarly, comparable performance advantages were observed on MMOTU, where EvoDiff reached 0.8095 accuracy, surpassing Core-Set (0.7630) and BALD (0.7630) by 4.65%↑4.65\%\uparrow. These results confirm EvoDiff’s superior sample selection ability in early AL stages.

Class imbalance is common in real-world medical datasets, and accuracy alone conceals how an acquisition strategy behaves under it. The CUBS dataset (Table 1) exemplifies this challenge with its pronounced distributional bias, comprising 81.57% normal versus 18.43% abnormal samples. Supplementary Table S1 reports accuracy, balanced accuracy, sensitivity, specificity, PPV, NPV, F1-score, and AUC for every method, dataset, and budget, and the CUBS entries at the 10% cumulative annotation budget separate the two behaviours. Max-Entropy attained the second highest accuracy at 0.7462 while recording the lowest balanced accuracy at 0.5097 and a minority-class sensitivity of 0.1769, which falls below the 0.1923 obtained by random sampling. EvoDiff instead reached a balanced accuracy of 0.6165, a sensitivity of 0.4154, an F1-score of 0.3769, and an AUC of 0.8242, exceeding Max-Entropy by 10.7, 23.9, 18.7 and 5.7 percentage points respectively, with q=0.0001q = 0.0001 against random sampling after Benjamini–Hochberg correction. The gain on the minority class costs 6.0 percentage points of specificity relative to random sampling, so the improvement reflects a shift of the operating point toward the underrepresented class rather than a uniform gain across both classes. This behaviour follows from the information-guided latent exploration of EvoDiff, which balances exploration of underrepresented classes against exploitation of decision boundaries. Uncertainty-based methods concentrate on the majority class, where predictive entropy is dense enough to dominate the ranking, and diversity-based methods may miss rare but critical cases, whereas EvoDiff selects samples by their contribution to model improvement rather than by their position in the class distribution.

A notable trend in our results reveals that at higher cumulative annotation budgets (20%-50%), most baseline methods exhibit diminishing returns, where each successive round of annotation yields progressively smaller improvements to downstream classifier performance. Under these conditions, EvoDiff maintained consistent performance gains. For example, on the TN3K dataset at 50% cumulative annotation budget, EvoDiff achieved 0.8656 accuracy, substantially outperforming BALD (5.33%↑\uparrow) and Core-Set (7.10%↑\uparrow). Such a trend is also reflected in the AUC progression shown in Figure 2. Unlike baseline methods that plateau after the third AL round, EvoDiff demonstrates a monotonic increase in AUC throughout all six AL rounds across all datasets, indicating that EvoDiff continues to acquire informative and discriminative samples even in later acquisition steps. Furthermore, EvoDiff exhibits smaller variance across runs, particularly in BUSI, BUS-BRA, and MMOTU, suggesting more stable learning behavior and resilience to sampling bias. The steeper upward trajectory of EvoDiff in Figure 2 reinforces its advantage in sustained knowledge acquisition, while other methods tend to over-exploit boundary uncertainty or suffer from sample redundancy.

The sustained performance improvement stems from a curriculum learning strategy within the EvoDiff framework that progressively transitions from generating easily learnable samples to challenging, informative samples as the classifier becomes more robust, thereby effectively preventing the performance plateaus commonly observed in conventional AL methods. In contrast, traditional uncertainty-based sampling often suffers from redundant selection of similar boundary samples after initial iterations, while diversity-based methods may favor outliers with limited learning utility. EvoDiff overcomes these limitations by maintaining a dynamic sampling policy that adapts to the model’s evolving knowledge state. Strategy-versus-Random comparisons with Benjamini–Hochberg correction are reported in Supplementary Table S1. EvoDiff achieved a higher AUC than random sampling in all 24 evaluated dataset and budget settings, and the difference reached q<0.05q < 0.05 in 10 of them. At the 50% cumulative annotation budget, EvoDiff comes closest to the fully supervised learning (FSL) upper bound on Fatty-Liver and Appendix, where the remaining accuracy gaps are 0.17 and 0.31 percentage points respectively. In terms of accuracy, EvoDiff remains below the FSL upper bound on all eight datasets, which shows that principled data acquisition can approach dense labeling in medical imaging pipelines at a fraction of the annotation cost.

Table S1. Comparison of ResNet-50 classification metrics between EvoDiff and other AL methods on the test sets. Values are reported as mean ±\pm s.d. over five folds.

Statistical differences between each active learning strategy and Random sampling were assessed using two-sided paired t-tests on fold-wise AUC, pairing the kk-th cross-validation fold of a strategy with the kk-th fold of Random. The reported qq-values are Benjamini–Hochberg-adjusted pp-values for AUC, with correction applied across the five strategy-versus-Random comparisons that share a given dataset and annotation budget. The appended (+) or (−) indicates the sign of the mean fold-wise AUC difference between the strategy and Random. (Bold represents the best result, and Underline represents the second best result).

DatasetMethodAnnotation BudgetAccbAccSensSpecPPVNPVF1AUCqq-value
Appendix[44]FSL (upper bound)100%0.6404±\pm0.01040.5024±\pm0.07450.7931±\pm0.06170.2118±\pm0.10910.6323±\pm0.04500.3697±\pm0.15430.7032±\pm0.04900.6390±\pm0.0948-
Appendix[44]Random10%0.4870±\pm0.11630.4933±\pm0.03270.8690±\pm0.05930.1176±\pm0.09840.6277±\pm0.01730.3033±\pm0.18090.7277±\pm0.02180.5154±\pm0.0383-
Appendix[44]VAAL[57]10%0.4891±\pm0.07630.4856±\pm0.02470.9241±\pm0.02580.0471±\pm0.04400.6233±\pm0.01220.2467±\pm0.20830.7444±\pm0.01460.5162±\pm0.05610.0940 (+)
Appendix[44]Core-Set[58]10%0.5096±\pm0.01490.4680±\pm0.02650.9241±\pm0.04570.0118±\pm0.02350.6144±\pm0.01340.0667±\pm0.13330.7379±\pm0.02360.5314±\pm0.05060.0940 (+)
Appendix[44]Max-Entropy[59]10%0.5277±\pm0.06450.4306±\pm0.05210.6966±\pm0.16230.1647±\pm0.15520.5839±\pm0.04430.1682±\pm0.14740.6288±\pm0.08670.5406±\pm0.02970.0171 (+)
Appendix[44]BALD[60]10%0.5145±\pm0.06920.4915±\pm0.04100.7241±\pm0.22560.2588±\pm0.17690.6187±\pm0.02680.4672±\pm0.27260.6523±\pm0.12440.5431±\pm0.06480.0982 (+)
Appendix[44]EvoDiff (Ours)10%0.5652±\pm0.02170.5136±\pm0.04430.7448±\pm0.17390.2824±\pm0.22750.6437±\pm0.02770.3058±\pm0.18320.6780±\pm0.07830.5962±\pm0.07980.0940 (+)
Appendix[44]Random20%0.4913±\pm0.06680.4850±\pm0.01980.8759±\pm0.12060.0941±\pm0.13720.6231±\pm0.01000.1417±\pm0.17400.7242±\pm0.04450.5232±\pm0.0810-
Appendix[44]VAAL[57]20%0.4936±\pm0.08690.4702±\pm0.04600.8345±\pm0.18940.1059±\pm0.11410.6092±\pm0.03020.3017±\pm0.36810.6980±\pm0.09010.5128±\pm0.07410.0981 (−)
Appendix[44]Core-Set[58]20%0.4858±\pm0.09180.4832±\pm0.01480.9310±\pm0.04360.0353±\pm0.04700.6221±\pm0.00710.1333±\pm0.16330.7455±\pm0.01610.5127±\pm0.07320.0007 (−)
Appendix[44]Max-Entropy[59]20%0.5556±\pm0.07940.4702±\pm0.06420.8345±\pm0.04020.1059±\pm0.13100.6161±\pm0.03690.1991±\pm0.20090.7079±\pm0.02890.5633±\pm0.08110.0981 (+)
Appendix[44]BALD[60]20%0.5769±\pm0.08700.4633±\pm0.03090.8207±\pm0.17500.1059±\pm0.13100.6060±\pm0.02550.1705±\pm0.15750.6907±\pm0.08850.5790±\pm0.03500.0981 (+)
Appendix[44]EvoDiff (Ours)20%………………………
Appendix[44]Random30%0.5391±\pm0.04220.4497±\pm0.03630.8759±\pm0.06390.0235±\pm0.04700.6043±\pm0.01950.0667±\pm0.04700.7174±\pm0.03260.5601±\pm0.0636-
Appendix[44]VAAL[57]30%0.5279±\pm0.04480.4531±\pm0.04730.9172±\pm0.06390.1529±\pm0.12670.6503±\pm0.02590.3600±\pm0.23860.7596±\pm0.03800.5546±\pm0.03800.0678 (−)
Appendix[44]Core-Set[58]30%0.5392±\pm0.05730.4797±\pm0.03380.9241±\pm0.03830.0353±\pm0.01880.6205±\pm0.00220.1300±\pm0.05000.7242±\pm0.01360.5860±\pm0.02420.0678 (−)
Appendix[44]Max-Entropy[59]30%0.5804±\pm0.04900.4858±\pm0.04520.9241±\pm0.02580.1647±\pm0.15520.6205±\pm0.02520.3486±\pm0.26340.7418±\pm0.02650.5974±\pm0.01580.0852 (+)
Appendix[44]BALD[60]30%0.5932±\pm0.13830.9241±\pm0.05520.0353±\pm0.07360.6222±\pm0.02250.1333±\pm0.01280.6018±\pm0.04050.7418±\pm0.02450.6071±\pm0.03860.0678 (+)
Appendix[44]EvoDiff (Ours)30%0.5978±\pm0.01090.4643±\pm0.03220.8345±\pm0.07360.0941±\pm0.02880.6100±\pm0.02010.2597±\pm0.06410.7042±\pm0.04000.6152±\pm0.05090.0509 (+)
BUSI[45]FSL (upper bound)100%0.9424±\pm0.02610.9483±\pm0.05170.9143±\pm0.05170.9822±\pm0.04870.9659±\pm0.04870.9640±\pm0.05730.9333±\pm0.04730.9867±\pm0.0239-
BUSI[45]Random10%0.7154±\pm0.03400.6441±\pm0.09580.5906±\pm0.26760.6978±\pm0.26550.5599±\pm0.14840.7924±\pm0.10640.5040±\pm0.16640.7967±\pm0.0309-
BUSI[45]VAAL[57]10%0.6822±\pm0.08950.7156±\pm0.09980.6000±\pm0.18950.8311±\pm0.11440.6522±\pm0.14930.8258±\pm0.11620.6069±\pm0.16510.7566±\pm0.03920.0339 (−)
BUSI[45]Core-Set[58]10%0.7110±\pm0.03620.7548±\pm0.03120.5429±\pm0.19660.8933±\pm0.07990.6790±\pm0.07990.8279±\pm0.03080.6096±\pm0.09260.7588±\pm0.02880.0245 (+)
BUSI[45]Max-Entropy[59]10%0.7449±\pm0.08600.7181±\pm0.09140.2000±\pm0.16050.8622±\pm0.16650.3273±\pm0.23230.7007±\pm0.23130.5941±\pm0.15210.8083±\pm0.09420.0942 (+)
BUSI[45]BALD[60]10%0.7292±\pm0.03620.5311±\pm0.03520.5429±\pm0.19660.8622±\pm0.09570.6906±\pm0.07990.8133±\pm0.15210.5914±\pm0.03660.8083±\pm0.06510.0051 (+)
BUSI[45]EvoDiff (Ours)10%0.8711±\pm0.08600.7506±\pm0.05560.6476±\pm0.14940.8533±\pm0.08800.6750±\pm0.08800.8440±\pm0.07880.6550±\pm0.07880.8355±\pm0.05350.0229 (+)
BUSI[45]Random20%0.7758±\pm0.09240.6978±\pm0.09780.4667±\pm0.08930.9289±\pm0.06340.8252±\pm0.11800.7981±\pm0.19870.5534±\pm0.19870.8036±\pm0.0259-
BUSI[45]VAAL[57]20%0.7228±\pm0.01460.7175±\pm0.04380.5238±\pm0.06020.9111±\pm0.11030.7529±\pm0.11030.8041±\pm0.01970.6130±\pm0.05220.7732±\pm0.02520.0869 (−)
BUSI[45]Core-Set[58]20%0.7567±\pm0.01860.7657±\pm0.01860.6381±\pm0.08830.8963±\pm0.08690.8432±\pm0.02260.8423±\pm0.02260.6813±\pm0.02260.8034±\pm0.01960.0799 (−)
BUSI[45]Max-Entropy[59]20%0.7592±\pm0.03630.7613±\pm0.07030.6381±\pm0.08830.8844±\pm0.12160.7780±\pm0.12160.8540±\pm0.10740.6583±\pm0.10740.8293±\pm0.02260.0179 (−)
BUSI[45]BALD[60]20%0.8318±\pm0.04680.8444±\pm0.04880.8000±\pm0.05550.8889±\pm0.07070.7829±\pm0.04990.9944±\pm0.04660.7826±\pm0.04660.8563±\pm0.03310.0081 (+)
BUSI[45]EvoDiff (Ours)20%0.8138±\pm0.03630.8444±\pm0.04480.8000±\pm0.08190.8889±\pm0.07070.7892±\pm0.04990.9944±\pm0.04660.7826±\pm0.04660.8563±\pm0.03310.0012 (+)
BUSI[45]Random30%0.8273±\pm0.04850.7549±\pm0.05550.5810±\pm0.10690.9284±\pm0.08360.7948±\pm0.05270.8265±\pm0.04350.6694±\pm0.04350.8600±\pm0.0283-
BUSI[45]VAAL[57]30%0.7892±\pm0.07970.7349±\pm0.10690.5810±\pm0.08190.8889±\pm0.08010.7184±\pm0.08010.8216±\pm0.06920.6351±\pm0.06920.8157±\pm0.03100.0947 (−)
BUSI[45]Core-Set[58]30%0.8260±\pm0.03780.7784±\pm0.08190.7429±\pm0.03870.8889±\pm0.08360.7615±\pm0.06860.8216±\pm0.06920.6351±\pm0.06920.8376±\pm0.03280.0947 (−)
BUSI[45]Max-Entropy[59]30%0.7941±\pm0.03780.7289±\pm0.10100.5810±\pm0.08190.5911±\pm0.20300.5275±\pm0.12690.8959±\pm0.10530.6463±\pm0.10530.8522±\pm0.01780.0947 (−)
BUSI[45]BALD[60]30%0.8232±\pm0.08710.8667±\pm0.08190.7429±\pm0.08190.8731±\pm0.01280.7792±\pm0.02690.8919±\pm0.04210.8053±\pm0.04210.8522±\pm0.01780.0947 (−)
BUSI[45]EvoDiff (Ours)30%0.8460±\pm0.02660.8571±\pm0.03020.8024±\pm0.03020.8731±\pm0.02690.7792±\pm0.02690.8919±\pm0.04210.8053±\pm0.04210.8746±\pm0.02450.0947 (+)

Table S1.

EvoDiff obtains these gains at a higher acquisition-side computational cost than uncertainty-based selection. For a representative acquisition round on BUS-BRA at the 10% cumulative annotation budget on a single NVIDIA A40, EvoDiff requires 6.8 GPU-hours, compared with 19.4 seconds for Max-Entropy and 86.5 seconds for BALD. Nearly all of this cost arises from information-gain-guided sampling, whereas synthetic-to-real retrieval takes approximately 34 seconds. Pretraining of the diffusion model and guidance classifier is a one-time cost shared across organs, datasets, and acquisition rounds. The acquisition step runs offline and requires no radiologist time. The stage-wise costs of all compared methods are reported in Supplementary Table S10.

MethodSelection model(s)Params (M)Cost per roundPeak GPU memory
Randomnone000
Max-EntropyResNet-18 proxy11.719.4 s66 MB
Core-SetResNet-18 proxy11.719.4 s66 MB
BALDResNet-18 proxy, 25 MC passes11.786.5 s66 MB
VAALVAE and discriminator23.6104.1 s1,997 MB
EvoDiff Stage 1 (pretraining)diffusion UNet and guidance classifier462one-time, amortizedN/A
EvoDiff Stage 2 (guided generation)diffusion UNet and guidance classifier4626.8 GPU-hours1,915 MB
EvoDiff Stage 3 (retrieval selection)ResNet-50 retrieval encoder25.6≈34\approx34 s123 MB
EvoDiff (all stages)UNet, guidance classifier, retrieval encoder487.66.8 GPU-hours1,915 MB

Table S10. Acquisition cost per round, measured on a single NVIDIA A40 for a representative round (BUS-BRA at a 10% budget). EvoDiff is resolved into its pipeline stages; Stage 1 pretraining is a one-time cost amortized across organs, datasets, and rounds, whereas Stages 2 and 3 recur at each round.

Validation Performance on Foundation-based Downstream Model

To evaluate the generalizability and scalability of EvoDiff beyond CNN architectures, we conducted comprehensive validation using USFM, a state-of-the-art Transformer-based US foundation model pre-trained on 2,187,915 US images of 12 common organs. This evaluation addresses a critical gap in current AL research, whether sample selection strategies optimized for traditional architectures can effectively transfer to large-scale pre-trained models with different inductive biases.

As demonstrated in Figure 3 and Supplementary Table S2, which reports accuracy, balanced accuracy, sensitivity, specificity, PPV, NPV, F1-score, and AUC for every method, dataset, and budget under USFM, EvoDiff consistently outperforms random sampling across most datasets under three cumulative annotation budgets (10%, 20%, 30%), establishing its effectiveness in fine-tuning foundation models under label-scarce conditions. The performance gains are particularly pronounced in balanced accuracy metric (bAcc), with the most substantial improvements observed in breast imaging tasks (Figure 3 (b)-(d)), where USFM possesses strong prior domain knowledge derived from its pre-training corpus comprising >90%> 90\% breast US images. Under a 30% annotation budget, EvoDiff improves bAcc by 9.37%↑ on BUSI and 8.04% ↑ on BUS-BRA compared to random sampling, indicating that EvoDiff sample selection effectively guides foundation models toward diagnostically meaningful training signals early in fine-tuning. Confusion matrices in Figure 3 show that EvoDiff shifts the operating point toward higher sensitivity on several datasets. Pooling the eight datasets and five folds into 40 paired observations at each budget, the mean sensitivity difference between EvoDiff and random sampling was +15.04, +11.17, and −0.13 percentage points at the 10%, 20%, and 30% budgets for ResNet-50, and +6.33, +8.41, and +2.29 percentage points for USFM. For example, on BUS-BRA at 10% cumulative annotation budget (Figure 3 (c)), the number of missed malignancies decreased from 25 cases with random sampling to 14 cases with EvoDiff, corresponding to a 44% reduction. A similar pattern is observed on TN3K, where sensitivity increases with the annotation budget. Because the sensitivity gain is not uniform across budgets and is not accompanied by a matching specificity gain, we report these differences as performance observations under the evaluated settings and do not interpret them as evidence of clinical safety. The method’s robustness extends to challenging imbalanced datasets, where conventional sampling strategies typically struggle with minority class recognition. In UDIAT and MMOTU, EvoDiff improves minority class recognition without sacrificing specificity. Notably, in UDIAT at 30% sampling, EvoDiff achieves perfect specificity while increasing sensitivity, showing that the curriculum learning strategy mitigates class imbalance bias during active selection.

Validation of USFM performance comparing random sampling with EvoDiff IG-guided curriculum learning across multiple ultrasound datasets

Figure 3. Validation of USFM performance comparing random sampling with EvoDiff IG-guided curriculum learning across multiple ultrasound datasets. Each row corresponds to one internal dataset. a) Appendix. b) BUSI. c) BUS-BRA. d) UDIAT. e) Fatty-Liver. f) MMOTU. g) TN3K. Within each row, the leftmost bar chart reports the balanced accuracy (bAccuracy) of the ultrasound foundation model (USFM) fine-tuned on samples selected by random sampling at cumulative annotation budgets of 10%, 20%, and 30%, drawn as light-to-dark red bars, and by EvoDiff at the same three budgets, drawn as light-to-dark blue bars, with the value printed above each bar. The six confusion matrices to the right of the bar chart, titled from random 10% to EvoDiff 30%, report the corresponding predictions on the fixed internal test set, with rows giving the true label and columns the predicted label for the two task-specific classes, normal versus abnormal or benign versus malignant. Matrix cells are shaded in blue, with darker shading indicating a larger number of samples, and the sample count is printed in each cell. IG, information gain.

These findings suggest a synergistic interaction between EvoDiff and USFM. First, the EvoDiff methodology’s sample selection through diffusion sampling demonstrates exceptional precision in identifying samples with maximal IG. While the USFM has been pre-trained on an extensive dataset, it provides strong anatomical priors that accelerate convergence. Together, they form a label-efficient training paradigm that generalizes across organs and imaging distributions. This intelligent sample curation process allows the foundation model to efficiently leverage its pre-learned representations while focusing computational resources on the most informative examples, resulting in rapid adaptation capabilities that facilitate seamless transfer to diagnostic tasks across different medical centers with varying imaging protocols and patient populations. Overall, EvoDiff delivers consistent and clinically meaningful performance gains over random sampling, reinforcing its value as a practical and scalable AL strategy for real-world foundation model deployment in medical US analysis.

Informative Sample Selection for Improved Foundation Model Generalizability

Recent studies suggest that foundation models can learn transferable representations through large-scale pretraining [61, 62, 63]. However, their generalizability to new clinical environments remains constrained by annotation quality and sample informativeness rather than dataset size alone. To investigate whether informative sample selection can unlock latent transferable capacity, we conducted a cross-domain external validation of USFM using the training data selected by EvoDiff and a randomly sampled baseline. As shown in Figure 4 and Supplementary Table S3, EvoDiff improves cross-domain performance in 42 of the 45 comparisons spanning fifteen source-to-target transfer settings and three annotation budgets, without any fine-tuning. The overall generalization gains demonstrate EvoDiff’s strong ability to support the rapid migration of the foundation model (USFM) to new domains.

Foundation model performance in cross-domain validation and annotation efficiency analysis

Figure 4. Foundation model performance in cross-domain validation and annotation efficiency analysis. a) External validation pipeline. The frozen EvoDiff model, marked by a snowflake symbol, selects informative samples from the internal source datasets (BUS-BRA, BUSI, UDIAT, and TN3K) for expert annotation, the ultrasound foundation model (USFM) is fine-tuned on the annotated samples, and the fine-tuned model is evaluated on four previously unseen external datasets (ALN-Ultra, BUS_UC, QAMEBI, and TN5000). b) Balanced accuracy of USFM on each external target dataset at annotation ratios of 10%, 20%, and 30%, arranged from the top row to the bottom row. Bars are grouped in pairs by source dataset, where the lighter bar of each pair gives the result without EvoDiff, that is with randomly sampled annotation, and the darker bar gives the result with EvoDiff-selected annotation. Orange bars denote BUS-BRA as the source dataset, green bars denote BUSI, blue bars denote UDIAT, and red bars denote TN3K, with the value printed above each bar. c) Model-based estimates of annotation cost across query rounds for the six organ tasks, computed using the mean unit annotation time within each experience stratum of the six-reader panel. The teal dashed line with circular markers shows the estimated annotation time for random sampling in the senior stratum, the dark blue dashed line shows the estimate for EvoDiff selection in the senior stratum, the orange dashed line shows the estimate for random sampling in the junior stratum, and the red dotted line shows the estimate for EvoDiff selection in the junior stratum. Error bars indicate the between-reader s.d. (standard deviation) within each stratum. The pale blue band and the pale orange band between the paired curves mark the estimated reduction in annotation workload with EvoDiff for the senior and junior strata, respectively.

Specifically, across the public external breast and thyroid targets, USFMs fine-tuned with EvoDiff-selected samples outperformed their randomly-trained counterparts in 29 of the 30 comparisons. The relative gains in bAcc differ by source and target rather than forming a single range. For the three source-to-BUS_UC breast evaluations they span 2.9% to 12.7%, for the source-to-ALN-Ultra and source-to-QAEMBI breast evaluations they reach 16.9%, and for TN3K→TN5000 the gain is 4.8%. At a 10% cumulative annotation budget, EvoDiff can improve transfer performance in most cases, indicating its effective generalization ability in the early stage. These gains typically amplify at the 20% and 30% annotation budgets, reflecting EvoDiff’s curriculum strategy of gradually exploring more informative samples over AL iterations.

Cross-domain transfer validation on the external breast targets showed the largest relative improvement for UDIAT→\toQAMEBI at 16.9%. Notably, EvoDiff also shows cross-domain generalization ability in the thyroid task (i.e., TN3K→\toTN5000), improving the bAcc from 0.6428 to 0.6737 at 30% annotation budget (4.8% ↑\uparrow). This further demonstrates EvoDiff’s applicability across different anatomies. The three private cohorts collected at Zhejiang Cancer Hospital were not accessible to the USFM pretraining corpus and therefore provide an overlap-free test of cross-domain transfer. On these cohorts EvoDiff improved balanced accuracy over random sampling in 13 of the 15 comparisons, with relative gains reaching 25.0% on PrivAppendix, 18.4% on PrivBreast and 18.3% on PrivLiver. Both exceptions arose in the UDIAT to PrivBreast setting, where the two strategies differed by 0.02 percentage points at the 10% annotation budget and random sampling was higher at the 30% budget. Because these cohorts were never exposed to the foundation model during pretraining, the observed generalization gains cannot be attributed to prior familiarity with the evaluation data.

The results above highlight two key conclusions. First, random sampling fails to unlock the inherent transferability of USFM because it cannot ensure the selection of high-quality and informative samples. Second, EvoDiff can act as a quality-controlled supervision mechanism, selecting domain-bridging and information-dense samples that promote cross-domain representation alignment. Consequently, EvoDiff addresses the critical bottleneck of sample informativeness and annotation efficiency, enabling foundation models to generalize without massive data scaling or costly target-domain fine-tuning. These findings further confirm that generalization is driven more by supervision quality than data quantity, positioning EvoDiff as a scalable and annotation-efficient solution for deployment across heterogeneous clinical sites.

Reader Study of Annotation Cost Reduction Across Multi-Organ Ultrasound Datasets

To evaluate EvoDiff’s clinical utility in reducing radiologist annotation burden, we conducted a reader study to estimate the Human Annotation Effort (HAE) and annotation workload savings (△\triangleHAE) of EvoDiff-selected annotation relative to random sampling at matched downstream performance. Both HAE and △\triangleHAE are model-based estimates that combine annotation counts at matched downstream performance with unit annotation times measured in the reader study, and the same unit annotation time is applied to both strategies. The study used randomized unlabeled subsets from our collected multi-organ US dataset and involved six board-certified radiologists spanning two experience strata, namely three senior readers with 7, 8, and 13 years of clinical experience and three junior readers with 2, 3, and 3 years of experience. Each reader independently reviewed US images and provided clinical diagnoses for the assigned cases within a fixed reading session. We recorded the number of correctly diagnosed samples and calculated the unit annotation cost for every reader, establishing baseline annotation performance at both experience levels.

Results revealed substantial variability in both accuracy and efficiency across readers and organ tasks, and the per-reader values are reported in Supplementary Table S6. Averaged within each experience stratum, senior readers reached 70.0% accuracy and junior readers 67.1% across the six tasks. Both strata performed best on liver, thyroid, breast, and carotid, with stratum means between 74.7% and 86.5%, and worst on ovary, where the stratum means were 36.0% for senior and 38.4% for junior readers. These findings expose critical realities in clinical annotation workflows. First, annotation quality depends more on the difficulty of the organ task than on reader experience, with both strata falling below 40% accuracy on ovary while exceeding 74% on liver, thyroid, breast, and carotid. Second, error-induced waste represents a hidden cost burden rarely acknowledged in deep learning literature.

Using Table 2 as performance benchmarks, we evaluated EvoDiff’s HAE effectiveness across multiple cumulative annotation budgets (5%, 10%, 20%, 30%, 40%, and 50%). For each cumulative annotation budget, we calculated the HAE required for random sampling to achieve equivalent performance to EvoDiff-optimized selection. This analysis yields model-based estimates of the total annotation workload and the corresponding workload savings (△\triangleHAE) for achieving comparable diagnostic accuracy with random sampling and with EvoDiff selection. We plotted the estimated annotation costs separately for the senior and junior strata. Figure 4c shows the estimated HAE across AL rounds for random sampling and for EvoDiff selection in each stratum, in which the shaded regions represent the estimated workload saving (△\triangleHAE) for each stratum. And Supplementary Table S4 shows the corresponding numerical results. The estimated annotation workload is lower with EvoDiff than with random sampling in both experience strata, in all six organ tasks and at every cumulative annotation budget. Under the common unit-time assumption, the estimated relative reduction is identical in the senior and junior strata. The estimated absolute time saving is larger in the junior stratum, reflecting its longer measured time per correctly annotated image in this panel. At the 5% cumulative annotation budget, the estimated workload reduction on the carotid artery task reaches 77% for both the junior and the senior stratum. Under the 10% cumulative annotation budget, the estimated workload for breast tumor diagnosis is 58% lower with EvoDiff for both senior and junior readers. Moreover, the estimated workload reduction is present at every cumulative annotation budget in both strata, including later AL stages when numerous diverse samples had been labeled. These estimates indicate that EvoDiff’s sample selection strategy reduces the annotation workload required to reach a given diagnostic accuracy. The consistent pattern of estimated HAE reduction across all six anatomical regions supports the applicability of EvoDiff’s label efficiency across diverse US applications.

Ablation Study of EvoDiff Framework

To assess the contribution of each proposed component in the EvoDiff framework, we conduct ablation experiments on two representative datasets, i.e., Appendix with the fewest training samples, and TN3K contains the largest. Therefore, the Appendix dataset reflects a highly data-scarce scenario, while TN3K provides a data-rich environment. This specific design allows us to rigorously evaluate both the effectiveness and scalability of EvoDiff under varying annotation budgets. As shown in Table 3 and Table 4, the framework is progressively assembled from three key components, including IG Guidance, Local Optimization Evolutionary Algorithm (LOEA), and Curriculum Learning (CL). All experiments are conducted under 10%, 30%, and 50% cumulative annotation budgets, using Accuracy (ACC), F1-score, and AUC as evaluation metrics.

IG GuidanceLOEACLAnnotation Ratio=10%Annotation Ratio=30%Annotation Ratio=50%Upper Bound
ACCF1-scoreAUCACCF1-scoreAUCACCF1-scoreAUCACCF1-scoreAUC
✓××0.53860.49380.53050.55030.52650.56270.55870.52940.5846–––
✓✓×0.53830.49560.53470.54790.52940.55180.55920.53190.5731–––
✓✓✓0.56520.53740.59600.59780.57610.61530.63730.60950.64740.64040.70320.6390

Table 3. Ablation study on the Appendix dataset[44] under 10%, 30%, and 50% cumulative annotation budgets. Each reduced configuration was compared with the complete EvoDiff framework using two-sided paired tt-tests, with the five cross-validation folds used as matched pairs. The Upper Bound column reports fully supervised learning on the complete labeled set.

IG GuidanceLOEACLAnnotation Ratio=10%Annotation Ratio=30%Annotation Ratio=50%Upper Bound
ACCF1-scoreAUCACCF1-scoreAUCACCF1-scoreAUCACCF1-scoreAUC
✓××0.65920.60510.65600.75310.68590.78030.80430.72750.8092–––
✓✓×0.66310.63360.69580.78940.70470.78510.83830.75520.8365–––
✓✓✓0.70480.67290.71250.81260.74030.82890.86560.78490.85190.89370.76210.8927

Table 4. Ablation study on the TN3K dataset[53] under 10%, 30%, and 50% cumulative annotation budgets. Each reduced configuration was compared with the complete EvoDiff framework using two-sided paired tt-tests, with the five cross-validation folds used as matched pairs. The Upper Bound column reports fully supervised learning on the complete labeled set.

On the Appendix dataset (see Table 3), the baseline configuration, which includes only IG Guidance, achieved modest performance with F1-score of 0.4938, 0.5265, and 0.5294 at 10%, 30%, and 50% cumulative annotation budgets, respectively. Incorporating LOEA can yield minimal improvement at lower cumulative annotation budgets (i.e., 0.4938 v.s. 0.4956 at 10% annotation), but showed better performance at higher cumulative annotation budgets (i.e., 0.5294 v.s. 0.5319 at 50% annotation). The incorporation of CL produced notable performance improvements across all metrics. The configuration without CL differed significantly from the complete EvoDiff framework (p<0.01p < 0.01, two-sided paired tt-test with the five cross-validation folds used as matched pairs). This validates that progressive sampling strategies effectively stabilize learning dynamics in low-data conditions. The full EvoDiff model (the last row in Table 3) consistently achieves the highest performance (e.g., 0.5652 ACC and 0.5960 AUC at 10% annotation), with improvements over the IG-only baseline reaching statistical significance (p<0.01p < 0.01, two-sided paired tt-test with the five cross-validation folds used as matched pairs), demonstrating its robust capability under limited supervision. Similar improvement pattern is observed on the TN3K dataset (see Table 4), where LOEA alone offers incremental gains, while CL significantly enhances performance. With 50% annotations, EvoDiff reaches satisfactory ACC (0.8656), F1-score (0.7849), and AUC (0.8519), substantially closing the gap toward the fully supervised performance. These findings demonstrate that EvoDiff maintains consistent advantages in both label-scarce and data-rich settings.

Information Gain-Guided Synthesis Results

To gain deeper insight into the IG-guided generative behavior of EvoDiff and its role in facilitating progressive downstream classifier learning under limited annotations, we further visualize US images generated under varying guidance strengths. This experiment not only examines feature fidelity across anatomical categories but also reveals how EvoDiff constructs a curriculum learning mechanism at the data level through controllable difficulty modulation. As illustrated in Figure 5, representative images were synthesized from multiple organs, including appendix, breast, carotid artery, liver, ovary and thyroid, under five guidance strength (GS) settings ranging from 1 to 9.

EvoDiff-generated synthetic ultrasound probes across varying guidance strengths

Figure 5. EvoDiff-generated synthetic probes across varying guidance strengths. Each row shows synthetic ultrasound images generated for one organ and class, and the five columns correspond to guidance strengths (GS) of 1, 3, 5, 7, and 9. In the row labels, N denotes normal, A denotes abnormal, B denotes benign, M denotes malignant, and CVE(+) and CVE(-) denote cases with and without a follow-up cardiovascular event. Progressive intensification of the information-gain (IG) guidance across active learning (AL) iterations creates a curriculum learning framework in which sample difficulty systematically increases, moving generation from typical appearances toward diagnostically ambiguous appearances.

The sampling process in EvoDiff is regulated by IG gradient guidance, which directs diffusion trajectories toward semantically informative regions of the feature space, while controlled stochastic perturbations preserve sample diversity. In parallel, LOEA performs non-backpropagation-based refinements, enabling EvoDiff to maintain stability during iterative AL rounds. At lower guidance strengths (GS = 1 or 3), synthesized images exhibit clear intra-class coherence and strong expression of discriminative pathology-related attributes. For example, benign and malignant lesions in breast US remain visually separable, and vessel wall continuity in carotid cases is preserved, demonstrating that EvoDiff can accurately model early-stage class structure with high semantic clarity.

As guidance strength increases, however, we observe a progressive transformation in synthesized samples, that intra-class distinctions gradually become more ambiguous, while inter-organ anatomical characteristics remain identifiable. This controlled blurring of class-specific pathological cues can be attributed to the fact that stronger guidance steers generation toward higher-entropy regions in the classifier’s decision space, effectively emphasizing samples near classification boundaries. Notably, this transition from semantically clear to increasingly ambiguous examples is not a degradation of generative quality but rather a deliberate mechanism that enables curriculum learning within the AL pipeline. Specifically, low-GS samples act as “easy (certain)” synthetic queries that support initial classifier convergence, whereas higher-GS samples introduce “hard (uncertain)” cases that stimulate discriminative refinement in subsequent learning rounds.

This visualization confirms two critical properties of EvoDiff. First, the framework has the potential to control the certainty of synthesized data, enabling the construction of a continuous curriculum where sample informativeness gradually increases alongside model capability. Second, the integration of IG guidance and LOEA ensures that as annotation rounds progress and additional supervision becomes available, EvoDiff continues to produce high-information synthetic probes that remain informative for sample retrieval rather than collapsing into trivial noise. Consequently, the progressive increase in guidance strength aligns naturally with the iterative structure of AL, contributing to a stable and monotonic improvement in model performance as more annotations are introduced. These results reinforce the advantage of EvoDiff as a controllable, information-aware synthesis framework that enhances AL efficiency in medical US imaging scenarios constrained by annotation scarcity.

The synthetic images serve exclusively as retrieval probes and are neither annotated nor used for downstream training. We nevertheless assessed whether they preserve pathology-related information and a realistic US appearance. Diagnostic ResNet-50 models trained exclusively on real images recognized the synthetic probes with a pooled AUC between 0.733 and 0.966 across the six organ tasks (Supplementary Table S7). In a blinded assessment by four radiologists, the pooled accuracy of real-versus-synthetic identification was 45%, close to the 50% chance level (Supplementary Table S8), and synthetic diseased images received higher mean diagnostic scores than synthetic non-diseased images in all six organ tasks (Supplementary Table S9 and Supplementary Figures S9 and S10).

Guidance-Driven Curriculum Learning Sample Analysis and Uncertainty Interpretation

Figure 6 provides a comprehensive visualization of the EvoDiff framework selecting information-rich samples from a diverse multi-organ US dataset for downstream model training, and the evolutionary dynamics of classification uncertainty in multi-organ classification tasks. The visualization results show that as EvoDiff constructs a curriculum learning dataset through IG-guided sampling and increasing guided strength values, it optimizes the feature learning of the downstream model, resulting in a systematic reduction in the prediction uncertainty of the downstream model as the number of AL rounds increases.

Systematic uncertainty reduction in downstream model predictions across multi-organ ultrasound test sets

Figure 6. Systematic uncertainty reduction in downstream model predictions across multi-organ ultrasound test sets. Uncertainty maps are shown for six anatomical sites (appendix, breast, carotid, liver, ovary, and thyroid) with two representative cases per organ, where the suffix 0 or 1 in each row label gives the binary ground-truth label, benign 0 and malignant 1, or normal 0 and abnormal 1. Within each half of the figure, the first column shows the original test image, the second column shows the uncertainty map of the initialized model, and the following three columns show the maps after active learning (AL) rounds 1, 3, and 5, generated with guidance strengths of 1, 5, and 9, respectively. Each map is coloured from dark purple for an uncertainty of 0, through blue and green, to yellow for an uncertainty of 1, as indicated by the colour bars, and the shrinking bright regions from left to right illustrate decreasing predictive uncertainty.

The uncertainty maps reveal distinct organ-specific patterns of information extraction. In the appendix cases, we observe initially diffuse uncertainty distributions that progressively concentrate toward anatomically relevant regions. Notably, the abnormal appendix case shows more rapid uncertainty reduction compared to normal label, suggesting that malignant/abnormal cases may contain more distinctive features that facilitate faster model convergence. Similar differential learning patterns are evident in breast tissue samples, where the uncertainty concentrations progressively shift from globally distributed to locally focused regions, likely corresponding to diagnostically relevant anatomical structures. The carotid artery uncertainty maps exhibit a particularly interesting transformation, with initial broad uncertainty that gradually localizes to vessel wall boundaries, which are critical regions for pathological assessment.

The color-coded uncertainty scale (0.0-1.0) illustrates how the guidance strength parameter (increasing from 1 to 9) systematically modulates the uncertainty landscape. At iteration 1 with minimal guidance (strength=1), uncertainty patterns appear diffuse and relatively uniform across organ types. By iteration 3 (guidance strength=5), we observe a critical transition phase where uncertainty begins to concentrate in diagnostically informative regions, with noticeable differentiation between benign/normal and malignant/abnormal cases. By iteration 5 (guidance strength=9), uncertainty has converged to minimal, highly localized patterns, particularly evident in thyroid and ovary samples where anatomical complexity typically presents diagnostic challenges. This convergence pattern validates our hypothesis that the EvoDiff framework effectively navigates the solution space toward optimal annotation subsets through iterative refinement.

The visualization reveals that different organ systems exhibit varying rates of uncertainty reduction, reflecting their inherent complexity and feature distinctiveness. Liver and thyroid samples demonstrate relatively slower uncertainty convergence, likely due to their heterogeneous parenchymal textures that present more complex feature spaces for model interpretation. In contrast, ovarian and carotid samples show more efficient uncertainty reduction, suggesting greater feature discriminability in these anatomical contexts. This visualization empirically validates our theoretical framework by demonstrating how the integration of evolutionary algorithms with diffusion sampling progressively identifies the most informative regions for annotation across diverse anatomical contexts, thus enabling truly adaptive and organ-agnostic AL for US image classification.

Discussion

Progress in clinical deep learning continues to be constrained by the scarcity of high-quality annotations and the absence of principled strategies for prioritizing informative training samples [64, 65]. Conventional CNN-based medical AI models rely heavily on dense supervision yet lack mechanisms to identify which images contribute most to model refinement. Although recent foundation models offer transferable representations across tasks and modalities, their adaptation to new clinical settings still demands high-information data, limiting scalability in real-world practice[18, 41]. These fundamental challenges are particularly pronounced in US imaging, where technical limitations intersect with unique workflow characteristics to create compounding barriers for supervised learning. From a technical perspective, operator-dependent acquisition, heterogeneous scanning protocols, and pervasive speckle noise hinder reliable model training. However, the most critical challenge stems from US’s distinct clinical workflow. The temporal separation between image acquisition and diagnostic labeling creates an “image-label separation” phenomenon, where individual frames often lack direct diagnostic associations at the time of storage[66, 67]. This labeling challenge is exacerbated by US’s inherent data redundancy and imbalance. Sequential scanning generates overlapping visual content, while normal anatomical variations vastly outnumber pathological cases. Consequently, traditional full-dataset training approaches become suboptimal, as models may overfit to redundant normal samples while failing to adequately learn from rare but clinically critical pathological cases.

To address these limitations, we introduce EvoDiff, a generative AL framework designed to improve label efficiency by integrating data quality awareness with information-driven sampling[68]. By systematically identifying high-information samples and reducing reliance on labeled data, EvoDiff addresses both the technical constraints of US imaging and the workflow-specific challenges of clinical practice. Importantly, EvoDiff remains compatible with both classical CNN workflows and foundation model architectures, enabling its adoption across diverse clinical pipelines without requiring fundamental infrastructure changes. EvoDiff reconceptualizes AL as a diffusion-based feature space optimization process rather than a heuristic query selection problem. By embedding IG directly into the generative process, the framework steers sampling along entropy-increasing gradients with respect to the sampling state, without any parameter update to the pretrained networks. In contrast to uncertainty-based strategies that passively depend on posterior variance, EvoDiff actively explores the latent feature manifold to identify samples expected to provide maximal information. An evolutionary algorithm mitigates premature convergence to suboptimal regions of the data distribution, while a curriculum learning-inspired progression from low- to high-uncertainty samples promotes stable model optimization under sparse supervision. Together, these components establish a principled formulation of AL as a controllable generative search process. We next evaluated how this theoretical advantage translates to empirical performance across established AL strategies such as margin sampling and entropy-based querying, EvoDiff alleviates the instability that arises from unreliable posterior probability estimates, a problem that is particularly acute in the US due to artifacts and class imbalance[69]. In contrast to clustering and diversity-based selection methods, which may inadvertently select redundant samples, EvoDiff synthesizes semantic probes that retrieve previously underrepresented yet informative regions of the data distribution.

Beyond these methodological advantages, EvoDiff exhibited broad architectural compatibility, consistently enhancing performance across both CNN-based models and Transformer-based foundation models. Experimental validation on ResNet-50 revealed superior performance compared to four baseline methods across 6 benchmark tasks encompassing 8 datasets. Under low annotation resource conditions (10% annotation), the data selected by EvoDiff achieved an accuracy of 79.51% on the downstream cardiovascular event prediction task, while the comparative method BALD only reached 74.03%. Under higher annotation resource scenarios, such as 50% cumulative annotation budget, the curriculum learning design of EvoDiff ensures consistent performance improvements for downstream models. Furthermore, in the fatty liver diagnosis task, data selected by EvoDiff achieved an accuracy of 97.24% with 50% cumulative annotation budget, approaching the upper bound of FSL results at 97.41% accuracy. Similar findings were observed in the appendicitis diagnosis task, where EvoDiff achieved 63.73% accuracy with 50% cumulative annotation budget, exhibiting only a 0.31 percentage point accuracy degradation compared to the fully supervised results. To further validate the generalization capabilities, external experimental evaluation demonstrated that EvoDiff substantially enhanced USFM’s cross-domain performance, particularly in cross-center evaluations, indicating robust resilience to domain shift and reliable sample selection across heterogeneous clinical cohorts. On three unseen external breast validation sets, fine-tuning USFM with EvoDiff-selected samples yielded generalization gains ranging from 2.9% to 16.9%. For the thyroid dataset, when generalizing from TN3K to the TN5000 dataset, EvoDiff achieved a 4.8% improvement. This demonstrates that EvoDiff can serve as an effective approach in the medical US imaging domain to reduce annotation costs while automatically excluding noisy annotations and enhancing the generalization capability of foundation models on unseen domains. Collectively, these findings establish EvoDiff as a versatile and transferable AL paradigm that is not constrained by model architecture or training paradigm. In addition to improving the diagnostic accuracy of downstream models, EvoDiff offers practical value for real-world annotation workflows. EvoDiff employs a curriculum guided sampling strategy that aligns with clinical reasoning by first selecting high-confidence cases and then progressively introducing diagnostically challenging cases. This learning trajectory stabilizes early optimization and improves annotation usefulness. At matched downstream performance, EvoDiff is estimated to require 24% to 84% fewer annotations than random sampling. The corresponding workloads in minutes are estimated using unit annotation times measured in the six-reader panel (Supplementary Table S4). This estimated reduction reflects the smaller number of annotations required, as EvoDiff preferentially selects samples that maximize model learning rather than those that are merely ambiguous.

Beyond accuracy, a practical motivation for label-efficient learning in ultrasound is what we term image-label separation. Unlike static modalities such as CT or MRI, ultrasound examination proceeds through real-time scanning, storage of key frames, and case-level report generation only after all stored images have been reviewed[66]. Individual frames therefore carry no frame-level diagnostic label at the time of storage. In a thyroid examination a radiologist may store 15 to 35 images across bilateral transverse and longitudinal views, yet the report records a single case-level conclusion, so only 3 to 5 images directly demonstrate the finding while the remainder cannot be associated with that label without expert annotation. Hospital information systems compound the problem, because imaging archives and diagnostic reports are typically held in separate systems under different access permissions.

Active learning also remains useful when labels already exist, because ultrasound data are highly redundant. Consecutive frames in dynamic acquisition overlap substantially in content, standard views of the same organ repeat structurally, and normal studies vary little across patients, which leaves datasets in long-tailed distributions[67]. Training on the full archive therefore spends computation on repeated patterns, amplifies class imbalance, and lowers annotation consistency when large batches are labeled at once. EvoDiff addresses both settings by selecting the informative minority of an archive, which reduces expert labeling burden while preserving downstream performance.

Despite its advantages, EvoDiff has limitations. First, the framework relies on a frozen feature encoder during the hybrid generation-selection process. Although this encoder is used solely for representation embedding rather than for universal uncertainty estimation across downstream classifiers, potential embedding shifts may still influence sample prioritization. Second, EvoDiff is currently validated on 2D B-mode US images; future work is required to extend the framework to MRI and CT, where spatiotemporal coherence may alter AL dynamics. Third, EvoDiff involves a trade-off between downstream performance gains and acquisition time. In the representative BUS-BRA benchmark reported in Supplementary Table S10, one EvoDiff acquisition round requires 6.8 GPU-hours on a single NVIDIA A40, compared with 19.4 seconds for Max-Entropy and 86.5 seconds for BALD. This acquisition step runs offline and requires no radiologist time. The present implementation incorporates DDIM acceleration. Future work could investigate reduced-step sampling, diffusion distillation, or lightweight generative adapters to reduce acquisition time and improve scalability. Fourth, the 24% to 84% reduction in annotation workload is estimated with the HAE metric, which combines annotation counts at matched downstream performance with unit annotation times measured in the six-reader panel. For each reader and organ task, the same unit annotation time is applied to both strategies, an assumption that follows from diagnostic classification annotation, in which every image receives one categorical label. Future work could develop more fine-grained measures of human annotation effort that account for sample-level variation in annotation difficulty, confidence, and decision time.

Looking forward, EvoDiff opens promising directions for generative AL. Future extensions may include learning adaptive guidance schedules instead of a fixed curriculum, enabling the sampler to dynamically adjust difficulty based on downstream classifiers’ uncertainty. Another promising avenue is integrating EvoDiff with segmentation tasks to reduce annotation time. Although the present study focuses on the US, the framework is intrinsically modality agnostic and may be extended to other medical imaging domains, such as MRI and CT, where acquisition variability and annotation scarcity continue to limit scalability. In summary, EvoDiff reframes AL as a process of guided generative exploration rather than passive sample selection. By integrating IG with diffusion priors and curriculum sampling, the framework offers a principled strategy for label-efficient model development. Its consistent benefits across CNN and foundation model architectures, together with its demonstrated annotation efficiency, indicate strong potential for enabling data-centric clinical AI under real-world constraints.

Methods

Overview

As illustrated in Figure 7, our EvoDiff framework follows a two-phase training scheme: (1) Organ-conditioned coarse pretraining. EvoDiff and a guidance classifier are pretrained on DiU\mathcal{D}^{U}_{i} with organ identity as the only conditioning signal, so that anatomical representations are learned without any pathology-level annotation. The diffusion denoising objective itself remains self-supervised, because its regression target is the injected noise. (2) Full supervised fine-tuning. The pretrained guidance classifier will be incorporated as part of the classifier-guided EvoDiff architecture for joint fine-tuning on DiL\mathcal{D}^{L}_{i} using class-level pathology annotations to capture diagnostic features. During sampling, we apply varying guidance strengths for generating the Synthetic set DiG={(xji,pji)}j=1G\mathcal{D}^{G}_{i} = \{(x^{i}_{j},p^{i}_{j})\}_{j=1}^{G}, where pjip^{i}_{j} denotes pseudo-labels, and GG is the synthetic sample count (set to 100 to speed up the denoising process). Subsequently, we compute the cosine similarity between the generated samples and unlabeled data in DiU\mathcal{D}^{U}_{i}, and select the most informative samples for expert annotation. This forms the Actively annotated set DiA={(xki,yki)}k=1Z\mathcal{D}^{A}_{i} = \{(x^{i}_{k},y^{i}_{k})\}_{k=1}^{Z}, where ZZ denotes the constrained number of annotated samples. EvoDiff therefore follows a generate-to-retrieve paradigm. The synthetic set DiG\mathcal{D}^{G}_{i} serves only as a set of retrieval probes, and its members are neither annotated by experts nor added to the downstream training data. Only the retrieved real samples in DiA\mathcal{D}^{A}_{i} are annotated and used for downstream fine-tuning. This process is iteratively applied to each dataset. Downstream models are then finetuned on the EvoDiff-selected labeled data DiA\mathcal{D}^{A}_{i}. The downstream CNN experiments use one independently trained ResNet-50 per dataset rather than a unified pan-organ model or one model per organ, and no model parameters are shared across datasets or organs. For each dataset the ResNet-50 is initialized from the same ImageNet-pretrained weights, so that the acquisition strategy is the only factor that differs across the compared methods.

Model Pre-training

In order to fully utilize the large amount of data without diagnostic labels DiU\mathcal{D}^{U}_{i} and the small amount of data with diagnostic labels DiL\mathcal{D}^{L}_{i}, the training process was divided into two phases, namely (1) an organ-conditioned coarse pretraining phase and (2) a pathology-level supervised fine-tuning phase. However, both training objectives are learning discriminative US image features. Inspired by[70], we optimize guidance classifier’s loss and then used a loss function to train the diffusion model as follow:

Ldiff=∑tEx0,ε[∥ε−εθ(αˉtx0+1−αˉtε,y,t)∥2]+0.001Lvlb.(1)L_{\mathrm{diff}}=\sum_t\mathbb{E}_{x_0,\varepsilon}\left[\left\|\varepsilon-\varepsilon_\theta\left(\sqrt{\bar{\alpha}_t}x_0+\sqrt{1-\bar{\alpha}_t}\varepsilon,y,t\right)\right\|^2\right]+0.001L_{\mathrm{vlb}}. \tag*{(1)}

To facilitate category-based feature distinguish, which serves as the foundation for IG-based gradient sampling, we adopted the approach proposed by Luo et al.¹ to train a classifier fϕ(xt,t)f_\phi(x_t,t). This guidance classifier architecture is based on the downsampling trunk of the U-Net model augmented with an attention pooling mechanism. The training loss function is formulated as follows:

Lguidance_classifier=∑tEx0,ε[∥y−fϕ(xt,t)∥c],(2)L_{\mathrm{guidance\_classifier}}=\sum_t\mathbb{E}_{x_0,\varepsilon}\left[\left\|y-f_\phi(x_t,t)\right\|_c\right], \tag*{(2)}

where ∥⋅∥c\left\|\cdot\right\|_c represents the cross-entropy loss function.

Information-Guided Diffusion Sampling

Houlsby et al.⁷² proposed an information-theoretic AL approach that approximates mutual information scores through the inconsistency of Bayesian network parameter posterior distributions, seeking data points that maximize IG. Inspired by this information-theoretic principle, we distinguish the ideal Bayesian information-gain objective from a tractable trajectory-local information-gain surrogate used during classifier-guided Denoising Diffusion Implicit Model (DDIM) sampling. At each reverse-diffusion step, the trajectory-local surrogate evaluates the predictive-entropy gain of an LOEA-refined candidate relative to the current diffusion state at the same timestep. This criterion provides a tractable mechanism for steering continuous latent-space sampling toward more informative regions while keeping the diffusion model and guidance-classifier parameters fixed during sampling. Ultimately, the generated images serve as information-guided synthetic probes that direct retrieval toward corresponding real samples in the unlabeled candidate pool; the probes themselves are not annotated or used for downstream training.

According to the Denoising Diffusion Probabilistic Model (DDPM), the forward diffusion process progressively adds random Gaussian noise to data distribution q(x0)q(x_0) through a fixed Markov chain:

q(x1:T∣x0)=∏t=1Tq(xt∣xt−1),(3)q(x_{1:T}\mid x_0)=\prod_{t=1}^{T}q(x_t\mid x_{t-1}), \tag*{(3)}
q(xt∣xt−1)=N(xt;αtxt−1,(1−αt)I),(4)q(x_t\mid x_{t-1})=\mathcal{N}\left(x_t;\sqrt{\alpha_t}x_{t-1},(1-\alpha_t)I\right), \tag*{(4)}

where xtx_t is original data xx after tt timesteps noise added, αt=1−βt\alpha_t=1-\beta_t is the noise variance schedule factor, βt\beta_t is the hyperparameter that controls the noise variance increment. At each step, the noise follows N(0,βtI)\mathcal{N}(0,\beta_tI).

Equation (4) indicates that at each timestep, xtx_t is sampled from a Gaussian distribution, which is scaled of the previous timestep xt−1x_{t-1}, with added noise. Therefore, we can directly compute xtx_t from the initial state x0x_0 at any arbitrary timestep tt using the following closed-form expression:

xt=αˉtx0+1−αˉtε,ε∼N(0,I),(5)x_t=\sqrt{\bar{\alpha}_t}x_0+\sqrt{1-\bar{\alpha}_t}\varepsilon,\qquad\varepsilon\sim\mathcal{N}(0,I), \tag*{(5)}

where ε\varepsilon represents noise. And the reverse denoising process occurs through network εθ(xt,t)\varepsilon_\theta(x_t,t) predicting added noise ε\varepsilon to estimated x0x_0.

We use deterministic DDIM sampling with σt=0\sigma_t=0 and the cumulative schedule αˉt=∏s=1tαs\bar{\alpha}_t=\prod_{s=1}^{t}\alpha_s, with αˉ0=1\bar{\alpha}_0=1. The raw diffusion-model prediction and IG-guided noise estimate are distinguished as

εtraw=εθ(xt,t,y),ε~t=εtraw−1−αˉt gt,(6)\varepsilon_t^{\mathrm{raw}}=\varepsilon_\theta(x_t,t,y),\qquad\widetilde{\varepsilon}_t=\varepsilon_t^{\mathrm{raw}}-\sqrt{1-\bar{\alpha}_t}\,g_t, \tag*{(6)}

where gtg_t is the sampling-state guidance vector defined below. The guided estimate ε~t\widetilde{\varepsilon}_t is used consistently in both parts of the deterministic transition:

x^0=xt−1−αˉt ε~tαˉt,xt−1=αˉt−1x^0+1−αˉt−1 ε~t.(7)\widehat{x}_0=\frac{x_t-\sqrt{1-\bar{\alpha}_t}\,\widetilde{\varepsilon}_t}{\sqrt{\bar{\alpha}_t}},\qquad x_{t-1}=\sqrt{\bar{\alpha}_{t-1}}\widehat{x}_0+\sqrt{1-\bar{\alpha}_{t-1}}\,\widetilde{\varepsilon}_t. \tag*{(7)}

No additional Gaussian-noise term is introduced during the reverse update.

Previous BALD method demonstrates that the core objective of information theory-based AL is to reduce the number of possible hypotheses as rapidly as possible. Accordingly, we formulate an ideal Bayesian information-gain objective to express Algorithm 1 Information Gain-Guided Diffusion Sampling

Input: diffusion model εθ\varepsilon_\theta; guidance classifier fϕf_\phi; guidance strength λ\lambda; LOEA; number of diffusion timesteps TT; cumulative DDIM noise schedule {αˉt}t=0T\{\bar{\alpha}_t\}_{t=0}^{T} with αˉ0=1\bar{\alpha}_0=1

Output: x0x_0: generated synthetic probe

  1. Sample xT∼N(0,I)x_T \sim\mathcal{N}(0,I)

  2. for t=T,…,1t=T,\ldots,1 do Step 1: Trajectory-local information-gain evaluation

    1. pt←softmax⁡(fϕ(xt,t))p_t \leftarrow\operatorname{softmax}(f_\phi(x_t,t))

    2. Hϕ(Y∣xt,t)←−∑c=1Cpt,clog⁡pt,cH_\phi(Y\mid x_t,t) \leftarrow-\sum_{c=1}^{C}p_{t,c}\log p_{t,c}

    3. zt∗←LOEA⁡(xt,t)z_t^* \leftarrow\operatorname{LOEA}(x_t,t)

    4. pt∗←softmax⁡(fϕ(zt∗,t))p_t^* \leftarrow\operatorname{softmax}(f_\phi(z_t^*,t))

    5. Hϕ(Y∣zt∗,t)←−∑c=1Cpt,c∗log⁡pt,c∗H_\phi(Y\mid z_t^*,t) \leftarrow-\sum_{c=1}^{C}p_{t,c}^*\log p_{t,c}^*

    6. IG⁡tlocal(zt∗;xt)←Hϕ(Y∣zt∗,t)−Hϕ(Y∣xt,t)\operatorname{IG}_t^{\mathrm{local}}(z_t^*;x_t) \leftarrow H_\phi(Y\mid z_t^*,t)-H_\phi(Y\mid x_t,t)

    Step 2: Sampling-state guidance

    1. Lguide(xt)←λIG⁡tlocal(zt∗;xt)\mathcal{L}_{\mathrm{guide}}(x_t)\leftarrow\lambda\operatorname{IG}_t^{\mathrm{local}}(z_t^*;x_t)

    2. gt←∇xtLguide(xt)g_t\leftarrow\nabla_{x_t}\mathcal{L}_{\mathrm{guide}}(x_t)

    3. εtraw←εθ(xt,t,y)\varepsilon_t^{\mathrm{raw}}\leftarrow\varepsilon_\theta(x_t,t,y)

    4. ε~t←εtraw−1−αˉt gt\tilde{\varepsilon}_t\leftarrow\varepsilon_t^{\mathrm{raw}}-\sqrt{1-\bar{\alpha}_t}\,g_t

    Step 3: Deterministic DDIM update

    1. x^0←xt−1−αˉt ε~tαˉt\hat{x}_0\leftarrow\frac{x_t-\sqrt{1-\bar{\alpha}_t}\,\tilde{\varepsilon}_t}{\sqrt{\bar{\alpha}_t}}

    2. xt−1←αˉt−1x^0+1−αˉt−1 ε~tx_{t-1}\leftarrow\sqrt{\bar{\alpha}_{t-1}}\hat{x}_0+\sqrt{1-\bar{\alpha}_{t-1}}\,\tilde{\varepsilon}_t

    3. end for

    4. return x0x_0

the information-seeking principle underlying EvoDiff. Let Φ\Phi denote the random guidance-classifier parameters under the Bayesian posterior, ϕ\phi the fixed guidance-classifier parameters used during practical sampling, and DL\mathcal{D}_L the labeled data available at the current AL stage. For an intermediate diffusion state xtx_t, the ideal Bayesian IG objective is defined as

IG⁡∗(xt)=I(Y;Φ∣xt,t,DL)=H(Y∣xt,t,DL)−EΦ∼p(Φ∣DL)[H(Y∣xt,t,Φ)].(8)\operatorname{IG}^*(x_t)=I(Y;\Phi\mid x_t,t,\mathcal{D}_L) =H(Y\mid x_t,t,\mathcal{D}_L)-\mathbb{E}_{\Phi\sim p(\Phi\mid\mathcal{D}_L)}[H(Y\mid x_t,t,\Phi)]. \tag*{(8)}

This quantity represents the ideal information-theoretic objective. Its exact evaluation requires integration over the posterior distribution of the model parameters and is therefore not computed directly at each diffusion-sampling step.

Exact evaluation of the Bayesian objective in Equation (8) is intractable during iterative diffusion sampling because it requires integration over the model-parameter posterior. Rather than numerically estimating this posterior mutual information at every reverse-diffusion step, EvoDiff operationalizes the same information-seeking principle using a tractable trajectory-local information-gain surrogate. For the current diffusion state xtx_t and an LOEA candidate zz evaluated at the same noise level tt, we define

IG⁡tlocal(z;xt)=Hϕ(Y∣z,t)−Hϕ(Y∣xt,t).(9)\operatorname{IG}_t^{\mathrm{local}}(z;x_t)=H_\phi(Y\mid z,t)-H_\phi(Y\mid x_t,t). \tag*{(9)}

A positive value indicates that the local candidate induces greater predictive uncertainty than the current diffusion state. Importantly, Equation (9) compares xtx_t and its locally refined candidate at the same timestep; it does not compute an entropy difference between adjacent DDIM timesteps.

For any state uu evaluated at timestep tt, predictive entropy is computed from the frozen guidance classifier as

Hϕ[Y∣u,t]=−∑i=1Cpϕ(yi∣u,t)log⁡pϕ(yi∣u,t),(10)H_\phi[Y\mid u,t]=-\sum_{i=1}^{C}p_\phi(y_i\mid u,t)\log p_\phi(y_i\mid u,t), \tag*{(10)}

where pϕ(yi∣u,t)p_\phi(y_i\mid u,t) denotes the softmax probability for category yiy_i predicted by fϕf_\phi, and CC is the number of categories. To incorporate the trajectory-local IG surrogate into diffusion sampling, we define the scalar guidance objective

Lguide←λ⋅IG⁡tlocal(zt∗;xt)(11)\mathcal{L}_{\mathrm{guide}}\leftarrow\lambda\cdot\operatorname{IG}_t^{\mathrm{local}}(z_t^*;x_t) \tag*{(11)}

where zt∗z_t^* is the LOEA-refined candidate. The corresponding sampling-state guidance vector is

gt=∇xtLguide(xt).(12)g_t = \nabla_{x_t}\mathcal{L}_{\mathrm{guide}}(x_t). \tag*{(12)}

Thus, Lguide\mathcal{L}_{\mathrm{guide}} is a scalar objective, whereas gtg_t is the vector used to correct the DDIM noise estimate. The gradient is computed once with respect to xtx_t while the diffusion-model and guidance-classifier parameters remain frozen.

Thus, Equation (8) specifies the ideal Bayesian information-gain principle, whereas Equation (9) provides the tractable trajectory-local surrogate used during sampling. As illustrated in Algorithm 1, this surrogate constructs an IG-based implicit guidance field that redirects the reverse-diffusion trajectory toward regions of higher predictive uncertainty without updating model parameters.

Local Optimization Evolutionary Algorithm

Evolutionary algorithms are optimization algorithms that simulate the process of natural selection by iteratively optimizing solutions through operations such as selection, crossover, and mutation. Zhang et al.[73] were the first to mathematically establish a link between diffusion models and evolutionary algorithms. Leveraging the noise-addition and denoising characteristics of diffusion models, evolutionary algorithms can precisely refine solutions within the parameter space.

We propose a LOEA with a category-similarity constraint to refine candidate states locally at each reverse-DDIM timestep. As detailed in Algorithm 2, LOEA searches a candidate population Pt\mathcal{P}_t and favors candidates with high predictive entropy while penalizing candidates that violate the feature-similarity constraint.

This entropy term is consistent with the trajectory-local IG surrogate in Equation (9). Within a single LOEA call, Hϕ(Y∣xt,t)H_{\phi}(Y \mid x_t,t) is constant for all candidates z∈Ptz \in\mathcal{P}_t; therefore, ranking feasible candidates by Hϕ(Y∣z,t)H_{\phi}(Y \mid z,t) yields the same information-term ranking as IGtlocal(z;xt)\mathrm{IG}^{\mathrm{local}}_t(z;x_t). The similarity term acts as an additional constraint that preserves category consistency rather than defining a different information-gain quantity. Specifically, ResNet-50 features from labeled US images are used to construct the feature bank BB, and cosine similarity is used to penalize candidates that deviate from the corresponding category feature space. Selection retains the top-performing half of the population according to the constrained fitness score, followed by uniform crossover and Gaussian mutation. After GG generations, LOEA returns the candidate with the highest constrained fitness. Thus, Algorithm 2 provides the local candidate optimization used by Algorithm 1, while the information-seeking component remains consistent with the trajectory-local IG surrogate defined in Equation (9).

Curriculum Learning Strategy for Progressive Sample Selection

Directly prioritizing high-uncertainty samples in early AL stages can introduce unstable optimization and slow convergence to downstream classifiers due to noisy gradients and unreliable supervision signals. To address this challenge, EvoDiff incorporates a Curriculum Learning (CL) strategy for progressive sample selection, enabling a controlled transition from certain (easy) to uncertain (hard) samples over AL iterations. This design ensures that the downstream classifier first establishes robust decision primitives before being progressively exposed to ambiguous, boundary-level cases that refine decision surfaces.

We modulate sample difficulty using a discrete scheduling of the GS parameter λ\lambda applied during IG-guided diffusion sampling. Let ii denote the AL iteration index, the curriculum schedule is defined as:

λ(i)∈{1,3,5,7,9},(13)\lambda^{(i)} \in\{1,3,5,7,9\}, \tag*{(13)}

where lower λ\lambda emphasizes more certain regions and larger λ\lambda guides sampling toward higher predictive entropy. Round 0 uses only the fixed labeled initialization set. Rounds 1–6 correspond to cumulative candidate-pool budgets of 5%, 10%, 20%, 30%, 40%, and 50%, respectively. The corresponding newly annotated batches are therefore 5%, 5%, 10%, 10%, 10%, and 10% of the original candidate-pool size. The lowest guidance strength λ=1\lambda=1 spans the first two rounds, and the remaining guidance strengths 3, 5, 7, and 9 are each used for one subsequent round.

Following Equation (11), the curriculum-stage guidance objective becomes

Lguide(i)(xt)=λ(i)⋅IGtlocal(zt∗;xt),(14)\mathcal{L}_{\mathrm{guide}}^{(i)}(x_t) = \lambda^{(i)} \cdot\mathrm{IG}^{\mathrm{local}}_t(z_t^*;x_t), \tag*{(14)}

which modifies the DDIM denoising through the sampling-state gradient

gt(i)=∇xtLguide(i)(xt),ϵ~t(i)=ϵtraw−1−αˉtgt(i).(15)g_t^{(i)} = \nabla_{x_t}\mathcal{L}_{\mathrm{guide}}^{(i)}(x_t), \qquad\widetilde{\epsilon}_t^{(i)} = \epsilon_t^{\mathrm{raw}} - \sqrt{1-\bar{\alpha}_t}g_t^{(i)}. \tag*{(15)}

This formulation provides explicit control over sample difficulty while keeping the model parameters frozen during sampling.

EvoDiff uses curriculum-guided synthetic images as probes to retrieve real samples from the remaining candidate pool U\mathcal{U}. At stage ii, the frozen retrieval encoder F\mathcal{F} maps a generated probe g∈Gig \in\mathcal{G}_i and a real candidate u∈Uiu \in\mathcal{U}_i to ϕg=F(g)\phi_g=\mathcal{F}(g) and ϕu=F(u)\phi_u=\mathcal{F}(u). Their cosine similarity is

cos⁡(ϕg,ϕu)=ϕg⊤ϕu∥ϕg∥2∥ϕu∥2.(16)\cos(\phi_g,\phi_u) = \frac{\phi_g^\top\phi_u}{\lVert\phi_g\rVert_2\lVert\phi_u\rVert_2}. \tag*{(16)}

Algorithm 2 Local Optimization Evolutionary Algorithm

Input: current diffusion state xtx_t; population size nn; timestep tt; feature extractor F\mathcal{F}; frozen guidance classifier fϕf_\phi; feature bank B\mathcal{B}; similarity threshold τ\tau

Output: zt∗z_t^*: locally refined candidate with the highest constrained fitness

1: P←InitializePopulation⁡(xt,n)P \leftarrow\operatorname{InitializePopulation}(x_t,n) // Local perturbations around xtx_t

2: G←40G \leftarrow40, pmut←0.1p_{\mathrm{mut}} \leftarrow0.1, σmut←0.1\sigma_{\mathrm{mut}} \leftarrow0.1

3: for g=1g = 1 to GG do

4: p←softmax⁡(fϕ(P,t))p \leftarrow\operatorname{softmax}(f_\phi(P,t)), log⁡p←log⁡softmax⁡(fϕ(P,t))\log p \leftarrow\log\operatorname{softmax}(f_\phi(P,t))

5: H←−∑ipilog⁡pi\mathcal{H} \leftarrow-\sum_i p_i \log p_i // Entropy term; equivalent local-IG ranking before similarity penalty

6: V←F(P)V \leftarrow\mathcal{F}(P), π←0\pi\leftarrow0

7: for i=1i = 1 to nn do

8: fi←V[i]f_i \leftarrow V[i]

9: sc←cos⁡(fi,B[c]), ∀cs_c \leftarrow\cos(f_i,\mathcal{B}[c]),\ \forall c

10: if max⁡c(sc)<τ\max_c(s_c) < \tau then

11: π[i]←1\pi[i] \leftarrow1

12: end if

13: end for

14: fitness⁡←H−10⋅π\operatorname{fitness} \leftarrow\mathcal{H} - 10 \cdot\pi // Local information term with similarity penalty

15: Psel←TopK⁡(P,fitness⁡,k=⌊n/2⌋)P_{\mathrm{sel}} \leftarrow\operatorname{TopK}(P,\operatorname{fitness},k = \lfloor n/2 \rfloor)

/* Crossover */

16: for i=1i = 1 to nn do

17: Sample p1,p2∼Pselp_1,p_2 \sim P_{\mathrm{sel}}

18: Sample m∼{0,1}m \sim\{0,1\}

19: ci←m⋅p1+(1−m)⋅p2c_i \leftarrow m \cdot p_1 + (1-m) \cdot p_2

20: end for

21: P←{ci}i=1nP \leftarrow\{c_i\}_{i=1}^{n}

/* Mutation */

22: ϵ∼N(0,σmut2)\epsilon\sim\mathcal{N}(0,\sigma_{\mathrm{mut}}^2)

23: M∼Bernoulli⁡(pmut)M \sim\operatorname{Bernoulli}(p_{\mathrm{mut}})

24: P←P+M⋅ϵP \leftarrow P + M \cdot\epsilon

25: end for

26: p←softmax⁡(fϕ(P,t))p \leftarrow\operatorname{softmax}(f_\phi(P,t)), log⁡p←log⁡softmax⁡(fϕ(P,t))\log p \leftarrow\log\operatorname{softmax}(f_\phi(P,t))

27: H←−∑ipilog⁡pi\mathcal{H} \leftarrow-\sum_i p_i \log p_i

28: Recompute π\pi using same procedure

29: fitness⁡←H−10⋅π\operatorname{fitness} \leftarrow\mathcal{H} - 10 \cdot\pi // Local information term with similarity penalty

30: zt∗←P[argmax⁡(fitness⁡)]z_t^* \leftarrow P[\operatorname{argmax}(\operatorname{fitness})]

31: return zt∗z_t^*

The KiK_i highest-ranked real candidates are selected, where KiK_i equals the stage-specific batch size above (equivalently, the round-specific αi%\alpha_i\% selection ratio). Previously selected images are removed before ranking, so acquisition is without replacement.

Only the retrieved real images are annotated and used for downstream training.

EvoDiff adopts a cumulative curriculum rather than resetting the labeled set at each stage. Every selected image is retained in the expanded labeled training set, and subsequent acquisition operates only on the remaining candidate pool.

Implementation details

During the pre-training phase, all US images were resized to 224×224224 \times224 pixels. For pre-training with the large-scale unlabeled data pool, the diffusion model was optimized using a total of 1,000 timesteps with a linear noise schedule, where the noise coefficient βt\beta_t ranged from β1=1×10−4\beta_1 = 1 \times10^{-4} to βT=0.02\beta_T = 0.02. The training was conducted for 20,000 iterations with a batch size of 4.

During the fine-tuning stage with pathologically labeled data, the noise coefficient range was adjusted to β1=1×10−5\beta_1 = 1 \times10^{-5} to βT=0.02\beta_T = 0.02. For classifier pre-training, we employed the same number of training iterations with a batch size of 2 and a learning rate of 2×10−42 \times10^{-4}. The learning rate was subsequently reduced to 1×10−41 \times10^{-4} during fine-tuning with pathologically labeled data.

CNN-based diagnostic model ResNet-50[42]: During the image preprocessing stage, all images were resized to 224 × 224 pixels and subjected to a series of data augmentation techniques, including random horizontal flipping, Color Jitter (brightness=0.2, contrast=0.2, saturation=0.2, hue=0.1), and random rotation (Random Rotation with degrees ranging from −10∘-10^\circ to 10∘10^\circ). The experimental procedure was as follows: we first loaded ResNet-50 weights pre-trained on ImageNet, then fine-tuned the model using pathologically labeled data to obtain the initialized downstream model. The training hyperparameters were set as follows: batch size of 2, learning rate of 1×10−41 \times10^{-4}, with a cosine annealing strategy to facilitate convergence, and a total of 40 epochs. During the AL iterations, the initialized weights were loaded at the beginning of each round while maintaining consistent experimental settings, and model weights were saved after each AL iteration. Transformer-based foundation model USFM[43]. The data preprocessing pipeline followed the original paper’s configuration, with images resized to 256×256256 \times256 pixels and augmented with random rotations and flips. We first loaded the pre-trained weights released by USFM[43], then fine-tuned the model using pathologically labeled data and saved the weights before proceeding with incremental learning across AL iterations. Training was performed using the Adam optimizer with a cosine learning rate adjustment strategy. To ensure convergence during USFM fine-tuning, we configured dataset-specific hyperparameters, as detailed in Supplementary Table S5.

All experiments were conducted on four NVIDIA GeForce A40 GPUs. All models were developed using the PyTorch framework. The global random seed was 42 for PyTorch, CUDA, NumPy, and the Python random module, and deterministic cuDNN execution was enabled. Each internal dataset was partitioned once using a fixed 8:1:1 split into an unlabeled candidate pool (80%), a labeled initialization set (10%), and an internal test set (10%), and the case-level separation of the internal datasets, including the BUSI-specific limitation, is reported in the Cross-organ Multi-task Ultrasound Dataset subsection. The cumulative candidate-pool budgets were 5%, 10%, 20%, 30%, 40%, and 50%. The corresponding newly annotated batches, and therefore the round-specific αi\alpha_i selection ratios, were 5%, 5%, 10%, 10%, 10%, and 10% of the original candidate-pool size, drawn without replacement. EvoDiff generated 100 synthetic probes per requested class condition at each curriculum stage. The diffusion sampler followed the DDIM configuration described above. LOEA used a population size of 50, 40 generations, top-50% selection, a mutation probability of 0.1, and a Gaussian mutation s.d. of 0.1. Random used uniform sampling without replacement. Max-Entropy ranked candidates by predictive entropy. BALD used 25 Monte Carlo dropout passes. Core-Set used greedy kk-center selection. VAAL used its published VAE and discriminator acquisition procedure. Within each downstream backbone setting, all comparison methods used the same labeled initialization set, unlabeled candidate pool, stage-specific batch sizes, downstream architecture, initialization weights, optimizer, learning-rate schedule, number of training epochs, and fixed internal test set, so that the acquisition strategy is the only varying factor.

Evaluation metrics and statistical analysis

Downstream model classification tasks were evaluated using different metrics tailored to the specific characteristics and capabilities of each model architecture. For CNN-based models, we primarily use Accuracy (ACC) as the evaluation metric, while for foundation models, we employ balanced accuracy (bAcc) as the primary metric. Additionally, Area Under the Curve (AUC) is used as a supplementary metric for comprehensive performance assessment.

The ACC metric measures the overall proportion of correctly classified samples across all classes, providing a general assessment of model performance. The ACC is defined as follows:

Accuracy=TP+TNTP+FN+FP+TN(17)\mathrm{Accuracy} = \frac{TP + TN}{TP + FN + FP + TN} \tag*{(17)}

where TP refers to true positives, FP refers to false positives, FN refers to false negatives, TN refers to true negatives. For CNN-based downstream models (e.g. ResNet-50), ACC serves as an appropriate metric due to their training on relatively balanced subsets during the AL process, where the iterative sample selection helps mitigate severe class imbalance issues.

However, foundation models, which are pre-trained on large-scale datasets and fine-tuned for our tasks, demonstrate different sensitivity to class distribution variations. To address this and provide a more robust evaluation that accounts for the inherent class imbalance in medical imaging datasets, we employ bAcc for foundation model evaluation. The bAcc metric provides equal weight to each class regardless of sample size:

sensitivity=TPTP+FNspecificity=TNTN+FPbAcc=sensitivity+specificity2(18)\begin{aligned} \mathrm{sensitivity} &= \frac{TP}{TP + FN} \\ \mathrm{specificity} &= \frac{TN}{TN + FP} \\ \mathrm{bAcc} &= \frac{\mathrm{sensitivity} + \mathrm{specificity}}{2} \tag*{(18)} \end{aligned}

The AUC metric evaluates the model’s ability to distinguish between classes by measuring the area under the Receiver Operating Characteristic (ROC) curve.

All reported performance is obtained by five-fold cross-validation within the expanded labeled training set at each cumulative annotation budget, and results are presented as mean values with s.d. across the five fold-specific models evaluated on the fixed internal test set. Each sample-selection strategy is compared with random sampling using two-sided paired tt-tests, and multiplicity is controlled with the Benjamini–Hochberg procedure applied across the five strategy-versus-Random comparisons that share a given dataset and annotation budget. Both unadjusted pp-values and Benjamini–Hochberg-calibrated qq-values are reported. Supplementary Figure S1 to S8 present these comparisons as heat maps, one metric per figure, reporting the unadjusted pp-values and the mean differences relative to random sampling for both downstream architectures across the eight internal datasets and the three cumulative annotation budgets, with Benjamini–Hochberg-corrected significance marked in each map. For the ablation study, each reduced configuration was compared with the complete EvoDiff framework using two-sided paired tt-tests, with the five cross-validation folds used as matched pairs. The five cross-validation runs constitute the repeated experiments referred to throughout, rather than independently repeated dataset splits.

Figure S1. Statistical comparison of sample-selection strategies against Random for accuracy across eight datasets and three cumulative annotation budgets. a) Results for the ResNet-50 downstream model. b) Results for the ultrasound foundation model (USFM). In each panel, the three heat maps correspond to cumulative annotation budgets of 10%, 20%, and 30%, rows list the five sample-selection strategies compared against Random (VAAL, Core-Set, Max-Entropy, BALD, and EvoDiff), and columns list the eight internal datasets. In each cell, the upper value is the unadjusted pp-value from a two-sided paired tt-test on the five matched cross-validation folds, and the lower value is the mean difference in accuracy relative to Random, with positive values indicating better performance than Random. The cell colour encodes −log⁡10(p)-\log_{10}(p) from light yellow for values near 1 to dark red for values of 3.5 and above, as shown by the colour bar. A black cell border marks an unadjusted p<0.05p < 0.05, and an asterisk marks significance after Benjamini–Hochberg correction, with q<0.05q < 0.05 within the family of the five strategy-versus-Random comparisons that share a dataset and annotation budget. The testing procedure is identical for both downstream models.

Algorithm 3 HAE Metric Computation

Input: DUD^{U}: unlabeled candidate pool; SALS_{AL}: active learning strategy; Θ={θ1,θ2,…,θm}\Theta= \{\theta_1,\theta_2,\ldots,\theta_m\}: performance thresholds; R={r1,r2,…,rn}R = \{r_1,r_2,\ldots,r_n\}: cumulative annotation budgets; T=30T = 30: time window; ε=0.01\varepsilon= 0.01: precision parameter

Output: HAElistHAE_{list}: HAEHAE: human annotation effort, △HAE\triangle HAE: human annotation effort saved for each threshold

1: Wunit←calibrate_annotation_time⁡(T)W_{unit} \leftarrow\operatorname{calibrate\_annotation\_time}(T) // Measure unit annotation cost 2: Arandom,AAL←evaluate_strategies⁡(DU,SAL,R)A_{random},A_{AL} \leftarrow\operatorname{evaluate\_strategies}(D^{U},S_{AL},R) // Generate performance curves 3: HAElist←[]HAE_{list} \leftarrow[] // Initialize HAE results 4: for k=1k = 1 to ∣Θ∣|\Theta| do 5: θk←Θ[k]\theta_k \leftarrow\Theta[k] 6: rALθk←min⁡{r:AAL(r)≥θk}r_{AL}^{\theta_k} \leftarrow\min\{r:A_{AL}(r)\geq\theta_k\} // Eq. (1) 7: ri,ri+1←find_adjacent_ratios⁡(Arandom,θk)r_i,r_{i+1} \leftarrow\operatorname{find\_adjacent\_ratios}(A_{random},\theta_k) 8: if Arandom(ri)≤θk≤Arandom(ri+1)A_{random}(r_i)\leq\theta_k\leq A_{random}(r_{i+1}) then 9: ρ←Arandom(ri+1)−Arandom(ri)(ri+1−ri)×∣DU∣\rho\leftarrow\frac{A_{random}(r_{i+1})-A_{random}(r_i)}{(r_{i+1}-r_i)\times|D^{U}|} // Performance improvement rate 10: requivθk←ri+θk−Arandom(ri)ρ×∣DU∣r_{equiv}^{\theta_k}\leftarrow r_i+\frac{\theta_k-A_{random}(r_i)}{\rho\times|D^{U}|} // Equivalent ratio interpolation 11: end if 12: NALθk←rALθk×∣DU∣N_{AL}^{\theta_k}\leftarrow r_{AL}^{\theta_k}\times|D^{U}| // AL sampling requirement 13: Nrandomθk←requivθk×∣DU∣N_{random}^{\theta_k}\leftarrow r_{equiv}^{\theta_k}\times|D^{U}| // Random sampling requirement 14: HAEALθk←NALθk×WunitHAE_{AL}^{\theta_k}\leftarrow N_{AL}^{\theta_k}\times W_{unit} // AL sampling workload 15: HAErandomθk←Nrandomθk×WunitHAE_{random}^{\theta_k}\leftarrow N_{random}^{\theta_k}\times W_{unit} // Random sampling workload 16: △HAE←(Nrandomθk−NALθk)×Wunit\triangle HAE\leftarrow(N_{random}^{\theta_k}-N_{AL}^{\theta_k})\times W_{unit} // Workload savings 17: HAElist.append⁡(HAErandomθk,HAEALθk,△HAEθk)HAE_{list}.\operatorname{append}(HAE_{random}^{\theta_k},HAE_{AL}^{\theta_k},\triangle HAE^{\theta_k}) 18: end for

Human Annotation Effort estimation

Given a selected and annotated dataset DD from unlabeled candidate pool DUD^{U} with ∣DU∣|D^{U}| samples and corresponding downstream diagnostic model performance metrics (e.g., accuracy), existing evaluation approaches for AL typically focus on measuring improvements in model performance or convergence rates. However, these metrics fail to quantify the practical annotation costs that radiologists encounter in real-world clinical scenarios, where evaluating the efficiency gains of AL over random sampling is crucial for AL algorithm deployment. Therefore, we propose a novel evaluation metric, Human Annotation Effort (HAE), which quantifies the annotation workload required by different sample selection strategies when reaching equivalent downstream diagnostic performance. Based on HAE, we further define Human Annotation Effort Saved (△\triangleHAE) to estimate the workload reduction achieved by AL algorithms. Diagnostic classification annotation primarily involves cognitive operations; the annotation process consists of: (1) medical image interpretation through visual examination, (2) differential diagnosis reasoning based on clinical knowledge, and (3) diagnostic decision confirmation through categorical labeling. Classification annotation focuses on diagnostic decision-making, quantifiable through time-based measurements.

Let AAL(r)A_{AL}(r) and Arandom(r)A_{random}(r) represent the downstream classification accuracy achieved by AL and random sampling strategies after annotating samples at ratio rr (where r∈[0,1]r\in[0,1] represents the fraction of unlabeled candidate pool DUD^{U} annotated), respectively. Given a target performance threshold θ\theta (e.g., 85% accuracy), we define:

rALθ=min⁡{r:AAL(r)≥θ},(19)r_{AL}^{\theta}=\min\{r:A_{AL}(r)\geq\theta\}, \tag*{(19)}
rrandomθ=min⁡{r:Arandom(r)≥θ},(20)r_{random}^{\theta}=\min\{r:A_{random}(r)\geq\theta\}, \tag*{(20)}

where rALθr_{\mathrm{AL}}^{\theta} represents the minimum cumulative annotation budget required by AL to achieve performance threshold θ\theta, and rrandomθr_{\mathrm{random}}^{\theta} represents the minimum cumulative annotation budget required by random sampling to achieve the same performance threshold.

The unit annotation time used by the HAE metric is obtained from empirical time measurements. We conduct controlled experiments where experienced radiologists perform diagnostic classification tasks within fixed time windows TT (e.g., 30 minutes). The unit annotation cost is calculated as Wunit=T/CcorrectW_{\mathrm{unit}} = T/C_{\mathrm{correct}}, where CcorrectC_{\mathrm{correct}} represents the number of correctly annotated samples within the time window. Reader responses were verified against the ground-truth labels, so that only correctly annotated samples contribute to CcorrectC_{\mathrm{correct}}. The diagnostic workload encompasses multiple measurable components, including visual examination duration, differential diagnosis consideration time, and clinical reasoning complexity.

To establish equivalent performance benchmarks, we systematically evaluate both random sampling and AL strategies across multiple cumulative annotation budgets r1,r2,…,rnr_1,r_2,\ldots,r_n and measure their downstream diagnostic accuracy Arandom(ri)A_{\mathrm{random}}(r_i) and AAL(rj)A_{\mathrm{AL}}(r_j). For cases where the AL performance AAL(rj)A_{\mathrm{AL}}(r_j) falls between two consecutive random sampling performance points, we employ interpolation to determine the equivalent random sampling ratio. Specifically, we identify adjacent random sampling ratios rir_i and ri+1r_{i+1} such that Arandom(ri)≤AAL(rj)≤Arandom(ri+1)A_{\mathrm{random}}(r_i) \le A_{\mathrm{AL}}(r_j) \le A_{\mathrm{random}}(r_{i+1}). The performance improvement rate within this interval is calculated as:

ρ=Arandom(ri+1)−Arandom(ri)(ri+1−ri)×∣D∣(21)\rho= \frac{A_{\mathrm{random}}(r_{i+1}) - A_{\mathrm{random}}(r_i)}{(r_{i+1}-r_i)\times|D|} \tag*{(21)}

representing the accuracy gain per additional annotated sample. The equivalent random sampling ratio is then determined as:

requiv=ri+AAL(rj)−Arandom(ri)ρ×∣D∣(22)r_{\mathrm{equiv}} = r_i + \frac{A_{\mathrm{AL}}(r_j)-A_{\mathrm{random}}(r_i)}{\rho\times|D|} \tag*{(22)}

The core HAE computation follows:

HAErandom=Nrandom×Wunit,(23)\mathrm{HAE}_{\mathrm{random}} = N_{\mathrm{random}} \times W_{\mathrm{unit}}, \tag*{(23)}
HAEAL=NAL×Wunit,(24)\mathrm{HAE}_{\mathrm{AL}} = N_{\mathrm{AL}} \times W_{\mathrm{unit}}, \tag*{(24)}
△HAE=(HAErandom−HAEAL),(25)\triangle\mathrm{HAE} = (\mathrm{HAE}_{\mathrm{random}}-\mathrm{HAE}_{\mathrm{AL}}), \tag*{(25)}

where Nrandom=requiv×∣D∣N_{\mathrm{random}} = r_{\mathrm{equiv}} \times|D| and NAL=rj×∣D∣N_{\mathrm{AL}} = r_j \times|D| represent the sample quantities required by random sampling and AL, respectively, to achieve equivalent downstream performance. The performance equivalence is established through the interpolation described above: Arandom(requiv)=AAL(rj)A_{\mathrm{random}}(r_{\mathrm{equiv}}) = A_{\mathrm{AL}}(r_j). For each reader and organ task, the same unit annotation time WunitW_{\mathrm{unit}} is applied to both strategies, assuming comparable average time per correctly annotated image. Under this assumption, the relative saving equals 1−NAL/Nrandom1-N_{\mathrm{AL}}/N_{\mathrm{random}} and does not depend on WunitW_{\mathrm{unit}}, whereas the absolute workload estimates depend on the measured unit annotation times. Overall, the computation procedure is systematically outlined in Algorithm 3, encompassing empirical annotation time calibration through controlled radiologist sessions, performance curve generation across varying cumulative annotation budgets, equivalent performance sample requirement estimation, and workload savings quantification across multiple benchmarks.

Ethics approval

This study was conducted in accordance with the Declaration of Helsinki. The retrospective collection and analysis of the three private external cohorts were approved by the Institutional Review Board of Zhejiang Cancer Hospital (approval No. 2026ZN004-1). Owing to the retrospective design of the study and the use of de-identified imaging data, the requirement for individual informed consent was waived by the same board. All public datasets used in this study are fully anonymized, openly available collections that were released under the ethical oversight of their original studies and were used in accordance with their respective licenses and access conditions.

Data availability

The public medical datasets used in this study are accessible from the following sources: Appendix, https://zenodo.org/records/7669442; BUSI, https://www.kaggle.com/datasets/aryashah2k/breast-ultrasound-images-dataset; BUS-BRA, https://zenodo.org/records/8231412; UDIAT, https://ieeexplore.ieee.org/abstract/document/8003418; ALN-Ultra, https://zenodo.org/records/15003119; QAMEBI, https://qamebi.com/breast-ultrasound-images-database; CUBS, https://data.mendeley.com/datasets/fpv535fss7/1; Fatty-Liver, https://zenodo.org/records/1009146; MMOTU, https://figshare.com/articles/dataset/zip/25058690?file=44222642; TN3K, https://github.com/haifangong/TRFE-Net-for-thyroid-nodule-segmentation; TN5000, https://figshare.com/s/cb6a67f17c04b29e7edd. The datasets generated and/or analyzed during the current study from the three private external cohorts collected at Zhejiang Cancer Hospital are not publicly available due to institutional ethics approval and patient privacy restrictions that do not permit open redistribution of the imaging data, but are available from the corresponding author on reasonable request. Shared data will be provided in de-identified form, and requests are subject to approval by the institutional review board.

Code availability

The source code for EvoDiff is publicly available without access restrictions at https://github.com/IsBaSO4/EvoDiff. All experiments were implemented in Python using PyTorch 1.12.1 and torchvision 0.13.1, together with scikit-learn 1.3.2, SciPy 1.9.1, pandas 2.0.3, timm 0.6.12, OpenCV 4.10.0 and albumentations 1.3.0; the complete pinned software environment is listed in the requirements.txt file of the repository. The specific hyperparameters, random seed, and acquisition settings used to generate and analyze the current datasets are reported in the Methods (Implementation details) and are provided as default configurations in the repository. Pre-trained weights and implementation code for the downstream models are accessible through the following repositories: USFM at https://github.com/openmedlab/USFM and ResNet-50 at https://github.com/pytorch/vision/blob/main/torchvision/models/resnet.py.

Acknowledgements

Not applicable.

Funding

This work was supported by Shenzhen Medical Research Fund (D2501013), the Science and Technology Development Fund of Macao (0041/2023/RIB2), the National Natural Science Foundation of China (12326619), the Frontier Technology Development Program of Jiangsu Province (BF2024078), the Science and Technology Planning Project of Guangdong Province (2023A0505020002), the National Natural Science Foundation of China (62601018), and the Macao Polytechnic University Grant (RP/FCA-25/2025). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Author Contributions

Y.D., Y.H., Y.G., YL.G. and T.T. conceived the conception and design of the study; Y.D. and Y.H. developed the deep learning model; T.T. and D.N. contributed to device support for model training and inference; Y.D. and D.X. contributed to acquisition and annotation of the data; Y.D., Y.H., R.Q., Y.S., R.M. and T.T. contributed to analysis and interpretation of the data; T.Z., D.X. and R.M. conducted the reader study. Y.D. and Y.H. wrote the initial manuscript; S.I., R.M., and T.T. supervised the work. All authors contributed to the revision of the manuscript.

Competing Interests

The authors declare no competing financial or non-financial interests.

Supplementary Information

Generative Active Learning for Label-Efficient Pan-Organ Ultrasound Diagnosis

DatasetMethodAnnotation BudgetAccbAccSensSpecPPVNPVF1AUCq-value
BUS-BRA [46]FSL (upper bound)100%0.9558±0.01230.9558 \pm0.01230.9583±0.00540.9583 \pm0.00540.9323±0.01580.9323 \pm0.01580.9844±0.01480.9844 \pm0.01480.9679±0.02990.9679 \pm0.02990.9679±0.00690.9679 \pm0.00690.9492±0.01010.9492 \pm0.01010.9875±0.00410.9875 \pm0.0041-
Random10%0.7347±0.07390.7347 \pm0.07390.5507±0.05200.5507 \pm0.05200.1968±0.17740.1968 \pm0.17740.9047±0.07960.9047 \pm0.07960.3817±0.19820.3817 \pm0.19820.7030±0.03230.7030 \pm0.03230.2374±0.18820.2374 \pm0.18820.7523±0.04390.7523 \pm0.0439-
VAAL [57]10%0.6905±0.01010.6905 \pm0.01010.5995±0.05530.5995 \pm0.05530.3194±0.22840.3194 \pm0.22840.8797±0.13060.8797 \pm0.13060.5998±0.10110.5998 \pm0.10110.7369±0.04910.7369 \pm0.04910.3631±0.16620.3631 \pm0.16620.7309±0.02730.7309 \pm0.02730.06130.0613 (−)
Core-Set [58]10%0.6959±0.02910.6959 \pm0.02910.5877±0.02900.5877 \pm0.02900.4097±0.04160.4097 \pm0.04160.7656±0.03200.7656 \pm0.03200.4598±0.04650.4598 \pm0.04650.7280±0.01780.7280 \pm0.01780.4327±0.04080.4327 \pm0.04080.7343±0.04870.7343 \pm0.04870.06270.0627 (−)
Max-Entropy [59]10%0.7317±0.07270.7317 \pm0.07270.6193±0.02150.6193 \pm0.02150.5323±0.16190.5323 \pm0.16190.7063±0.13990.7063 \pm0.13990.4929±0.06490.4929 \pm0.06490.7626±0.02510.7626 \pm0.02510.4851±0.07070.4851 \pm0.07070.7554±0.02640.7554 \pm0.02640.06600.0660 (+)
BALD [60]10%0.7364±0.02700.7364 \pm0.02700.5435±0.05920.5435 \pm0.05920.1871±0.15890.1871 \pm0.15890.9000±0.07610.9000 \pm0.07610.3758±0.21570.3758 \pm0.21570.6981±0.03430.6981 \pm0.03430.2348±0.17590.2348 \pm0.17590.7625±0.08840.7625 \pm0.08840.06130.0613 (+)
EvoDiff (Ours)10%0.7826±0.06190.7826 \pm0.06190.6367±0.04920.6367 \pm0.04920.5000±0.15300.5000 \pm0.15300.7734±0.07250.7734 \pm0.07250.5154±0.05530.5154 \pm0.05530.7666±0.04020.7666 \pm0.04020.4960±0.09700.4960 \pm0.09700.8358±0.04980.8358 \pm0.04980.06270.0627 (+)
Random20%0.7663±0.04370.7663 \pm0.04370.5762±0.05140.5762 \pm0.05140.2742±0.20510.2742 \pm0.20510.8781±0.10520.8781 \pm0.10520.5573±0.05720.5573 \pm0.05720.7203±0.03910.7203 \pm0.03910.3153±0.16100.3153 \pm0.16100.8014±0.01560.8014 \pm0.0156-
VAAL [57]20%0.7118±0.03750.7118 \pm0.03750.5901±0.06250.5901 \pm0.06250.2645±0.20670.2645 \pm0.20670.9156±0.08490.9156 \pm0.08490.6075±0.07530.6075 \pm0.07530.7266±0.04630.7266 \pm0.04630.3239±0.18040.3239 \pm0.18040.7661±0.04730.7661 \pm0.04730.05430.0543 (−)
Core-Set [58]20%0.7444±0.09070.7444 \pm0.09070.6310±0.03510.6310 \pm0.03510.3839±0.13740.3839 \pm0.13740.8781±0.11040.8781 \pm0.11040.6978±0.18130.6978 \pm0.18130.7489±0.02630.7489 \pm0.02630.4566±0.08240.4566 \pm0.08240.7834±0.05030.7834 \pm0.05030.04250.0425 (−)
Max-Entropy [59]20%0.7455±0.03440.7455 \pm0.03440.6010±0.06160.6010 \pm0.06160.4613±0.13000.4613 \pm0.13000.7406±0.15370.7406 \pm0.15370.4852±0.08920.4852 \pm0.08920.7378±0.03780.7378 \pm0.03780.4585±0.08530.4585 \pm0.08530.7979±0.05600.7979 \pm0.05600.04250.0425 (−)
BALD [60]20%0.7807±0.01180.7807 \pm0.01180.5957±0.04940.5957 \pm0.04940.2742±0.13260.2742 \pm0.13260.9172±0.06930.9172 \pm0.06930.6690±0.15280.6690 \pm0.15280.7247±0.02870.7247 \pm0.02870.3599±0.13790.3599 \pm0.13790.8184±0.07390.8184 \pm0.07390.04250.0425 (+)
EvoDiff (Ours)20%0.8327±0.02760.8327 \pm0.02760.6571±0.05510.6571 \pm0.05510.5065±0.17680.5065 \pm0.17680.8078±0.11370.8078 \pm0.11370.6017±0.12410.6017 \pm0.12410.7783±0.05150.7783 \pm0.05150.5183±0.09380.5183 \pm0.09380.8640±0.04650.8640 \pm0.04650.04540.0454 (+)
Random30%0.8084±0.03440.8084 \pm0.03440.6419±0.05860.6419 \pm0.05860.4323±0.14370.4323 \pm0.14370.8516±0.10710.8516 \pm0.10710.6253±0.10970.6253 \pm0.10970.7579±0.03770.7579 \pm0.03770.4859±0.10850.4859 \pm0.10850.8441±0.06780.8441 \pm0.0678-
VAAL [57]30%0.7558±0.04560.7558 \pm0.04560.6471±0.05200.6471 \pm0.05200.4097±0.16570.4097 \pm0.16570.8797±0.10790.8797 \pm0.10790.6053±0.09940.6053 \pm0.09940.7578±0.04480.7578 \pm0.04480.4637±0.13240.4637 \pm0.13240.8059±0.06000.8059 \pm0.06000.09950.0995 (−)
Core-Set [58]30%0.7611±0.03770.7611 \pm0.03770.6445±0.06200.6445 \pm0.06200.4274±0.15960.4274 \pm0.15960.8751±0.09880.8751 \pm0.09880.5909±0.08530.5909 \pm0.08530.7557±0.03860.7557 \pm0.03860.4526±0.12180.4526 \pm0.12180.8080±0.04860.8080 \pm0.04860.09950.0995 (−)
Max-Entropy [59]30%0.7757±0.03150.7757 \pm0.03150.6299±0.06120.6299 \pm0.06120.4645±0.16640.4645 \pm0.16640.8535±0.09970.8535 \pm0.09970.6313±0.09140.6313 \pm0.09140.7694±0.03960.7694 \pm0.03960.4526±0.12180.4526 \pm0.12180.8180±0.04860.8180 \pm0.04860.09950.0995 (−)
BALD [60]30%0.7939±0.04680.7939 \pm0.04680.6435±0.05490.6435 \pm0.05490.4323±0.14370.4323 \pm0.14370.8562±0.09340.8562 \pm0.09340.6599±0.11190.6599 \pm0.11190.7625±0.04070.7625 \pm0.04070.4799±0.12300.4799 \pm0.12300.8238±0.04650.8238 \pm0.04650.09950.0995 (+)
EvoDiff (Ours)30%0.8517±0.04360.8517 \pm0.04360.6895±0.06240.6895 \pm0.06240.5258±0.11830.5258 \pm0.11830.8853±0.06760.8853 \pm0.06760.6473±0.05950.6473 \pm0.05950.7904±0.04720.7904 \pm0.04720.5703±0.04670.5703 \pm0.04670.8941±0.06620.8941 \pm0.06620.09950.0995 (+)
UDIAT [47]FSL (upper bound)100%0.6889±0.05670.6889 \pm0.05670.6117±0.13330.6117 \pm0.13330.1667±0.25820.1667 \pm0.25820.9833±0.25610.9833 \pm0.25610.1111±0.17210.1111 \pm0.17210.6254±0.04880.6254 \pm0.04880.1333±0.20650.1333 \pm0.20650.7510±0.12810.7510 \pm0.1281-
Random10%0.6772±0.09000.6772 \pm0.09000.4750±0.06770.4750 \pm0.06770.2667±0.37420.2667 \pm0.37420.6833±0.35510.6833 \pm0.35510.3167±0.36670.3167 \pm0.36670.5192±0.26180.5192 \pm0.26180.1971±0.18830.1971 \pm0.18830.7554±0.07210.7554 \pm0.07210.04200.0420
VAAL [57]10%0.7186±0.02780.7186 \pm0.02780.5417±0.05270.5417 \pm0.05270.3333±0.35610.3333 \pm0.35610.7500±0.29810.7500 \pm0.29810.2750±0.22910.2750 \pm0.22910.7485±0.26180.7485 \pm0.26180.2691±0.18830.2691 \pm0.18830.7598±0.14080.7598 \pm0.14080.02580.0258 (+)
Core-Set [58]10%0.6685±0.01750.6685 \pm0.01750.4000±0.08670.4000 \pm0.08670.2333±0.22610.2333 \pm0.22610.5667±0.27590.5667 \pm0.27590.1395±0.12910.1395 \pm0.12910.5693±0.15560.5693 \pm0.15560.1715±0.12020.1715 \pm0.12020.7020±0.18670.7020 \pm0.18670.07690.0769 (+)
Max-Entropy [59]10%0.6827±0.01650.6827 \pm0.01650.5471±0.06870.5471 \pm0.06870.2333±0.22610.2333 \pm0.22610.7500±0.29810.7500 \pm0.29810.2750±0.22910.2750 \pm0.22910.5693±0.15560.5693 \pm0.15560.1715±0.12020.1715 \pm0.12020.7020±0.18670.7020 \pm0.18670.07690.0769 (+)
BALD [60]10%0.6850±0.01750.6850 \pm0.01750.4077±0.06810.4077 \pm0.06810.2000±0.24470.2000 \pm0.24470.8833±0.11780.8833 \pm0.11780.4833±0.31830.4833 \pm0.31830.7447±0.06870.7447 \pm0.06870.2967±0.21170.2967 \pm0.21170.7620±0.09130.7620 \pm0.09130.07690.0769 (+)
EvoDiff (Ours)10%0.7851±0.07610.7851 \pm0.07610.5667±0.07730.5667 \pm0.07730.2000±0.20000.2000 \pm0.20000.8333±0.09130.8333 \pm0.09130.4833±0.09130.4833 \pm0.09130.7042±0.03440.7042 \pm0.03440.2510±0.10390.2510 \pm0.10390.8590±0.05700.8590 \pm0.05700.04200.0420 (+)
Random20%0.7355±0.05870.7355 \pm0.05870.5667±0.12470.5667 \pm0.12470.1333±0.19170.1333 \pm0.19171.0000±0.00001.0000 \pm0.00000.6000±0.48990.6000 \pm0.48990.6690±0.19170.6690 \pm0.19170.2143±0.3090.2143 \pm0.3090.7690±0.12670.7690 \pm0.12670.07670.0767 (−)
VAAL [57]20%0.7537±0.04080.7537 \pm0.04080.5083±0.03330.5083 \pm0.03330.0333±0.06670.0333 \pm0.06670.9167±0.16670.9167 \pm0.16670.2000±0.20000.2000 \pm0.20000.6706±0.01920.6706 \pm0.01920.0500±0.10000.0500 \pm0.10000.7923±0.17670.7923 \pm0.17670.08960.0896 (+)
Core-Set [58]20%0.7579±0.10530.7579 \pm0.10530.5083±0.01670.5083 \pm0.01670.0333±0.06670.0333 \pm0.06670.9167±0.10450.9167 \pm0.10450.1000±0.20000.1000 \pm0.20000.6708±0.03430.6708 \pm0.03430.0500±0.10000.0500 \pm0.10000.7840±0.1450.7840 \pm0.1450.02860.0286 (+)
Max-Entropy [59]20%0.7519±0.07610.7519 \pm0.07610.5583±0.09220.5583 \pm0.09220.2000±0.20000.2000 \pm0.20000.7833±0.09130.7833 \pm0.09130.1000±0.20000.1000 \pm0.20000.6708±0.03430.6708 \pm0.03430.0500±0.10000.0500 \pm0.10000.7840±0.10450.7840 \pm0.10450.02800.0280 (+)
BALD [60]20%0.7851±0.07610.7851 \pm0.07610.5667±0.07730.5667 \pm0.07730.2000±0.20000.2000 \pm0.20000.8333±0.09130.8333 \pm0.09130.4833±0.09130.4833 \pm0.09130.7042±0.03440.7042 \pm0.03440.2510±0.10390.2510 \pm0.10390.8590±0.05700.8590 \pm0.05700.07670.0767 (+)
EvoDiff (Ours)20%0.8111±0.06670.8111 \pm0.06670.5833±0.09500.5833 \pm0.09500.3333±0.16670.3333 \pm0.16670.8333±0.16670.8333 \pm0.16670.6157±0.20170.6157 \pm0.20170.7137±0.05130.7137 \pm0.05130.3955±0.16070.3955 \pm0.16070.8544±0.06670.8544 \pm0.06670.07670.0767 (+)
Random30%0.7601±0.10680.7601 \pm0.10680.5500±0.03340.5500 \pm0.03340.2000±0.12470.2000 \pm0.12470.9000±0.02390.9000 \pm0.02390.4333±0.24950.4333 \pm0.24950.6936±0.23940.6936 \pm0.23940.2722±0.16220.2722 \pm0.16220.8353±0.07220.8353 \pm0.07220.08780.0878 (−)
VAAL [57]30%0.7776±0.05310.7776 \pm0.05310.5833±0.04760.5833 \pm0.04760.4000±0.24950.4000 \pm0.24950.7667±0.14030.7667 \pm0.14030.6190±0.31300.6190 \pm0.31300.7578±0.12940.7578 \pm0.12940.3883±0.20430.3883 \pm0.20430.8401±0.14470.8401 \pm0.14470.04820.0482 (−)
Core-Set [58]30%0.7776±0.05310.7776 \pm0.05310.5833±0.04760.5833 \pm0.04760.4000±0.24950.4000 \pm0.24950.7667±0.14030.7667 \pm0.14030.6190±0.31300.6190 \pm0.31300.7578±0.12940.7578 \pm0.12940.3883±0.20430.3883 \pm0.20430.8401±0.14470.8401 \pm0.14470.04820.0482 (−)
Max-Entropy [59]30%0.7766±0.05130.7766 \pm0.05130.5833±0.04560.5833 \pm0.04560.1333±0.16670.1333 \pm0.16670.9500±0.04080.9500 \pm0.04080.3333±0.28790.3333 \pm0.28790.7683±0.16990.7683 \pm0.16990.1889±0.12430.1889 \pm0.12430.8460±0.14630.8460 \pm0.14630.08820.0882 (−)
BALD [60]30%0.7511±0.07260.7511 \pm0.07260.5833±0.05060.5833 \pm0.05060.3333±0.12470.3333 \pm0.12470.8333±0.11780.8333 \pm0.11780.5667±0.02570.5667 \pm0.02570.7144±0.05670.7144 \pm0.05670.3955±0.11490.3955 \pm0.11490.8494±0.08580.8494 \pm0.08580.08820.0882 (+)
EvoDiff (Ours)30%0.8544±0.06670.8544 \pm0.06670.5967±0.06670.5967 \pm0.06670.1000±0.20000.1000 \pm0.20000.9667±0.06670.9667 \pm0.06670.1200±0.24000.1200 \pm0.24000.6872±0.04100.6872 \pm0.04100.1091±0.21820.1091 \pm0.21820.9039±0.08660.9039 \pm0.08660.04820.0482 (+)

Table S1 (PDF p. 36). Comparison of ResNet-50 classification metrics between EvoDiff and other AL methods on test sets. Values are reported as mean ±\pm s.d. over five folds. Differences between each active learning strategy and random sampling were assessed by two-sided paired t-tests, and all reported q-values are Benjamini–Hochberg-adjusted p-values computed across the five strategy-versus-random comparisons within each annotation budget and metric. (Bold represents the best result, Underline represents the second best result).

DatasetMethodAnnotation BudgetAccbAccSensSpecPPVNPVF1AUCq-value
CUBS[50]FSL (upper bound)100%0.8906±0.04510.8648±0.03120.7615±0.06320.9681±0.01840.8515±0.07620.9465±0.01350.8016±0.04750.9368±0.0136-
CUBS[50]Random10%0.7122±0.04570.5351±0.02290.1923±0.09020.8779±0.08400.2901±0.09440.8254±0.00790.2176±0.05270.7290±0.0287-
CUBS[50]VAAL[57]10%0.7257±0.05280.5567±0.03170.2692±0.08600.8442±0.03230.2805±0.03740.8343±0.01260.2718±0.06310.7421±0.04180.0029 (+)
CUBS[50]Core-Set[58]10%0.7317±0.02040.5601±0.04120.3769±0.06880.7434±0.05700.2556±0.04870.8380±0.01680.3030±0.05340.7423±0.04580.0029 (+)
CUBS[50]Max-Entropy[59]10%0.7462±0.03210.5097±0.01240.1769±0.02110.8425±0.03830.2091±0.02470.8163±0.00450.1900±0.01230.7674±0.03680.0003 (+)
CUBS[50]BALD[60]10%0.7403±0.04160.5610±0.04680.3308±0.07500.7912±0.06980.2805±0.08300.8367±0.01860.2980±0.06060.7850±0.04510.0038 (+)
CUBS[50]EvoDiff (Ours)10%0.7951±0.00450.6165±0.03360.4154±0.06320.8177±0.04450.3488±0.05470.8588±0.01330.3769±0.04840.8242±0.02740.0001 (+)
CUBS[50]Random20%0.7410±0.05610.5586±0.03640.3154±0.06880.8018±0.05030.2735±0.06220.8358±0.01420.2899±0.05310.7753±0.0367-
CUBS[50]VAAL[57]20%0.7113±0.01510.6334±0.05970.4154±0.15000.8513±0.06300.4067±0.08180.8647±0.02510.3934±0.09580.7693±0.04740.0089 (−)
CUBS[50]Core-Set[58]20%0.7413±0.05580.5653±0.02320.2615±0.03220.8690±0.02020.3175±0.05360.8364±0.00810.2864±0.03960.7912±0.03190.0116 (+)
CUBS[50]Max-Entropy[59]20%0.7916±0.03920.5717±0.02650.2462±0.04390.8973±0.03460.3722±0.10890.8380±0.00910.2918±0.04820.7855±0.05020.0485 (+)
CUBS[50]BALD[60]20%0.7756±0.07970.5671±0.02110.2615±0.05700.8726±0.02840.3215±0.04050.8372±0.00810.2863±0.04220.8246±0.05050.0089 (+)
CUBS[50]EvoDiff (Ours)20%0.8218±0.01840.6360±0.02590.4385±0.05830.8336±0.02960.3792±0.03850.8660±0.01060.4051±0.03950.8671±0.04930.0993 (+)
CUBS[50]Random30%0.7899±0.04110.5809±0.03250.2769±0.05700.8850±0.01080.3554±0.06630.8418±0.01160.3111±0.06110.8219±0.0237-
CUBS[50]VAAL[57]30%0.7657±0.02030.6043±0.04290.3308±0.10030.8580±0.06110.3979±0.07940.8494±0.01900.3501±0.07230.8219±0.03290.0137 (−)
CUBS[50]Core-Set[58]30%0.7799±0.04960.6631±0.06640.4077±0.11410.9186±0.02110.5325±0.12870.8701±0.01590.4613±0.12030.8228±0.04520.0538 (+)
CUBS[50]Max-Entropy[59]30%0.8186±0.02080.6602±0.04020.4000±0.10390.9204±0.05310.5758±0.14780.8701±0.01590.4550±0.07230.8450±0.06230.0974 (+)
CUBS[50]BALD[60]30%0.8023±0.08020.6592±0.03940.4154±0.06880.9027±0.01390.5973±0.10590.8704±0.07220.4526±0.07220.8587±0.04110.0546 (+)
CUBS[50]EvoDiff (Ours)30%0.8424±0.02610.6291±0.03010.4077±0.05540.9544±0.01840.5983±0.12750.8565±0.01040.4373±0.06390.8692±0.06390.0538 (+)
Fatty Liver[51]FSL (upper bound)100%0.9471±0.01350.8846±0.03610.8486±0.11340.9785±0.04680.9419±0.04680.9488±0.02820.8888±0.02880.9258±0.0282-
Fatty Liver[51]Random10%0.8469±0.07660.8492±0.04910.8108±0.04910.8875±0.07740.9419±0.02840.8677±0.07690.8335±0.02960.9258±0.0769-
Fatty Liver[51]VAAL[57]10%0.8294±0.03600.8480±0.03600.7459±0.10890.9500±0.04680.9374±0.04680.8397±0.03600.8308±0.03600.9163±0.04680.0434 (−)
Fatty Liver[51]Core-Set[58]10%0.8566±0.07200.7921±0.07200.6216±0.15190.9625±0.03600.9774±0.03600.8067±0.07200.7447±0.07200.9222±0.03600.0417 (−)
Fatty Liver[51]Max-Entropy[59]10%0.8414±0.05350.8537±0.05350.8332±0.09930.9625±0.03600.9796±0.03600.8428±0.05350.8585±0.05350.9376±0.03600.0434 (+)
Fatty Liver[51]BALD[60]10%0.8851±0.03360.8557±0.03360.8232±0.11340.9625±0.03600.9416±0.03600.8723±0.03360.8797±0.03360.9367±0.03600.0709 (+)
Fatty Liver[51]EvoDiff (Ours)10%0.8815±0.03360.8857±0.03360.8324±0.11340.9750±0.03600.9416±0.03600.8880±0.03360.8845±0.03360.9534±0.02800.0799 (+)
Fatty Liver[51]Random20%0.8753±0.08100.9328±0.02110.9405±0.05090.9250±0.02500.9668±0.02500.9531±0.01920.9531±0.01920.9431±0.0134-
Fatty Liver[51]VAAL[57]20%0.8417±0.09220.9534±0.02850.9568±0.04710.9500±0.02500.9779±0.02690.9666±0.01950.9666±0.01950.9195±0.00870.0019 (−)
Fatty Liver[51]Core-Set[58]20%0.8490±0.07970.9218±0.05000.8811±0.10620.9625±0.03600.9819±0.01510.9254±0.01610.9254±0.01610.9240±0.01870.0019 (−)
Fatty Liver[51]Max-Entropy[59]20%0.8934±0.09130.9423±0.02160.8686±0.04221.0000±0.00001.0000±0.00000.9980±0.00000.9939±0.02480.9800±0.02020.0019 (−)
Fatty Liver[51]BALD[60]20%0.8987±0.04420.9432±0.02050.8655±0.05160.9500±0.02500.9766±0.01950.9766±0.01950.9240±0.01870.9721±0.03010.0380 (+)
Fatty Liver[51]EvoDiff (Ours)20%0.9240±0.04910.9436±0.02160.9622±0.04711.0000±0.00000.9766±0.01950.9902±0.00980.9624±0.03010.9723±0.03690.0019 (+)
Fatty Liver[51]Random30%0.9157±0.06880.9392±0.03160.9784±0.02020.9000±0.02500.9579±0.02040.9784±0.02020.9679±0.01810.9669±0.0107-
Fatty Liver[51]VAAL[57]30%0.8763±0.08590.9507±0.03460.9514±0.02950.9500±0.02500.9781±0.02950.9637±0.02950.9637±0.02950.9333±0.00420.0011 (−)
Fatty Liver[51]Core-Set[58]30%0.8870±0.08310.9784±0.01080.9568±0.02161.0000±0.00001.0000±0.00000.9778±0.01140.9778±0.01140.9296±0.00370.0011 (−)
Fatty Liver[51]Max-Entropy[59]30%0.9011±0.04420.9328±0.02630.9405±0.03040.9500±0.02500.9767±0.02020.9525±0.03040.9525±0.03040.9424±0.00370.0011 (−)
Fatty Liver[51]BALD[60]30%0.9371±0.04420.9770±0.01910.9459±0.03041.0000±0.00001.0000±0.00000.9718±0.01800.9718±0.01800.9543±0.00280.0011 (−)
Fatty Liver[51]EvoDiff (Ours)30%0.9525±0.03770.9730±0.01910.9459±0.03041.0000±0.00001.0000±0.00000.9781±0.01800.9781±0.01800.9730±0.01410.0839 (+)

Table S1 (PDF p. 37). Comparison of ResNet-50 classification metrics between EvoDiff and other AL methods on test sets. Values are reported as mean ± s.d. over five folds. Differences between each active learning strategy and random sampling were assessed by two-sided paired t-tests, and all reported q-values are Benjamini–Hochberg-adjusted p-values computed across the five strategy-versus-random comparisons within each annotation budget and metric. (Bold represents the best result, Underline represents the second best result).

DatasetMethodAnnotation BudgetAccbAccSensSpecPPVNPVF1AUCq-value
MMOTU[52]FSL (upper bound)100%0.9410±0.02120.7282±0.02750.9422±0.02380.5143±0.04070.8986±0.00890.6707±0.10890.9198±0.01470.9897±0.0100-
MMOTU[52]Random10%0.7564±0.04810.5586±0.05770.8172±0.07820.3000±0.10290.8418±0.02120.2711±0.08650.8280±0.04800.8370±0.0539-
MMOTU[52]VAAL[57]10%0.7393±0.08660.6266±0.03860.8531±0.04800.4000±0.02990.8663±0.01220.3872±0.09800.8593±0.03030.8361±0.01860.0103 (−)
MMOTU[52]Core-Set[58]10%0.7630±0.03530.5323±0.02870.9359±0.01780.1286±0.06960.8310±0.00970.2951±0.06510.8802±0.00620.8409±0.03260.0204 (+)
MMOTU[52]Max-Entropy[59]10%0.7747±0.06210.5792±0.04740.9156±0.03470.2429±0.12470.8475±0.01700.3715±0.08990.8797±0.00980.8661±0.02480.0042 (+)
MMOTU[52]BALD[60]10%0.7630±0.08610.5934±0.02510.8797±0.03890.3071±0.08600.8535±0.01030.3615±0.02090.8658±0.01420.8509±0.02020.0204 (+)
MMOTU[52]EvoDiff (Ours)10%0.8095±0.04960.6242±0.04740.8484±0.04610.4000±0.11120.8666±0.01920.3703±0.06140.8567±0.02270.8748±0.04480.0270 (+)
MMOTU[52]Random20%0.7987±0.01190.6294±0.03600.9016±0.01960.3571±0.09110.8655±0.01370.4392±0.02330.8829±0.00300.8672±0.0420-
MMOTU[52]VAAL[57]20%0.7877±0.08440.6631±0.06150.8547±0.02950.4714±0.11120.8811±0.02380.4161±0.08100.8675±0.02270.8412±0.05630.0897 (−)
MMOTU[52]Core-Set[58]20%0.8168±0.08500.6475±0.03200.9094±0.03930.3857±0.06870.8714±0.01170.5042±0.12020.8896±0.01960.8903±0.03040.0256 (+)
MMOTU[52]Max-Entropy[59]20%0.8372±0.06930.6304±0.02850.9250±0.04080.3357±0.06490.8644±0.01000.5172±0.12020.8933±0.01980.8842±0.04490.0505 (+)
MMOTU[52]BALD[60]20%0.8248±0.06420.6221±0.02630.9156±0.03380.3286±0.06870.8620±0.00950.4708±0.07140.8877±0.01480.8911±0.02690.0294 (+)
MMOTU[52]EvoDiff (Ours)20%0.8320±0.01920.6987±0.02250.9187±0.02250.4786±0.05420.8897±0.00890.5682±0.05080.9038±0.01020.9018±0.03340.0112 (+)
MMOTU[52]Random30%0.8359±0.03590.7056±0.02110.9469±0.02610.4643±0.05650.8900±0.00860.6724±0.09010.9174±0.01070.8966±0.0131-
MMOTU[52]VAAL[57]30%0.8195±……………………0.0629 (−)
TNK[63]FSL (upper bound)100%0.8937±0.01570.8282±0.02410.7934±0.03700.8476±0.04160.9464±0.01460.8866±0.01460.9211±0.01030.9227±0.0179-
TNK[63]Random10%0.6018±0.01850.5747±0.01850.3696±0.01850.7825±0.02890.4784±0.03290.7024±0.02660.4040±0.06260.6180±0.0403-
TNK[63]VAAL[57]10%0.6625±0.02100.5123±0.02620.1141±0.19190.9205±0.13960.2004±0.17200.6638±0.17480.1025±0.17480.6044±0.03350.0329 (−)
TNK[63]Core-Set[58]10%0.7023±0.05990.5805±0.02330.3802±0.16720.7808±0.06820.5044±0.09850.7096±0.02800.4052±0.08550.6125±0.02430.0673 (−)
TNK[63]Max-Entropy[59]10%0.7202±0.05990.5427±0.02870.1719±0.10910.9135±0.05580.4113±0.09880.6774±0.01650.2366±0.13500.6582±0.03610.0673 (+)
TNK[63]BALD[60]10%0.7214±0.08310.5778±0.04220.3479±0.14230.8515±0.06800.5342±0.09880.7012±0.13670.3603±0.13670.6312±0.04510.0747 (+)
TNK[63]EvoDiff (Ours)10%0.7048±0.04510.5639±0.01130.4759±0.11130.6699±0.10020.7426±0.01680.7072±0.01680.4343±0.04410.6683±0.01950.0329 (+)
TNK[63]Random20%0.7109±0.01190.5530±0.03090.2562±0.08990.8498±0.04600.4714±0.02190.6869±0.07880.3008±0.04190.6666±0.1002-
TNK[63]VAAL[57]20%0.6372±0.08400.5486±0.03090.2264±0.13180.8428±0.08320.4744±0.07090.6816±0.11420.2992±0.08780.6284±0.10020.0862 (−)
TNK[63]Core-Set[58]20%0.6989±0.04840.5619±0.02690.2818±0.23790.8140±0.03180.4922±0.09190.6915±0.11420.3363±0.15780.6410±0.02630.0862 (−)
TNK[63]Max-Entropy[59]20%0.7463±0.05090.5986±0.04580.3635±0.20360.7371±0.14840.5520±0.06650.7191±0.04590.3792±0.15780.6960±0.06130.0862 (+)
TNK[63]BALD[60]20%0.7699±0.05590.5867±0.03720.4664±0.20230.7737±0.14880.4778±0.03590.7203±0.06140.4293±0.11960.7032±0.11160.0862 (+)
TNK[63]EvoDiff (Ours)20%0.7736±0.01540.5980±0.06920.4941±0.20360.7260±0.04010.5591±0.06650.7223±0.02800.4770±0.02680.7045±0.04160.0862 (+)
TNK[63]Random30%0.7377±0.03160.5797±0.03560.3289±0.05540.8306±0.05840.5066±0.05840.7112±0.04600.4061±0.12310.6742±0.0240-
TNK[63]VAAL[57]30%0.7075±0.05540.5918±0.03230.3653±0.17340.8183±0.07520.5112±0.04600.7043±0.0730.3897±0.08260.6463±0.0660.0882 (−)
TNK[63]Core-Set[58]30%0.7509±0.02600.5858±0.03220.3174±0.09120.8541±0.04410.5364±0.0730.6740±0.1390.3978±0.0860.6740±0.0660.0882 (−)
TNK[63]Max-Entropy[59]30%0.7682±0.07260.5751±0.02760.3388±0.17440.8114±0.11060.5375±0.02940.7025±0.12470.3684±0.15720.7428±0.01520.0797 (+)
TNK[63]BALD[60]30%0.7877±0.05120.5755±0.03070.3265±0.17440.8253±0.05420.5228±0.02140.7020±0.12550.3657±0.15550.7329±0.01520.0797 (+)
TNK[63]EvoDiff (Ours)30%0.8126±0.03820.5930±0.04220.3240±0.09070.8620±0.03530.5591±0.01930.7084±0.01930.4021±0.05970.8389±0.02000.0974 (+)

Table S1 (PDF p. 38). Comparison of ResNet-50 classification metrics between EvoDiff and other AL methods on test sets. Values are reported as mean ± s.d. over five folds. Differences between each active learning strategy and random sampling were assessed by two-sided paired t-tests, and all reported qq-values are Benjamini–Hochberg-adjusted pp-values computed across the five strategy-versus-random comparisons within each annotation budget and metric. (Bold represents the best result, Underline represents the second best result).

DatasetMethodAnnotation BudgetAccbAccSensSpecPPVNPVF1AUCq-value
Appendix⁴FSL (upper bound)100%0.6435±0.02490.5590±0.01800.8828±0.03080.2353±0.01440.6633±0.00830.5445±0.01800.7573±0.01800.6471±0.0349-
Random10%0.4652±0.07630.5150±0.05490.3241±0.15920.7059±0.11010.6291±0.11950.3828±0.03540.4154±0.16750.5509±0.0271-
VAAL⁵⁷10%0.4870±0.07630.5031±0.06460.4414±0.11540.5647±0.05260.6270±0.07050.3771±0.05630.5153±0.09870.5270±0.09490.0679 (−)
Core-Set⁵⁸10%0.4261±0.08080.5351±0.05410.1172±0.16280.9529±0.07670.4571±0.45340.3910±0.03310.1740±0.22410.4769±0.06660.0317 (−)
Max-Entropy⁵⁹10%0.3652±0.00970.4673±0.01550.0759±0.02880.8588±0.05260.4800±0.04470.3524±0.00830.1292±0.04460.4428±0.04660.0003 (−)
BALD⁶⁰10%0.4130±0.11090.5053±0.07080.1517±0.23460.8588±0.12200.3302±0.33740.3811±0.05590.1931±0.27220.4714±0.09680.0333 (−)
EvoDiff (Ours)10%0.5261±0.05630.5000±0.06650.6000±0.14130.4000±0.20960.6370±0.05350.3630±0.09300.6091±0.06770.5748±0.10120.0679 (+)
Random20%0.5609±0.02380.5592±0.03760.5655±0.04630.5530±0.10690.6866±0.03760.4248±0.03370.6183±0.02200.5655±0.0566-
VAAL⁵⁷20%0.5609±0.04180.5105±0.03880.7034±0.11330.3176±0.13540.6388±0.02790.3873±0.05960.6656±0.05280.5448±0.03300.0750 (−)
Core-Set⁵⁸20%0.6174±0.02910.5748±0.03680.7379±0.03930.4117±0.08320.6826±0.02820.4781±0.04720.7085±0.02200.5951±0.06990.0750 (+)
Max-Entropy⁵⁹20%0.5652±0.02660.5408±0.04970.6345±0.11060.4471±0.19330.6696±0.04700.4057±0.06560.6443±0.04550.5586±0.08370.0913 (−)
BALD⁶⁰20%0.5000±0.04610.4793±0.05160.5586±0.08590.4000±0.12750.6155±0.03920.3440±0.06460.5825±0.05180.4836±0.04380.0015 (−)
EvoDiff (Ours)20%0.6174±0.06630.5992±0.06730.6690±0.11070.5294±0.13800.7101±0.04930.4881±0.09990.6850±0.06620.6381±0.04910.0235 (+)
Random30%0.6478±0.02830.6163±0.04620.7448±0.04620.4823±0.14650.6619±0.03050.5444±0.04920.7248±0.03840.6783±0.0607-
VAAL⁵⁷30%0.5870±0.04070.5434±0.02550.7104±0.04620.3765±0.11470.6619±0.03840.4240±0.02840.6843±0.02840.5574±0.06390.0852 (−)
Core-Set⁵⁸30%0.5869±0.04070.5240±0.03380.7655±0.07070.2823±0.04920.6418±0.07280.4204±0.04000.6992±0.01820.5562±0.04060.0029 (−)
Max-Entropy⁵⁹30%0.6391±0.01950.6116±0.03070.7172±0.05110.5059±0.03100.7146±0.02890.5117±0.01820.7143±0.01820.6385±0.05010.0166 (−)
BALD⁶⁰30%0.5261±0.01140.5243±0.03070.5310±0.24210.5176±0.08870.6283±0.10390.4126±0.02080.5577±0.07700.5229±0.05750.0029 (−)
EvoDiff (Ours)30%0.7000±0.04070.6574±0.03380.8207±0.07800.4941±0.03130.7347±0.03130.6289±0.04550.7714±0.04550.6686±0.05350.0123 (+)
BBUS⁴⁵FSL (upper bound)100%0.8273±0.02850.8585±0.01960.9143±0.00960.7867±0.04640.8359±0.02190.7626±0.04740.8361±0.02060.9131±0.0123-
Random10%0.7485±0.03950.6378±0.05730.3333±0.11170.9422±0.02540.7271±0.11450.7528±0.03330.4511±0.01100.7874±0.04820.0088 (−)
VAAL⁵⁷10%0.7879±0.02140.7378±0.02970.6000±0.05430.8756±0.01220.6961±0.03290.8246±0.02030.6421±0.04350.8493±0.01020.0664 (−)
Core-Set⁵⁸10%0.7636±0.02770.6641±0.05050.3965±0.03620.9378±0.06170.4520±0.01910.7810±0.06190.4162±0.03830.7893±0.06250.0095 (−)
Max-Entropy⁵⁹10%0.7989±0.01970.7674±0.04280.5810±0.02440.9688±0.04290.7429±0.04890.6381±0.05720.5883±0.03840.9294±0.02480.0356 (−)
BALD⁶⁰10%0.7912±0.03070.7975±0.07830.5833±0.24410.8578±0.08580.7249±0.04290.6787±0.03320.5878±0.03570.8789±0.04890.0154 (−)
EvoDiff (Ours)10%0.8060±0.03110.8600±0.03240.7811±0.10430.9467±0.03310.8113±0.03330.9270±0.06060.8581±0.04020.9020±0.02360.0030 (+)
Random20%0.8485±0.03030.7975±0.07830.6572±0.07830.9378±0.04560.8432±0.09400.8551±0.02500.7332±0.04560.8534±0.0290-
VAAL⁵⁷20%0.7758±0.02290.6603±0.07060.3429±0.07060.9778±0.02220.8800±0.01690.7616±0.01960.4902±0.04070.8456±0.02220.0535 (−)
Core-Set⁵⁸20%0.8848±0.02030.8571±0.02700.7809±0.05430.9245±0.06960.8470±0.02200.9018±0.02200.8115±0.03300.9089±0.02490.0030 (+)
Max-Entropy⁵⁹20%0.7818±0.08340.7003±0.05430.4762±0.14380.9245±0.04380.6455±0.03780.8112±0.03900.4740±0.13800.8852±0.02490.0162 (+)
BALD⁶⁰20%0.8636±0.03550.8233±0.04670.7143±0.04760.9333±0.04760.8909±0.02000.8750±0.05170.7702±0.02000.8913±0.00810.0017 (+)
EvoDiff (Ours)20%0.9007±0.03510.8824±0.03110.7811±0.10430.9467±0.03310.8091±0.02000.9270±0.06060.8581±0.04020.9020±0.02360.0030 (+)
Random30%0.8636±0.01520.8060±0.04260.7333±0.04260.9645±0.03300.8955±0.02140.8545±0.01520.7509±0.03130.8994±0.0313-
VAAL⁵⁷30%0.8939±0.11930.8461±0.04260.7143±0.39980.9778±0.04260.7514±0.04260.8973±0.01520.7319±0.04070.9146±0.00940.0046 (+)
Core-Set⁵⁸30%0.8485±0.02400.8152±0.05210.7288±0.05210.9600±0.02510.7926±0.07520.8763±0.02750.7531±0.02750.8684±0.01780.0852 (+)
Max-Entropy⁵⁹30%0.8515±0.09090.7895±0.03510.6190±0.06400.9600±0.03510.7164±0.04140.8572±0.02700.6590±0.02700.9007±0.05260.0072 (−)
BALD⁶⁰30%0.8485±0.02400.8152±0.05210.7288±0.05210.9600±0.02510.7926±0.07520.8763±0.02750.7531±0.02750.8684±0.01780.0033 (+)
EvoDiff (Ours)30%0.8894±0.04490.8927±0.02030.9143±0.03990.8711±0.07920.7861±0.02000.9572±0.01660.8381±0.01660.9533±0.01890.0861 (+)

Table S2. Comparison of USFM classification metrics between EvoDiff and other AL methods on the test sets. Values are reported as mean ± s.d. over five folds. Statistical differences between each active learning strategy and Random sampling were assessed using two-sided paired t-tests based on fold-wise AUC. The reported q-values are the Benjamini–Hochberg-calibrated p-values across the five strategy-versus-Random comparisons within each dataset and annotation budget; the appended (+) or (−) indicates the sign of mean_diff(method-Random) for AUC. (Bold represents the best result, and Underline represents the second best result).

DatasetMethodAnnotation BudgetAccbAccSensSpecPPVNPVF1AUCq-value
BUS-BRA[46]FSL (upper bound)100%0.9263±0.03780.8937±0.05370.8000±0.09980.9875±0.00890.9669±0.02620.9119±0.04030.8737±0.07150.9459±0.0214-
Random10%0.7810±0.02460.7294±0.03580.5807±0.07820.8781±0.02680.6990±0.04410.8130±0.02670.6319±0.05410.7931±0.0518-
VAAL[57]10%0.7084±0.02310.6123±0.02480.3355±0.03500.8891±0.02370.5962±0.06750.7342±0.01320.4288±0.04310.7164±0.01680.0002 (−)
Core-Set[58]10%0.7758±0.04660.7213±0.06020.5645±0.10760.8781±0.03570.6916±0.07470.8075±0.04150.6189±0.08750.7279±0.03110.0249 (−)
Max-Entropy[59]10%0.7179±0.01510.6135±0.02510.3129±0.07080.9141±0.03220.6441±0.05440.7336±0.01540.4162±0.06290.7864±0.04430.0175 (−)
BALD[60]10%0.7590±0.01260.6756±0.02470.4355±0.07300.9156±0.03090.7215±0.05840.7707±0.01830.5383±0.04930.7430±0.03630.0012 (−)
EvoDiff (Ours)10%0.7705±0.02590.7324±0.03950.6226±0.08270.8422±0.01780.6554±0.03160.8227±0.03180.6372±0.05460.8277±0.01460.0012 (+)
Random20%0.8231±0.07040.7681±0.12830.6097±0.29640.9266±0.04370.8377±0.09540.8416±0.08420.6446±0.27770.8684±0.0850-
VAAL[57]20%0.7484±0.01760.6902±0.02340.5226±0.07620.8578±0.04570.6471±0.05580.7887±0.02060.5736±0.03830.7491±0.02520.0233 (−)
Core-Set[58]20%0.8116±0.01980.7953±0.01320.7484±0.05770.8422±0.05140.7031±0.05830.8747±0.02010.7218±0.01800.8681±0.01260.0992 (−)
Max-Entropy[59]20%0.8379±0.02330.8015±0.02690.6968±0.03850.9063±0.01830.7828±0.04030.8606±0.01730.7371±0.03780.8668±0.02980.0992 (−)
BALD[60]20%0.8526±0.01180.8224±0.01470.7355±0.03150.9094±0.01620.7981±0.02730.8767±0.01220.7650±0.01940.8821±0.02010.0992 (+)
EvoDiff (Ours)20%0.8495±0.01840.8259±0.02170.7581±0.06550.8938±0.04190.7819±0.06010.8852±0.02450.7664±0.02620.8987±0.01370.0992 (+)
Random30%0.8452±0.07650.7845±0.07340.6097±0.12610.9595±0.02800.8473±0.08020.8543±0.03410.6830±0.05610.8699±0.0896-
VAAL[57]30%0.7990±0.01800.7726±0.03740.6968±0.09570.8484±0.02380.6924±0.03140.8543±0.03410.6906±0.05170.8441±0.02360.0715 (−)
Core-Set[58]30%0.8581±0.01920.8760±0.01830.8613±0.01840.8906±0.02210.7931±0.02210.9298±0.00980.8256±0.05170.9237±0.01230.0647 (+)
Max-Entropy[59]30%0.7590±0.11130.6481±0.01920.3290±0.42990.9672±0.04700.5380±0.12490.7670±0.12490.3431±0.43030.8208±0.09260.0647 (−)
BALD[60]30%0.8242±0.08970.7847±0.05970.6171±0.10500.9894±0.05730.6014±0.10990.8692±0.03420.6388±0.05730.8596±0.12420.0903 (−)
EvoDiff (Ours)30%0.8361±0.07070.7747±0.03350.6097±0.09740.9931±0.03930.8450±0.10000.8444±0.03950.6662±0.12730.8489±0.07150.0715 (−)
UDIAT[47]FSL (upper bound)100%0.8555±0.03930.7983±0.05830.1667±0.11780.9833±0.05830.7144±0.16150.8590±0.04470.2714±0.17790.7139±0.1000-
Random10%0.7222±0.03930.5833±0.05830.2333±0.14910.9667±0.03330.6667±0.40820.7718±0.04560.3349±0.20360.7139±0.1014-
VAAL[57]10%0.7000±0.16010.6417±0.03280.4667±0.24270.8167±0.16690.7540±0.04510.7178±0.20360.5038±0.16690.7417±0.05780.1000 (+)
Core-Set[58]10%0.6778±0.02480.5517±0.00280.0367±0.07461.0000±0.00000.6745±0.12780.2000±0.20000.0571±0.12780.5917±0.08370.0555 (−)
Max-Entropy[59]10%0.7222±0.03930.5833±0.05830.0333±0.20411.0000±0.00000.2000±0.20000.7745±0.12440.0571±0.20410.6567±0.06900.1000 (+)
BALD[60]10%0.7667±0.01930.7000±0.01930.5000±0.00000.9000±0.00000.8489±0.04020.7800±0.08520.5855±0.04020.7256±0.06900.1000 (+)
EvoDiff (Ours)10%0.7667±0.03930.7000±0.03930.5000±0.11790.9000±0.05830.8489±0.04020.7800±0.08520.5855±0.04020.7056±0.06900.1000 (−)
Random20%0.7789±0.04650.6917±0.08120.4000±0.19000.9833±0.06560.9600±0.03730.7704±0.05390.5359±0.17240.8667±0.0654-
VAAL[57]20%0.7778±0.03930.7417±0.01860.6333±0.07460.8500±0.09130.7143±0.14700.8236±0.01470.6564±0.02290.8222±0.04650.0254 (−)
Core-Set[58]20%0.7667±0.04650.6833±0.08640.4333±0.22360.9333±0.06970.8267±0.06240.7737±0.06400.5248±0.17710.8000±0.03040.0191 (−)
Max-Entropy[59]20%0.6444±0.13800.6167±0.15140.5333±0.21730.7000±0.11790.4693±0.11030.7512±0.10350.4969±0.19130.6778±0.08700.0893 (−)
BALD[60]20%0.7667±0.06200.6667±0.07790.3667±0.23970.9667±0.08160.8533±0.04900.7546±0.04580.5389±0.14460.8567±0.09130.0893 (+)
EvoDiff (Ours)20%0.7667±0.03050.7417±0.07450.6667±0.11790.8667±0.08160.8467±0.06390.8467±0.06030.6389±0.14460.8570±0.04280.0893 (+)
Random30%0.7667±0.04650.7083±0.07450.2667±0.10031.0000±0.00001.0000±0.00000.7333±0.04990.4143±0.11650.8472±0.0581-
VAAL[57]30%0.7222±0.08780.6833±0.10030.5000±0.11790.9167±0.09130.7800±0.08520.7784±0.08520.5964±0.19180.7556±0.09750.0245 (−)
Core-Set[58]30%0.7778±0.03930.7083±0.05830.5000±0.11790.9167±0.09130.8583±0.08520.7834±0.08520.5946±0.08520.8583±0.03860.0777 (+)
Max-Entropy[59]30%0.7000±0.03040.5500±0.03040.1000±0.09131.0000±0.00000.6000±0.09130.7332±0.07470.1714±0.15650.5500±0.15450.0056 (−)
BALD[60]30%0.7556±0.03050.6333±0.03050.2667±0.10031.0000±0.00001.0000±0.00000.7324±0.11740.4143±0.15650.8194±0.07320.0540 (−)
EvoDiff (Ours)30%0.7889±0.04650.7000±0.04650.4333±0.04560.9667±0.07450.9200±0.07450.7738±0.08510.5758±0.08510.7960±0.05050.0245 (−)

Table S2 (PDF p. 40). Comparison of USFM classification metrics between EvoDiff and other AL methods on the test sets. Values are reported as mean ± s.d. over five folds. Statistical differences between each active learning strategy and Random sampling were assessed using two-sided paired tt-tests based on fold-wise AUC. The reported qq-values are the Benjamini–Hochberg-calibrated pp-values across the five strategy-versus-Random comparisons within each dataset and annotation budget; the appended (+) or (−) indicates the sign of mean_diff(method-Random) for AUC. (Bold represents the best result, and Underline represents the second best result).

DatasetMethodAnnotation BudgetAccbAccSensSpecPPVNPVF1AUCq-value
CUBS50FSL (upper bound)100%0.8259±0.00940.5346±0.02510.0692±0.05011.0000±0.00000.8000±0.44720.8237±0.00780.1262±0.08940.7455±0.0266-
CUBS50Random10%0.8129±0.01440.5770±0.02980.2000±0.07400.9540±0.02450.5180±0.10120.8385±0.01060.2802±0.07570.6170±0.0658-
CUBS50VAAL[57]10%0.7856±0.03070.5246±0.02850.1077±0.03220.9416±0.03340.3283±0.13280.8208±0.00930.1602±0.05270.5302±0.05670.0115 (−)
CUBS50Core-Set[58]10%0.7813±0.01810.5397±0.02200.1538±0.04710.9257±0.02390.3318±0.08000.8263±0.00760.2066±0.05020.5243±0.02280.0475 (−)
CUBS50Max-Entropy[59]10%0.8101±0.02130.5397±0.01570.1077±0.03220.9717±0.02900.5771±0.28800.8256±0.00490.1740±0.04150.5244±0.05960.0097 (−)
CUBS50BALD[60]10%0.7612±0.03200.5481±0.03290.2077±0.06440.8885±0.04090.3119±0.09640.8298±0.01160.2447±0.06140.5321±0.02780.0097 (−)
CUBS50EvoDiff (Ours)10%0.7971±0.02560.5850±0.03870.2462±0.07980.9239±0.03110.4389±0.11390.8421±0.01380.3095±0.07860.6555±0.07700.0097 (+)
CUBS50Random20%0.7727±0.02770.5433±0.03100.1769±0.04390.9097±0.02830.3231±0.10700.8276±0.01040.2268±0.05810.5361±0.0262-
CUBS50VAAL[57]20%0.7856±0.02120.5276±0.02550.1154±0.03850.9398±0.02110.3177±0.12370.8219±0.00830.1680±0.05700.5662±0.02840.0355 (+)
CUBS50Core-Set[58]20%0.8058±0.01130.5489±0.01500.1384±0.03440.9593±0.01610.4470±0.08610.8288±0.00490.2090±0.04590.6040±0.03520.0039 (+)
CUBS50Max-Entropy[59]20%0.8144±0.01560.5483±0.02740.1231±0.05010.9735±0.01250.5264±0.18420.8283±0.00890.1973±0.07390.5754±0.03130.0122 (+)
CUBS50BALD[60]20%0.7612±0.01860.5511±0.03030.2154±0.05830.8867±0.01920.3035±0.06070.8309±0.01080.2509±0.05970.5645±0.04120.0392 (+)
CUBS50EvoDiff (Ours)20%0.8043±0.02800.5865±0.02620.2385±0.03220.9345±0.03100.4765±0.12760.8420±0.00840.3152±0.05270.6637±0.03490.0039 (+)
CUBS50Random30%0.7957±0.02190.5733±0.02280.2231±0.03660.9277±0.02090.4278±0.10980.8384±0.00890.2909±0.04330.6395±0.0407-
CUBS50VAAL[57]30%0.8029±0.02270.5975±0.01730.2692±0.06660.9257±0.02770.4644±0.11550.8464±0.01150.3363±0.06160.6939±0.04050.0405 (+)
CUBS50Core-Set[58]30%0.8403±0.01860.6264±0.03870.2864±0.07500.9682±0.01340.6755±0.01330.8548±0.01330.3970±0.08760.6401±0.03620.0347 (+)
CUBS50Max-Entropy[59]30%0.8187±0.01710.6161±0.02270.2923±0.04930.9399±0.02020.5369±0.07390.8524±0.00780.3758±0.04830.6988±0.05320.0405 (+)
CUBS50BALD[60]30%0.7885±0.02860.5975±0.02470.2923±0.02100.9027±0.01340.4191±0.06130.8470±0.00800.3436±0.04270.6030±0.02130.0527 (−)
CUBS50EvoDiff (Ours)30%0.8490±0.01710.6583±0.03940.3513±0.06320.9628±0.02540.7016±0.02540.8662±0.01150.4652±0.08590.7061±0.03630.0986 (+)
Fatty-Liver[51]FSL (upper bound)100%0.9547±0.02150.9285±0.01210.9946±0.01480.8625±0.06230.9438±0.05230.9685±0.01210.9857±0.03190.9785±0.0246-
Fatty-Liver[51]Random10%0.7472±0.06200.8441±0.02990.9384±0.01480.2000±0.00000.7436±0.04580.9006±0.01710.8297±0.09790.7648±0.0906-
Fatty-Liver[51]VAAL[57]10%0.8604±0.10220.8042±0.03710.9459±0.06110.6625±0.01870.8831±0.10900.8987±0.09710.9063±0.06110.8987±0.09710.0028 (−)
Fatty-Liver[51]Core-Set[58]10%0.7962±0.03380.8556±0.02810.7135±0.13550.9875±0.02800.9926±0.03200.8497±0.09790.8266±0.09790.8427±0.07990.0128 (−)
Fatty-Liver[51]Max-Entropy[59]10%0.7902±0.03580.7649±0.04880.7351±0.05330.9625±0.02810.9436±0.03970.9625±0.02810.8075±0.09710.8427±0.09790.0028 (−)
Fatty-Liver[51]BALD[60]10%0.8320±0.03150.8498±0.02660.7767±0.05880.9000±0.04251.0000±0.00000.8713±0.03930.8731±0.04340.9375±0.04150.0174 (−)
Fatty-Liver[51]EvoDiff (Ours)10%0.8415±0.03150.8688±0.02990.7730±0.04301.0000±0.00001.0000±0.00000.8753±0.04150.8713±0.04340.9375±0.04150.0128 (−)
Fatty-Liver[51]Random20%0.9510±0.01690.9365±0.02790.9730±0.04240.9000±0.05590.9579±0.02240.9340±0.05050.9653±0.01150.9814±0.0266-
Fatty-Liver[51]VAAL[57]20%0.8642±0.09380.8034±0.17100.9567±0.02420.6500±0.06600.8767±0.10130.9112±0.05050.7568±0.04250.9399±0.06410.0373 (−)
Fatty-Liver[51]Core-Set[58]20%0.8038±0.02860.8488±0.02040.7351±0.04930.9625±0.03420.7824±0.04370.9135±0.05110.9415±0.04280.9215±0.09670.0373 (−)
Fatty-Liver[51]Max-Entropy[59]20%0.6642±0.03260.7488±0.02780.5351±0.05310.9625±0.05260.5836±0.02940.7941±0.05910.5567±0.05910.9476±0.06900.0411 (−)
Fatty-Liver[51]BALD[60]20%0.7920±0.01130.8775±0.01800.7730±0.03790.9625±0.03740.8288±0.05050.9065±0.06900.9280±0.05810.9655±0.05970.0373 (−)
Fatty-Liver[51]EvoDiff (Ours)20%0.8340±0.02640.9123±0.04150.7767±0.03790.9875±0.02800.9946±0.05180.9183±0.13560.9875±0.02800.9183±0.13560.0381 (−)
Fatty-Liver[51]Random30%0.9472±0.02070.9302±0.03240.9730±0.01910.8875±0.06850.9531±0.04330.9366±0.01460.9626±0.02300.9720±0.0230-
Fatty-Liver[51]VAAL[57]30%0.8491±0.13800.7606±0.23800.9384±0.03110.5375±0.05150.8529±0.07800.9074±0.05180.9074±0.05180.8829±0.13550.0518 (−)
Fatty-Liver[51]Core-Set[58]30%0.9019±0.11510.8552±0.19890.9730±0.02060.7375±0.07750.9188±0.06550.9373±0.06550.9373±0.06550.8507±0.13930.0518 (−)
Fatty-Liver[51]Max-Entropy[59]30%0.8604±0.14890.8362±0.19890.9027±0.02030.7625±0.04520.9264±0.04930.8960±0.11780.8960±0.11780.8831±0.11700.0776 (−)
Fatty-Liver[51]BALD[60]30%0.9213±0.02860.9123±0.02030.9622±0.02030.8625±0.04520.9420±0.02800.9515±0.02280.9515±0.02280.9301±0.08540.0086 (+)
Fatty-Liver[51]EvoDiff (Ours)30%0.8451±0.03180.8829±0.02150.7784±0.04510.9875±0.02850.7946±0.44430.8126±0.28550.8126±0.28550.9790±0.0384

Table S2 (PDF p. 41). Comparison of USFM classification metrics between EvoDiff and other AL methods on the test sets. Values are reported as mean ± s.d. over five folds. Statistical differences between each active learning strategy and Random sampling were assessed using two-sided paired tt-tests based on fold-wise AUC. The reported qq-values are the Benjamini–Hochberg-calibrated pp-values across the five strategy-versus-Random comparisons within each dataset and annotation budget; the appended (+) or (−) indicates the sign of mean_diff(method-Random) for AUC. (Bold represents the best result, and Underline represents the second best result).

DatasetMethodAnnotation BudgetAccbAccSensSpecPPVNPVF1AUCq-value
MMOTUFSL (upper bound)100%0.8692±0.01070.8692\pm0.01070.7473±0.02480.7473\pm0.02480.9375±0.00550.9375\pm0.00550.5571±0.04790.5571\pm0.04790.9064±0.00930.9064\pm0.00930.6605±0.03150.6605\pm0.03150.9217±0.00620.9217\pm0.00620.8322±0.01260.8322\pm0.0126-
MMOTURandom10%0.8013±0.02940.8013\pm0.02940.6808±0.03130.6808\pm0.03130.8688±0.03190.8688\pm0.03190.4929±0.04650.4929\pm0.04650.8867±0.01060.8867\pm0.01060.4569±0.07470.4569\pm0.07470.8774±0.01970.8774\pm0.01970.8091±0.01920.8091\pm0.0192-
MMOTUVAAL[57]10%0.8231±0.01540.8231\pm0.01540.5992±0.05540.5992\pm0.05540.9485±0.02800.9485\pm0.02800.2500±0.13120.2500\pm0.13120.8532±0.01950.8532\pm0.01950.5238±0.10850.5238\pm0.10850.8979±0.00910.8979\pm0.00910.7615±0.03380.7615\pm0.03380.01360.0136 (−)
MMOTUCore-Set[58]10%0.8243±0.01470.8243\pm0.01470.5972±0.04940.5972\pm0.04940.9516±0.01020.9516\pm0.01020.2429±0.10530.2429\pm0.10530.8521±0.01680.8521\pm0.01680.5095±0.07760.5095\pm0.07760.8990±0.00750.8990\pm0.00750.7369±0.03170.7369\pm0.03170.00290.0029 (−)
MMOTUMax-Entropy[59]10%0.7974±0.01730.7974\pm0.01730.6562±0.04580.6562\pm0.04580.8766±0.01500.8766\pm0.01500.4357±0.09580.4357\pm0.09580.8769±0.01750.8769\pm0.01750.4324±0.05620.4324\pm0.05620.8766±0.01010.8766\pm0.01010.7755±0.03250.7755\pm0.03250.01360.0136 (−)
MMOTUBALD[60]10%0.8359±0.01070.8359\pm0.01070.7103±0.02810.7103\pm0.02810.9062±0.01990.9062\pm0.01990.5143±0.06960.5143\pm0.06960.8953±0.01160.8953\pm0.01160.5478±0.03440.5478\pm0.03440.9006±0.00730.9006\pm0.00730.8311±0.02220.8311\pm0.02220.01780.0178 (+)
MMOTUEvoDiff (Ours)10%0.8474±0.01460.8474\pm0.01460.7173±0.03050.7173\pm0.03050.9203±0.01690.9203\pm0.01690.5143±0.06490.5143\pm0.06490.8967±0.01190.8967\pm0.01190.5874±0.05200.5874\pm0.05200.9082±0.00900.9082\pm0.00900.8267±0.02400.8267\pm0.02400.00290.0029 (+)
MMOTURandom20%0.8372±0.00730.8372\pm0.00730.7138±0.03030.7138\pm0.03030.9062±0.01660.9062\pm0.01660.5214±0.07400.5214\pm0.07400.8968±0.01290.8968\pm0.01290.5500±0.02640.5500\pm0.02640.9013±0.00470.9013\pm0.00470.8382±0.02030.8382\pm0.0203-
MMOTUVAAL[57]20%0.8166±0.02820.8166\pm0.02820.6651±0.04820.6651\pm0.04820.9016±0.03560.9016\pm0.03560.4286±0.10410.4286\pm0.10410.8786±0.01760.8786\pm0.01760.4928±0.07510.4928\pm0.07510.8895±0.01870.8895\pm0.01870.7990±0.01920.7990\pm0.01920.00900.0090 (−)
MMOTUCore-Set[58]20%0.8346±0.02290.8346\pm0.02290.6872±0.10490.6872\pm0.10490.9172±0.05570.9172\pm0.05570.4571±0.25680.4571\pm0.25680.8884±0.03810.8884\pm0.03810.4461±0.25860.4461\pm0.25860.9008±0.01490.9008\pm0.01490.8114±0.04040.8114\pm0.04040.03350.0335 (−)
MMOTUMax-Entropy[59]20%0.8462±0.02270.8462\pm0.02270.7109±0.03870.7109\pm0.03870.9219±0.01660.9219\pm0.01660.5000±0.06680.5000\pm0.06680.8940±0.01380.8940\pm0.01380.5847±0.07730.5847\pm0.07730.9077±0.01370.9077\pm0.01370.8125±0.03660.8125\pm0.03660.08570.0857 (−)
MMOTUBALD[60]20%0.8436±0.02670.8436\pm0.02670.7484±0.04460.7484\pm0.04460.8969±0.02100.8969\pm0.02100.6000±0.07740.6000\pm0.07740.9112±0.01660.9112\pm0.01660.5615±0.07430.5615\pm0.07430.9039±0.01650.9039\pm0.01650.8581±0.01570.8581\pm0.01570.00900.0090 (+)
MMOTUEvoDiff (Ours)20%0.8474±0.01240.8474\pm0.01240.7257±0.02630.7257\pm0.02630.9156±0.00860.9156\pm0.00860.5357±0.05050.5357\pm0.05050.9002±0.00990.9002\pm0.00990.5811±0.03770.5811\pm0.03770.9078±0.00730.9078\pm0.00730.8423±0.03180.8423\pm0.03180.04390.0439 (+)
MMOTURandom30%0.8910±0.00910.8910\pm0.00910.7745±0.02580.7745\pm0.02580.9622±0.00510.9622\pm0.00510.5928±0.05420.5928\pm0.05420.9149±0.01000.9149\pm0.01000.7481±0.00620.7481\pm0.00620.9351±0.00510.9351\pm0.00510.8395±0.02550.8395\pm0.0255-
MMOTUVAAL[57]30%0.8718±0.00780.8718\pm0.00780.7545±0.00680.7545\pm0.00680.9375±0.00830.9375\pm0.00830.5714±0.06880.5714\pm0.06880.9933±0.00840.9933\pm0.00840.6672±0.00340.6672\pm0.00340.9231±0.00340.9231\pm0.00340.8290±0.03700.8290\pm0.03700.00200.0020 (−)
MMOTUCore-Set[58]30%0.8890±0.01330.8890\pm0.01330.7884±0.01970.7884\pm0.01970.9707±0.00440.9707\pm0.00440.6143±0.29820.6143\pm0.29820.9444±0.03080.9444\pm0.03080.7840±0.13470.7840\pm0.13470.9444±0.00840.9444\pm0.00840.8290±0.03700.8290\pm0.03700.00860.0086 (−)
MMOTUMax-Entropy[59]30%0.8872±0.01400.8872\pm0.01400.7742±0.04000.7742\pm0.04000.9656±0.00640.9656\pm0.00640.5286±0.27820.5286\pm0.27820.9655±0.02610.9655\pm0.02610.6382±0.13470.6382\pm0.13470.9305±0.00880.9305\pm0.00880.8382±0.03430.8382\pm0.03430.04350.0435 (−)
MMOTUBALD[60]30%0.8562±0.02900.8562\pm0.02900.7120±0.04360.7120\pm0.04360.9312±0.04400.9312\pm0.04400.4928±0.27820.4928\pm0.27820.8967±0.03460.8967\pm0.03460.6078±0.08860.6078\pm0.08860.9122±0.01580.9122\pm0.01580.8078±0.08860.8078\pm0.08860.01720.0172 (−)
MMOTUEvoDiff (Ours)30%0.8859±0.01530.8859\pm0.01530.7470±0.03320.7470\pm0.03320.9672±0.01160.9672\pm0.01160.5143±0.06490.5143\pm0.06490.9011±0.01190.9011\pm0.01190.7760±0.06620.7760\pm0.06620.9330±0.00890.9330\pm0.00890.8362±0.03980.8362\pm0.03980.00690.0069 (−)
TNK3FSL (upper bound)100%0.6651±0.02270.6651\pm0.02270.5157±0.03290.5157\pm0.03290.0314±0.06570.0314\pm0.06571.0000±0.00001.0000\pm0.00000.4000±0.04770.4000\pm0.04770.6618±0.01570.6618\pm0.01570.7354±0.03240.7354\pm0.03240.7354±0.03240.7354\pm0.0324-
TNK3Random10%0.7103±0.01840.7103\pm0.01840.6773±0.02880.6773\pm0.02880.5554±0.02830.5554\pm0.02830.7921±0.03190.7921\pm0.03190.5871±0.03210.5871\pm0.03210.7713±0.01040.7713\pm0.01040.7700±0.01190.7700\pm0.01190.7475±0.01190.7475\pm0.01190.00070.0007 (−)
TNK3VAAL[57]10%0.6794±0.01870.6794\pm0.01870.6194±0.01870.6194\pm0.01870.4298±0.06830.4298\pm0.06830.8144±0.04360.8144\pm0.04360.5503±0.04220.5503\pm0.04220.7288±0.04230.7288\pm0.04230.7476±0.04220.7476\pm0.04220.6870±0.04220.6870\pm0.04220.00010.0001 (−)
TNK3Core-Set[58]10%0.6589±0.01790.6589\pm0.01790.6048±0.01790.6048\pm0.01790.4298±0.06830.4298\pm0.06830.7799±0.02990.7799\pm0.02990.5503±0.04220.5503\pm0.04220.7061±0.04230.7061\pm0.04230.7288±0.04220.7288\pm0.04220.6485±0.04380.6485\pm0.04380.00020.0002 (−)
TNK3Max-Entropy[59]10%0.6669±0.01080.6669\pm0.01080.6156±0.01650.6156\pm0.01650.4496±0.04400.4496\pm0.04400.7817±0.03760.7817\pm0.03760.5231±0.02000.5231\pm0.02000.7291±0.01830.7291\pm0.01830.6819±0.01830.6819\pm0.01830.7189±0.01750.7189\pm0.01750.00010.0001 (−)
TNK3BALD[60]10%0.6549±0.01380.6549\pm0.01380.6162±0.00870.6162\pm0.00870.4909±0.01500.4909\pm0.01500.7415±0.02680.7415\pm0.02680.5018±0.02000.5018\pm0.02000.7337±0.00770.7337\pm0.00770.6536±0.02000.6536\pm0.02000.6553±0.04800.6553\pm0.04800.00080.0008 (−)
TNK3EvoDiff (Ours)10%0.6389±0.02110.6389\pm0.02110.6008±0.06200.6008\pm0.06200.4777±0.32940.4777\pm0.32940.7240±0.20570.7240\pm0.20570.5266±0.11250.5266\pm0.11250.7457±0.20960.7457\pm0.20960.6870±0.02110.6870\pm0.02110.6536±0.04800.6536\pm0.04800.00080.0008 (−)
TNK3Random20%0.7440±0.01520.7440\pm0.01520.7112±0.01910.7112\pm0.01910.6060±0.03430.6060\pm0.03430.8175±0.01130.8175\pm0.01130.6364±0.02410.6364\pm0.02410.7968±0.02670.7968\pm0.02670.8066±0.02280.8066\pm0.02280.8066±0.02280.8066\pm0.02280.05630.0563 (−)
TNK3VAAL[57]20%0.7371±0.01110.7371\pm0.01110.7181±0.01920.7181\pm0.01920.6620±0.05500.6620\pm0.05500.7799±0.01370.7799\pm0.01370.6144±0.00970.6144\pm0.00970.8118±0.02910.8118\pm0.02910.7902±0.01070.7902\pm0.01070.7902±0.01070.7902\pm0.01070.08160.0816 (−)
TNK3Core-Set[58]20%0.7418±0.01480.7418\pm0.01480.7324±0.02770.7324\pm0.02770.6869±0.03230.6869\pm0.03230.7755±0.02700.7755\pm0.02700.6193±0.03660.6193\pm0.03660.8252±0.01920.8252\pm0.01920.8152±0.03370.8152\pm0.03370.8152±0.03370.8152\pm0.03370.05630.0563 (+)
TNK3Max-Entropy[59]20%0.7434±0.01690.7434\pm0.01690.7217±0.01690.7217\pm0.01690.6531±0.02440.6531\pm0.02440.7921±0.02260.7921\pm0.02260.6241±0.02750.6241\pm0.02750.8113±0.01660.8113\pm0.01660.7980±0.02090.7980\pm0.02090.7980±0.02090.7980\pm0.02090.05630.0563 (+)
TNK3BALD[60]20%0.7491±0.02530.7491\pm0.02530.7405±0.03060.7405\pm0.03060.7174±0.03060.7174\pm0.03060.7686±0.03600.7686\pm0.03600.6270±0.03650.6270\pm0.03650.8313±0.02750.8313\pm0.02750.8290±0.02760.8290\pm0.02760.8290±0.02760.8290\pm0.02760.08160.0816 (+)
TNK3EvoDiff (Ours)20%0.7903±0.01520.7903\pm0.01520.7793±0.01430.7793\pm0.01430.7438±0.01430.7438\pm0.01430.8149±0.01840.8149\pm0.01840.6802±0.02430.6802\pm0.02430.8575±0.01810.8575\pm0.01810.8544±0.01200.8544\pm0.01200.8544±0.01200.8544\pm0.01200.05630.0563 (+)
TNK3Random30%0.7920±0.01180.7920\pm0.01180.7646±0.01890.7646\pm0.01890.6760±0.01890.6760\pm0.01890.8533±0.00880.8533\pm0.00880.7088±0.00590.7088\pm0.00590.8329±0.01590.8329\pm0.01590.8456±0.01430.8456\pm0.01430.8456±0.01430.8456\pm0.01430.02160.0216 (−)
TNK3VAAL[57]30%0.7904±0.01180.7904\pm0.01180.7600±0.02230.7600\pm0.02230.7273±0.02020.7273\pm0.02020.7939±0.01130.7939\pm0.01130.6510±0.01450.6510\pm0.01450.8465±0.00810.8465\pm0.00810.8335±0.01450.8335\pm0.01450.8335±0.01450.8335\pm0.01450.01620.0162 (+)
TNK3Core-Set[58]30%0.7903±0.01850.7903\pm0.01850.7647±0.01560.7647\pm0.01560.7372±0.02650.7372\pm0.02650.8183±0.01330.8183\pm0.01330.6834±0.01130.6834\pm0.01130.8391±0.01400.8391\pm0.01400.8610±0.02290.8610\pm0.02290.8610±0.02290.8610\pm0.02290.02640.0264 (+)
TNK3Max-Entropy[59]30%0.7903±0.02490.7903\pm0.02490.7793±0.01520.7793\pm0.01520.7372±0.02650.7372\pm0.02650.8184±0.01840.8184\pm0.01840.6802±0.02430.6802\pm0.02430.8551±0.01380.8551\pm0.01380.8590±0.01380.8590\pm0.01380.8590±0.01380.8590\pm0.01380.05760.0576 (+)
TNK3BALD[60]30%0.7903±0.01520.7903\pm0.01520.7793±0.01520.7793\pm0.01520.7438±0.01430.7438\pm0.01430.8149±0.01840.8149\pm0.01840.6802±0.02430.6802\pm0.02430.8575±0.01810.8575\pm0.01810.8544±0.01200.8544\pm0.01200.8544±0.01200.8544\pm0.01200.05760.0576 (+)
TNK3EvoDiff (Ours)30%0.7617±0.01250.7617\pm0.01250.7602±0.01100.7602\pm0.01100.7554±0.04070.7554\pm0.04070.7651±0.00570.7651\pm0.00570.6292±0.01100.6292\pm0.01100.8559±0.00280.8559\pm0.00280.6863±0.02280.6863\pm0.02280.8616±0.01290.8616\pm0.01290.01690.0169 (+)

Table S2 (PDF p. 42). Comparison of USFM classification metrics between EvoDiff and other AL methods on the test sets. Values are reported as mean ±\pm s.d. over five folds. Statistical differences between each active learning strategy and Random sampling were assessed using two-sided paired tt-tests based on fold-wise AUC. The reported qq-values are the Benjamini–Hochberg-calibrated pp-values across the five strategy-versus-Random comparisons within each dataset and annotation budget; the appended (+) or (−) indicates the sign of mean_diff(method-Random) for AUC. (Bold represents the best result, and Underline represents the second best result).

Note. Higher is better for every metric. Bold indicates the best result and underline indicates the second-best result within each dataset and annotation budget. The qq-value is the Benjamini–Hochberg-calibrated pp-value for the two-sided paired tt-test against Random at the same dataset and annotation budget, and (+) or (−) indicates the sign of the mean difference relative to Random.

Table S3. Cross-domain generalization of USFM across external validation datasets. USFM was fine-tuned exclusively on source-domain data using training subsets selected by either random sampling or EvoDiff at annotation budgets of 10%, 20%, and 30%, and was subsequently evaluated without target-domain adaptation on seven unseen target-domain datasets, including four public and three private datasets. Performance is reported as balanced accuracy (bAcc). Statistical significance was assessed using two-sided (Random versus EvoDiff) paired tt-tests. The Generalization Gain column reports each row’s largest relative gain across the three budgets and is therefore a post-hoc selected maximum, it is not corrected for this selection. For each annotation budget, the higher result between random sampling and EvoDiff is shown in bold.

Source Domain → Target DomainCross-domain Generalization Performance (bAcc)Generalization Gain{sup}a
10% Annotation20% Annotation30% Annotation
RandomEvoDiff (Ours)pp-valueRandomEvoDiff (Ours)pp-valueRandomEvoDiff (Ours)pp-value
Target Domain: ALN-Ultra{sup}48 (n=257n=257; 121 benign, 136 malignant)
BUS-BRA → ALN-Ultra0.58580.60130.05660.59250.63760.01560.61740.63850.2422+7.6%
BUSI → ALN-Ultra0.59400.62880.04330.58810.63940.01560.61970.64530.1563+8.7%
UDIAT → ALN-Ultra0.53250.58260.04200.55950.57130.17970.55170.59820.1875+9.4%
Target Domain: BUS_UC{sup}74 (n=811n=811; 358 benign, 453 malignant)
BUS-BRA → BUS_UC0.58850.58940.07790.58950.61130.25000.63580.63730.2969+3.7%
BUSI → BUS_UC0.56700.57460.08250.57290.58950.06250.58810.60170.4922+2.9%
UDIAT → BUS_UC0.50430.50190.09060.50570.57000.00780.51720.57290.0656+12.7%
Target Domain: QAMEBI{sup}49 (n=232n=232; 109 benign, 123 malignant)
BUS-BRA → QAMEBI0.63250.64650.06640.63590.73140.21090.74100.75520.3750+15.0%
BUSI → QAMEBI0.59170.63420.08200.61240.63570.38280.57140.59720.6406+7.2%
UDIAT → QAMEBI0.57460.58570.05000.57570.67280.01560.58630.68410.2813+16.9%
Target Domain: TN5000{sup}54 (n=5,000n=5,000; 1,426 benign, 3,574 malignant)
TN3K → TN50000.61920.63020.05830.63670.63960.05940.64280.67370.0156+4.8%
Target Domain: PrivAppendix (n=45n=45; 21 normal, 24 appendicitis)
Appendix{sup}44 → PrivAppendix0.42860.53570.03670.42860.49400.03200.53570.57140.0156+25.0%
Target Domain: PrivBreast (n=100n=100; 59 benign, 41 malignant)
BUS-BRA{sup}46 → PrivBreast0.54770.61530.04100.57590.68190.00780.73440.76790.0781+18.4%
BUSI{sup}45 → PrivBreast0.52500.55890.05210.54730.57590.07270.57380.60770.0625+6.5%
UDIAT{sup}47 → PrivBreast0.60790.60770.02130.54880.55890.29690.59510.57590.0156+1.8%
Target Domain: PrivLiver (n=69n=69; 24 normal, 45 fatty-liver)
Fatty-Liver{sup}51 → PrivLiver0.56110.66390.00780.65140.66390.21880.65140.66110.0156+18.3%

Table S3. Cross-domain generalization of USFM across external validation datasets. USFM was fine-tuned exclusively on source-domain data using training subsets selected by either random sampling or EvoDiff at annotation budgets of 10%, 20%, and 30%, and was subsequently evaluated without target-domain adaptation on seven unseen target-domain datasets, including four public and three private datasets. Performance is reported as balanced accuracy (bAcc). Statistical significance was assessed using two-sided (Random versus EvoDiff) paired tt-tests. The Generalization Gain column reports each row’s largest relative gain across the three budgets and is therefore a post-hoc selected maximum, it is not corrected for this selection. For each annotation budget, the higher result between random sampling and EvoDiff is shown in bold.

BudgetAppendixBreastCarotidLiverOvaryThyroid
SeniorJuniorSeniorJuniorSeniorJuniorSeniorJuniorSeniorJuniorSeniorJunior
-/w EvoDiff-/w EvoDiff-/w EvoDiff-/w EvoDiff-/w EvoDiff-/w EvoDiff-/w EvoDiff-/w EvoDiff-/w EvoDiff-/w EvoDiff-/w EvoDiff-/w EvoDiff
5%17.1/2.829.6/4.819.3/10.326.8/14.323.3/5.228.7/6.54.1/1.66.5/2.531.6/17.137.3/20.353.0/17.161.5/19.8
10%19.1/5.532.9/9.649.0/20.668.2/28.635.6/10.543.9/12.96.7/3.110.5/4.978.8/34.493.0/40.665.8/34.276.3/39.6
20%20.1/11.134.7/19.273.8/41.2102.6/57.361.0/20.975.4/25.814.0/6.221.9/9.899.6/68.9117.6/81.2177.2/68.4205.3/79.2
30%22.0/16.738.0/28.886.6/61.8120.4/85.975.4/31.493.1/38.724.1/9.338.0/14.7138.1/103.3163.1/121.9225.0/102.5260.8/118.8
40%29.2/22.250.4/38.4131.5/82.4182.8/114.684.9/41.8104.8/51.626.2/12.541.2/19.6228.4/137.6273.1/164.5262.4/136.6304.1/158.4
50%52.9/27.891.3/48.0158.7/103.0220.6/143.296.0/52.3118.5/64.530.5/15.548.0/24.5282.5/172.0337.9/205.7301.2/170.8349.2/198.0

Table S4. Model-based estimates of Human Annotation Effort (HAE, minutes) required by senior and junior radiologists to achieve matched downstream ResNet-50 performance using random sampling versus EvoDiff-selected annotations. Estimates were recomputed using the expanded six-reader panel. “-/w EvoDiff” denotes random sampling/EvoDiff selection, respectively.

DatasetHyperparameter Setting
Warmup EpochWarmup LRBase LRMinimum LRWeight DecayLayer Decay
Appendix[44]21.25e-31.25e-31.25e-40.0011.0
BUSI[45]22.5e-41.25e-22.5e-70.050.65
BUS-BRA[46]22.5e-41.25e-22.5e-70.050.65
UDIAT[47]22.5e-41.25e-22.5e-70.050.65
Fatty-Liver[51]11.25e-31.25e-31.25e-40.0011.0
MMOTU[52]21.25e-31e-31.25e-40.0011.0
TN3K[53]21.25e-31e-31.25e-40.0011.0

Table S5. Hyperparameter settings for different datasets on USFM fine-tuning.

ReaderExperience (years)OrganReading time (min)Observed count (nn) AnnotatedObserved count (nn) CorrectAccuracyNormalized count (nn/30 min) AnnotatedNormalized count (nn/30 min) CorrectMean time (s) Per imageMean time (s) Per correct
R1 (Senior)7Appendix16.83802390.6296804282.654.21
R1 (Senior)7Breast21.14003260.8155694643.163.88
R1 (Senior)7Carotid12.54003200.8009597671.882.35
R1 (Senior)7Liver20.04003690.9236005543.003.25
R1 (Senior)7Ovary21.25001470.2947082082.548.65
R1 (Senior)7Thyroid22.75003900.7806615162.723.49
R2 (Senior)13Appendix30.02351430.6092351437.6612.59
R2 (Senior)13Breast30.03002410.8033002416.007.47
R2 (Senior)13Carotid30.03352610.7793352615.376.90
R2 (Senior)13Liver23.04003180.7955224153.454.34
R2 (Senior)13Ovary30.0329760.231329765.4723.68
R2 (Senior)13Thyroid30.02531990.7872531997.119.05
R3 (Senior)8Appendix30.03001890.6303001896.009.52
R3 (Senior)8Breast30.03002170.7233002176.008.29
R3 (Senior)8Carotid22.54003030.7585334043.384.46
R3 (Senior)8Liver30.04003510.8784003514.505.13
R3 (Senior)8Ovary30.03231790.5543231795.5710.06
R3 (Senior)8Thyroid30.04413540.8034413544.085.08
R4 (Junior)2Appendix30.02951750.5932951756.1010.29
R4 (Junior)2Breast30.03622480.6853622484.977.26
R4 (Junior)2Carotid13.04003080.7709237111.952.53
R4 (Junior)2Liver12.34003000.7509777331.842.46
R4 (Junior)2Ovary25.05001710.3426002053.008.77
R4 (Junior)2Thyroid25.05003940.7886004733.003.81
R5 (Junior)3Appendix25.03802370.6244562843.956.33
R5 (Junior)3Breast27.04003070.7684443414.055.28
R5 (Junior)3Carotid23.04003080.7705224023.454.48
R5 (Junior)3Liver30.03031860.6143031865.949.68
R5 (Junior)3Ovary29.05002180.4365172263.487.98
R5 (Junior)3Thyroid25.05003920.7846004703.003.83
R6 (Junior)3Appendix30.0119640.5381196415.1328.13
R6 (Junior)3Breast30.01441190.82614411912.5015.13
R6 (Junior)3Carotid30.02351830.7792351837.669.84
R6 (Junior)3Liver30.02662330.8762662336.777.73
R6 (Junior)3Ovary30.0142530.3731425312.6833.96
R6 (Junior)3Thyroid30.01851410.7621851419.7312.77

Table S6. Per-reader annotation performance across the six organ tasks. Six board-certified radiologists (R1–R3 senior, R4–R6 junior) annotated the same image sets under the same protocol. Accuracy is the proportion of correctly annotated images. Normalized counts project each reader’s observed rate to a 30-minute period. “Per image” and “Per correct” denote the mean seconds per annotated image and the mean seconds per correctly annotated image, the latter being the total annotation time in seconds divided by the number of correctly annotated images. Note. Readers are identified as R1–R3 (Senior) and R4–R6 (Junior), with clinical experience reported separately. Accuracy is the proportion of correctly annotated images. Normalized counts are the projected numbers of annotated and correctly annotated images within a 30-minute reading period, calculated from each reader’s observed annotation rate. “Per image” and “Per correct” denote the mean seconds per annotated image and per correctly annotated image, respectively.

OrganReal-image references1s3s5s7s9Pooled EvoDiff syntheticPaired tt-test vs reference
Appendix0.624±0.0630.624 \pm0.0630.7410.7860.6880.7030.7440.733±0.0790.733 \pm0.0790.176
Breast0.929±0.0140.929 \pm0.0140.7900.8670.9010.9140.7740.847±0.0180.847 \pm0.018<0.001< 0.001
Carotid0.657±0.0240.657 \pm0.0240.7790.7310.7060.7730.7630.751±0.0450.751 \pm0.0450.018
Liver0.970±0.0530.970 \pm0.0530.9450.9700.9630.9720.9810.966±0.0230.966 \pm0.0230.803
Ovary0.932±0.0310.932 \pm0.0310.8490.7660.8520.8980.8590.844±0.0300.844 \pm0.0300.034
Thyroid0.853±0.0120.853 \pm0.0120.8210.7620.7810.7260.8470.789±0.0160.789 \pm0.0160.007

Table S7. Diagnostic-model recognition of synthetic probes. AUC (mean ±\pm s.d. over five folds) from classifiers trained exclusively on real images, reported across five guidance strengths and for the pooled synthetic images. s1, s3, s5, s7, and s9 denote the guidance strengths used in the EvoDiff generation process. The real-image reference is the AUC of the same models on real images.

ReaderExperienceAppendixBreastCarotid
Synthetic DataReal DataAccSynthetic DataReal DataAccSynthetic DataReal DataAcc
Synthetic (Correct)Real (Wrong)Real (Correct)Synthetic (Wrong)Synthetic (Correct)Real (Wrong)Real (Correct)Synthetic (Wrong)Synthetic (Correct)Real (Wrong)Real (Correct)Synthetic (Wrong)
Senior 19 years7131280.4758121280.50015510100.625
Senior 28 years21810100.3002182000.5504161730.525
Junior 13 years8129110.4258122000.7007131280.475
Junior 24 years7131190.4505151370.4504161280.400
Average0.4130.5500.506
ReaderExperienceLiverOvaryThyroid
Synthetic DataReal DataAccSynthetic DataReal DataAccSynthetic DataReal DataAcc
Synthetic (Correct)Real (Wrong)Real (Correct)Synthetic (Wrong)Synthetic (Correct)Real (Wrong)Real (Correct)Synthetic (Wrong)Synthetic (Correct)Real (Wrong)Real (Correct)Synthetic (Wrong)
Senior 19 years8128120.4005156140.2755151550.500
Senior 28 years61410100.4000201640.4001191910.500
Junior 13 years7133170.2507131640.5758121280.500
Junior 24 years4166140.2501191640.4254161370.425
Average0.3250.4190.481

Table S8. Table S8. Blinded real-versus-synthetic discrimination. Four radiologists (two senior and two junior) each assessed 20 real and 20 synthetic images per organ. Synthetic images were evenly sampled from five guidance strengths. Accuracy denotes the proportion of correct image-origin judgments.

Table S9. Blinded reader diagnostic scores by organ and image category. Pooled 1–5 scores from four readers are reported for real and synthetic non-diseased and diseased images, where 1 denotes a confidently non-diseased appearance and 5 a confidently diseased appearance. For each category, nn, mean ±\pm s.d., median, and the proportion of scores ≥4\ge4 are shown; bold marks the category rated most disease-like within each organ.

OrganCategorynn12345Mean ±\pm s.d.MedianScore ≥4\ge4
AppendixReal normal4016115262.28 ±\pm 1.43220%
AppendixSynthetic normal401786542.28 ±\pm 1.40222%
AppendixReal diseased4041414173.98 ±\pm 1.25478%
AppendixSynthetic diseased4098210113.15 ±\pm 1.58452%
BreastReal normal4017132712.05 ±\pm 1.20220%
BreastSynthetic normal402555411.78 ±\pm 1.17112%
BreastReal diseased40798793.05 ±\pm 1.43340%
BreastSynthetic diseased40121016113.85 ±\pm 0.98468%
CarotidReal normal403072101.35 ±\pm 0.7012%
CarotidSynthetic normal402794001.42 ±\pm 0.6810%
CarotidReal diseased401757922.35 ±\pm 1.37228%
CarotidSynthetic diseased4010155822.42 ±\pm 1.22225%
LiverReal normal4025111211.58 ±\pm 0.9618%
LiverSynthetic normal4014135352.30 ±\pm 1.36220%
LiverReal diseased400325304.55 ±\pm 0.90588%
LiverSynthetic diseased400027314.72 ±\pm 0.55595%
OvaryReal normal4015714222.22 ±\pm 1.17210%
OvarySynthetic normal401659552.45 ±\pm 1.45225%
OvaryReal diseased4087101052.92 ±\pm 1.33338%
OvarySynthetic diseased401378842.58 ±\pm 1.392.530%
ThyroidReal normal402267411.90 ±\pm 1.17112%
ThyroidSynthetic normal402184701.92 ±\pm 1.16118%
ThyroidReal diseased407951093.12 ±\pm 1.45348%
ThyroidSynthetic diseased4010481083.05 ±\pm 1.48345%

Table S9.

Figure S2. Statistical comparison of sample-selection strategies against Random for balanced accuracy across eight datasets and three cumulative annotation budgets. a) Results for the ResNet-50 downstream model. b) Results for the ultrasound foundation model (USFM). In each panel, the three heat maps correspond to cumulative annotation budgets of 10%, 20%, and 30%, rows list the five sample-selection strategies compared against Random (VAAL, Core-Set, Max-Entropy, BALD, and EvoDiff), and columns list the eight internal datasets. In each cell, the upper value is the unadjusted pp-value from a two-sided paired tt-test on the five matched cross-validation folds, and the lower value is the mean difference in balanced accuracy (bAcc) relative to Random, with positive values indicating better performance than Random. The cell colour encodes −log⁡10(p)-\log_{10}(p) from light yellow for values near 1 to dark red for values of 3.5 and above, as shown by the colour bar. A black cell border marks an unadjusted p<0.05p < 0.05, and an asterisk marks significance after Benjamini–Hochberg correction, with q<0.05q < 0.05 within the family of the five strategy-versus-Random comparisons that share a dataset and annotation budget. The testing procedure is identical for both downstream models.

Figure S3. Figure S3. Statistical comparison of sample-selection strategies against Random for sensitivity across eight datasets and three cumulative annotation budgets. a) Results for the ResNet-50 downstream model. b) Results for the ultrasound foundation model (USFM). In each panel, the three heat maps correspond to cumulative annotation budgets of 10%, 20%, and 30%, rows list the five sample-selection strategies compared against Random (VAAL, Core-Set, Max-Entropy, BALD, and EvoDiff), and columns list the eight internal datasets. In each cell, the upper value is the unadjusted pp-value from a two-sided paired tt-test on the five matched cross-validation folds, and the lower value is the mean difference in sensitivity relative to Random, with positive values indicating better performance than Random. The cell colour encodes −log⁡10(p)-\log_{10}(p) from light yellow for values near 1 to dark red for values of 3.5 and above, as shown by the colour bar. A black cell border marks an unadjusted p<0.05p < 0.05, and an asterisk marks significance after Benjamini–Hochberg correction, with q<0.05q < 0.05 within the family of the five strategy-versus-Random comparisons that share a dataset and annotation budget. The testing procedure is identical for both downstream models.

Figure S4. Statistical comparison of sample-selection strategies against Random for specificity across eight datasets and three cumulative annotation budgets. a) Results for the ResNet-50 downstream model. b) Results for the ultrasound foundation model (USFM). In each panel, the three heat maps correspond to cumulative annotation budgets of 10%, 20%, and 30%, rows list the five sample-selection strategies compared against Random (VAAL, Core-Set, Max-Entropy, BALD, and EvoDiff), and columns list the eight internal datasets. In each cell, the upper value is the unadjusted pp-value from a two-sided paired tt-test on the five matched cross-validation folds, and the lower value is the mean difference in specificity relative to Random, with positive values indicating better performance than Random. The cell colour encodes −log⁡10(p)-\log_{10}(p) from light yellow for values near 1 to dark red for values of 3.5 and above, as shown by the colour bar. A black cell border marks an unadjusted p<0.05p < 0.05, and an asterisk marks significance after Benjamini–Hochberg correction, with q<0.05q < 0.05 within the family of the five strategy-versus-Random comparisons that share a dataset and annotation budget. The testing procedure is identical for both downstream models.

Figure S5. Statistical comparison of sample-selection strategies against Random for positive predictive value (PPV) across eight datasets and three cumulative annotation budgets. a) Results for the ResNet-50 downstream model. b) Results for the ultrasound foundation model (USFM). In each panel, the three heat maps correspond to cumulative annotation budgets of 10%, 20%, and 30%, rows list the five sample-selection strategies compared against Random (VAAL, Core-Set, Max-Entropy, BALD, and EvoDiff), and columns list the eight internal datasets. In each cell, the upper value is the unadjusted pp-value from a two-sided paired tt-test on the five matched cross-validation folds, and the lower value is the mean difference in PPV relative to Random, with positive values indicating better performance than Random. The cell colour encodes −log⁡10(p)-\log_{10}(p) from light yellow for values near 1 to dark red for values of 3.5 and above, as shown by the colour bar. A black cell border marks an unadjusted p<0.05p < 0.05, and an asterisk marks significance after Benjamini–Hochberg correction, with q<0.05q < 0.05 within the family of the five strategy-versus-Random comparisons that share a dataset and annotation budget. The testing procedure is identical for both downstream models.

Figure S6. Statistical comparison of sample-selection strategies against Random for negative predictive value (NPV) across eight datasets and three cumulative annotation budgets. a) Results for the ResNet-50 downstream model. b) Results for the ultrasound foundation model (USFM). In each panel, the three heat maps correspond to cumulative annotation budgets of 10%, 20%, and 30%, rows list the five sample-selection strategies compared against Random (VAAL, Core-Set, Max-Entropy, BALD, and EvoDiff), and columns list the eight internal datasets. In each cell, the upper value is the unadjusted pp-value from a two-sided paired tt-test on the five matched cross-validation folds, and the lower value is the mean difference in NPV relative to Random, with positive values indicating better performance than Random. The cell colour encodes −log⁡10(p)-\log_{10}(p) from light yellow for values near 1 to dark red for values of 3.5 and above, as shown by the colour bar. A black cell border marks an unadjusted p<0.05p < 0.05, and an asterisk marks significance after Benjamini–Hochberg correction, with q<0.05q < 0.05 within the family of the five strategy-versus-Random comparisons that share a dataset and annotation budget. The testing procedure is identical for both downstream models.

Figure S7. Statistical comparison of sample-selection strategies against Random for F1-score across eight datasets and three cumulative annotation budgets. a) Results for the ResNet-50 downstream model. b) Results for the ultrasound foundation model (USFM). In each panel, the three heat maps correspond to cumulative annotation budgets of 10%, 20%, and 30%, rows list the five sample-selection strategies compared against Random (VAAL, Core-Set, Max-Entropy, BALD, and EvoDiff), and columns list the eight internal datasets. In each cell, the upper value is the unadjusted pp-value from a two-sided paired tt-test on the five matched cross-validation folds, and the lower value is the mean difference in F1-score relative to Random, with positive values indicating better performance than Random. The cell colour encodes −log⁡10(p)-\log_{10}(p) from light yellow for values near 1 to dark red for values of 3.5 and above, as shown by the colour bar. A black cell border marks an unadjusted p<0.05p < 0.05, and an asterisk marks significance after Benjamini–Hochberg correction, with q<0.05q < 0.05 within the family of the five strategy-versus-Random comparisons that share a dataset and annotation budget. The testing procedure is identical for both downstream models.

Figure S8. Statistical comparison of sample-selection strategies against Random for area under the receiver operating characteristic curve (AUC) across eight datasets and three cumulative annotation budgets. a) Results for the ResNet-50 downstream model. b) Results for the ultrasound foundation model (USFM). In each panel, the three heat maps correspond to cumulative annotation budgets of 10%, 20%, and 30%, rows list the five sample-selection strategies compared against Random (VAAL, Core-Set, Max-Entropy, BALD, and EvoDiff), and columns list the eight internal datasets. In each cell, the upper value is the unadjusted pp-value from a two-sided paired tt-test on the five matched cross-validation folds, and the lower value is the mean difference in AUC relative to Random, with positive values indicating better performance than Random. The cell colour encodes −log⁡10(p)-\log_{10}(p) from light yellow for values near 1 to dark red for values of 3.5 and above, as shown by the colour bar. A black cell border marks an unadjusted p<0.05p < 0.05, and an asterisk marks significance after Benjamini–Hochberg correction, with q<0.05q < 0.05 within the family of the five strategy-versus-Random comparisons that share a dataset and annotation budget. The testing procedure is identical for both downstream models.

Reader scores for non-diseased images

Figure S9. Reader scores for non-diseased images. Four blinded readers rated 10 real and 10 synthetic images per organ on a 1–5 scale (1, confidently non-diseased; 5, confidently diseased). Panels show the appendix, breast, carotid, liver, ovary, and thyroid tasks, respectively. Bars show empirical probabilities, solid curves Gaussian kernel-density estimates, and dashed lines group means; legends report mean ± s.d.

Reader scores for diseased images

Figure S10. Reader scores for diseased images. The reader study used the same settings as Supplementary Figure S9.

References

  1. [1]Muhanna, A. et al. Exploring the multifaceted applications of ultrasound imaging in medical diagnostics. Int. J. 10 (2022).DOI
  2. [2]Yoshida, T. et al. Diagnostic accuracy of point-of-care ultrasound for shock: a systematic review and meta-analysis. Critical care 27, 1–11 (2023).
  3. [3]Thies, M. & Oelze, M. L. Combined therapy planning, real-time monitoring, and low intensity focused ultrasound treatment using a diagnostic imaging array. IEEE transactions on medical imaging 41, 1410–1419 (2022).
  4. [4]Yang, X. et al. Searching collaborative agents for multi-plane localization in 3d ultrasound. Med. Image Analysis 72, 102119 (2021).
  5. [5]Huang, Y. et al. Online reflective learning for robust medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 652–662 (Springer, 2022).arxiv.org/abs/2207.00476
  6. [6]Huang, Y. et al. Fourier test-time adaptation with multi-level consistency for robust classification. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 221–231 (Springer, 2023).arxiv.org/abs/2306.02544
  7. [7]Duan, Y. et al. Fetalflex: Anatomy-guided diffusion model for flexible control on fetal ultrasound image synthesis. Med. Image Analysis 103725 (2025).DOI
  8. [8]Akbulut, S., Kucukakali, Z. & Colak, C. Artificial intelligence in acute appendicitis: A comprehensive review of machine learning and deep learning applications. World J. Gastroenterol. 31 (2025).DOI
  9. [9]Huang, Y. et al. Flip learning: Weakly supervised erase to segment nodules in breast ultrasound. Med. Image Analysis 102, 103552 (2025).
  10. [10]Duan, Y. et al. 3mt-net: A multi-modal multi-task model for breast cancer and pathological subtype classification based on a multicenter study. IEEE J. Biomed. Heal. Informatics 29, 4680–4691 (2025).DOI
  11. [11]Xiang, H. et al. Development and validation of an interpretable model integrating multimodal information for improving ovarian cancer diagnosis. Nat. Commun. 15, 2681 (2024).DOI
  12. [12]Lu, R.-F. et al. Ai enhanced diagnostic accuracy and workload reduction in hepatocellular carcinoma screening. NPJ Digit. Medicine 8, 500 (2025).DOI
  13. [13]Su, K. et al. A fully autonomous robotic ultrasound system for thyroid scanning. Nat. Commun. 15, 4004 (2024).DOI
  14. [14]Huang, Y.-J., Chen, C.-h. & Yang, H.-C. Ai-enhanced integration of genetic and medical imaging data for risk assessment of type 2 diabetes. Nat. Commun. 15, 4230 (2024).
  15. [15]Won, D. et al. Sound the alarm: the sonographer shortage is echoing across healthcare. J. Ultrasound Medicine 43, 1289–1301 (2024).DOI
  16. [16]Cohen, C., Childs, J. & Maranna, S. Behind closed doors: are sonographers coping? a literature review of sonographer burnout. Sonography 8, 3–11 (2021).DOI
  17. [17]Wu, C. et al. Towards generalist foundation model for radiology by leveraging web-scale 2d&3d medical data. Nat. Commun. 16, 7866 (2025).
  18. [18]Pai, S. et al. Foundation model for cancer imaging biomarkers. Nat. machine intelligence 6, 354–367 (2024).
  19. [19]Lutnick, B. et al. An integrated iterative annotation technique for easing neural network training in medical image analysis. Nat. machine intelligence 1, 112–119 (2019).DOI
  20. [20]Zhang, S. et al. A generalist foundation model and database for open-world medical image segmentation. Nat. Biomed. Eng. 1–16 (2025).DOI
  21. [21]Herath, H., Herath, H., Madusanka, N. & Lee, B.-I. A systematic review of medical image quality assessment. J. Imaging 11, 100 (2025).
  22. [22]Ash, J. T., Zhang, C., Krishnamurthy, A., Langford, J. & Agarwal, A. Deep batch active learning by diverse, uncertain gradient lower bounds. arXiv preprint arXiv:1906.03671 (2019).arxiv.org/abs/1906.03671
  23. [23]Mahapatra, D. et al. Alfredo: Active learning with feature disentanglement and domain adaptation for medical image classification. Med. image analysis 97, 103261 (2024).DOI
  24. [24]Duan, Y. et al. ADaptation: Reconstruction-based Unsupervised Active Learning for Breast Ultrasound Diagnosis . In proceedings of Medical Image Computing and Computer Assisted Intervention – MICCAI 2025, vol. LNCS 15975 (Springer Nature Switzerland, 2025).arxiv.org/abs/2507.00474
  25. [25]Christiansen, F. et al. International multicenter validation of ai-driven ultrasound detection of ovarian cancer. Nat. medicine 31, 189–196 (2025).DOI
  26. [26]Wu, D. et al. Improving the generalizability and performance of an ultrasound deep learning model using limited multicenter data for lung sliding artifact identification. Diagnostics 14, 1081 (2024).DOI
  27. [27]Ren, Z., Ren, Z., Zhang, Z., Buonassisi, T. & Li, J. Autonomous experiments using active learning and ai. Nat. Rev. Mater. 8, 563–564 (2023).
  28. [28]Li, D. et al. A survey on deep active learning: Recent advances and new frontiers. IEEE Transactions on Neural Networks Learn. Syst. 36, 5879–5899 (2024).
  29. [29]Karamcheti, S., Krishna, R., Fei-Fei, L. & Manning, C. D. Mind your outliers! investigating the negative impact of outliers on active learning for visual question answering. arXiv preprint arXiv:2107.02331 (2021).arxiv.org/abs/2107.02331
  30. [30]Jin, C., Guo, Z., Lin, Y., Luo, L. & Chen, H. Label-efficient deep learning in medical image analysis: Challenges and future directions. arXiv preprint arXiv:2303.12484 (2023).arxiv.org/abs/2303.12484
  31. [31]Linmans, J., Elfwing, S., van der Laak, J. & Litjens, G. Predictive uncertainty estimation for out-of-distribution detection in digital pathology. Med. Image Analysis 83, 102655 (2023).
  32. [32]Yan, X. et al. A clustering-based active learning method to query informative and representative samples. Appl. Intell. 52, 13250–13267 (2022).DOI
  33. [33]Mittal, S., Tatarchenko, M., Çiçek, Ö. & Brox, T. Parting with illusions about deep active learning. arXiv preprint arXiv:1912.05361 (2019).arxiv.org/abs/1912.05361
  34. [34]Lüth, C. T., Bungert, T. J., Klein, L. & Jaeger, P. F. Navigating the pitfalls of active learning evaluation: A systematic framework for meaningful performance assessment. In Advances in Neural Information Processing Systems (2023).arxiv.org/abs/2301.10625
  35. [35]Brangbour, E., Bruneau, P., Tamisier, T. & Marchand-Maillet, S. Cold start active learning strategies in the context of imbalanced classification. arXiv preprint arXiv:2201.10227 (2022).arxiv.org/abs/2201.10227
  36. [36]Kazerouni, A. et al. Diffusion models in medical imaging: A comprehensive survey. Med. Image Analysis 88, 102846 (2023).DOI
  37. [37]Yang, L. et al. Diffusion models: A comprehensive survey of methods and applications. ACM computing surveys 56, 1–39 (2023).
  38. [38]Zhu, J.-J. & Bento, J. Generative adversarial active learning (2017). 1702.07956.
  39. [39]Tran, T., Do, T.-T., Reid, I. & Carneiro, G. Bayesian generative active deep learning. In Chaudhuri, K. & Salakhutdinov, R. (eds.) Proceedings of the 36th International Conference on Machine Learning, vol. 97 of Proceedings of Machine Learning Research, 6295–6304 (PMLR, 2019).
  40. [40]Awais, M. et al. Foundation models defining a new era in vision: a survey and outlook. IEEE Transactions on Pattern Analysis Mach. Intell. (2025).DOI
  41. [41]Huang, J. et al. Foundation models and intelligent decision-making: Progress, challenges, and perspectives. The Innov. (2025).DOI
  42. [42]He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778 (2016).
  43. [43]Jiao, J. et al. Usfm: A universal ultrasound foundation model generalized to tasks and organs towards label efficient image analysis. Med. image analysis 96, 103202 (2024).
  44. [44]Marcinkevics, R. et al. Regensburg pediatric appendicitis dataset. (No Title) (2023).
  45. [45]Al-Dhabyani, W., Gomaa, M., Khaled, H. & Fahmy, A. Dataset of breast ultrasound images. Data brief 28, 104863 (2020).
  46. [46]Gómez-Flores, W., Gregorio-Calas, M. J. & Coelho de Albuquerque Pereira, W. Bus-bra: a breast ultrasound dataset for assessing computer-aided diagnosis systems. Med. Phys. 51, 3110–3123 (2024).
  47. [47]Yap, M. H. et al. Automated breast ultrasound lesions detection using convolutional neural networks. IEEE journal biomedical health informatics 22, 1218–1226 (2017).DOI
  48. [48]Hospital, F. U. A. P. Aln-ultra, 10.5281/zenodo.15003119 (2025).
  49. [49]Abbasian Ardakani, A., Mohammadi, A., Mirza-Aghazadeh-Attari, M. & Acharya, U. R. An open-access breast lesion ultrasound image database: Applicable in artificial intelligence studies. Comput. Biol. Medicine 152, 106438, https://doi.org/10.1016/j.compbiomed.2022.106438 (2023).
  50. [50]Meiburger, K. et al. Dataset for “carotid ultrasound boundary study (cubs): an open multi-center analysis of computerized intima-media thickness measurement systems and their clinical impact”. Mendeley Data 1 (2021).
  51. [51]Byra, M. et al. Transfer learning with deep convolutional neural network for liver steatosis assessment in ultrasound images. Int. journal computer assisted radiology surgery 13, 1895–1903 (2018).DOI
  52. [52]Zhao, Q. et al. Mmotu: A multi-modality ovarian tumor ultrasound image dataset for unsupervised cross-domain semantic segmentation (2023). 2207.06799.arxiv.org/abs/2207.06799
  53. [53]Gong, H. et al. Thyroid region prior guided attention for ultrasound segmentation of thyroid nodules. Comput. biology medicine 155, 106389 (2023).
  54. [54]Zhang, H., Liu, Q., Han, X., Niu, L. & Sun, W. Tn5000: An ultrasound image dataset for thyroid nodule detection and classification. Sci. Data 12, 1437 (2025).
  55. [55]Zhong, L. et al. Unisal: Unified semi-supervised active learning for histopathological image classification. Med. Image Analysis 102, 103542 (2025).DOI
  56. [56]Ma, S., Du, H., Curran, K. M., Lawlor, A. & Dong, R. Adaptive curriculum query strategy for active learning in medical image classification. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 48–57 (Springer, 2024).DOI
  57. [57]Sinha, S., Ebrahimi, S. & Darrell, T. Variational adversarial active learning. In Proceedings of the IEEE/CVF international conference on computer vision, 5972–5981 (2019).
  58. [58]Sener, O. & Savarese, S. Active learning for convolutional neural networks: A core-set approach. arXiv preprint arXiv:1708.00489 (2017).arxiv.org/abs/1708.00489
  59. [59]Yang, Y. & Xu, Z. Rethinking the value of labels for improving class-imbalanced learning. Adv. neural information processing systems 33, 19290–19301 (2020).arxiv.org/abs/2006.07529
  60. [60]Gal, Y., Islam, R. & Ghahramani, Z. Deep bayesian active learning with image data. In International conference on machine learning, 1183–1192 (PMLR, 2017).
  61. [61]Moor, M. et al. Foundation models for generalist medical artificial intelligence. Nature 616, 259–265 (2023).DOI
  62. [62]Zhang, K. et al. A generalist vision–language foundation model for diverse biomedical tasks. Nat. Medicine 30, 3129–3141 (2024).DOI
  63. [63]Lu, M. Y. et al. A visual-language foundation model for computational pathology. Nat. medicine 30, 863–874 (2024).
  64. [64]Candemir, S., Nguyen, X. V., Folio, L. R. & Prevedello, L. M. Training strategies for radiology deep learning models in data-limited scenarios. Radiol. Artif. Intell. 3, e210014 (2021).DOI
  65. [65]Ching, T. et al. Opportunities and obstacles for deep learning in biology and medicine. J. royal society interface 15, 20170387 (2018).
  66. [66]Drukker, L., Noble, J. & Papageorghiou, A. Introduction to artificial intelligence in ultrasound imaging in obstetrics and gynecology. Ultrasound Obstet. & Gynecol. 56, 498–505 (2020).DOI
  67. [67]Prevedello, L. M. et al. Challenges related to artificial intelligence research in medical imaging and the importance of image analysis competitions. Radiol. Artif. Intell. 1, e180031 (2019).
  68. [68]Hu, H.-T. et al. Addressing data heterogeneity in distributed medical imaging with heterosync learning. Nat. Commun. 16, 9416 (2025).DOI
  69. [69]Rastogi, N. K. Opportunities and challenges in application of ultrasound in food processing. Critical reviews food science nutrition 51, 705–722 (2011).DOI
  70. [70]Nichol, A. Q. & Dhariwal, P. Improved denoising diffusion probabilistic models. In International conference on machine learning, 8162–8171 (PMLR, 2021).
  71. [73]Zhang, Y., Hartl, B., Hazan, H. & Levin, M. Diffusion models are evolutionary algorithms. arXiv preprint arXiv:2410.02543 (2024).arxiv.org/abs/2410.02543

Paper details

vlnc:W_wqsgx8g6
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 10

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents