WorksComputer Science
Different Models, Shared Failures: Auditing Frozen-Encoder Pools for Medical Segmentation
Abstract
Average segmentation accuracy and model count do not reveal whether models fail on the same cases. We present CoFailMed, an audit of correlated failures in pretrained-encoder pools for medical segmentation. Across five 2D tasks, we evaluate 11 public pretrained encoders (a controlled sample, not a census) using frozen features and separately trained readouts with four seeds per encoder. Shared-checkpoint pairs have the highest mean per-case Dice correlation. An expanded 17-encoder analysis shows that distinct checkpoints within the same encoder family remain more correlated than cross-family pairs. At the pooled 33rd-percentile Dice threshold, four-member pools drawn from distinct encoders have mean effective sizes of 1.56-2.92, estimated from failure correlations in pools with enough failures and passes per member. In the primary pool, richer readouts and fine-tuning narrow the Dice-correlation gap between pairs sharing a checkpoint (or the DINOv2-B/S release) and all other pairs. Yet dependence between different encoders can increase, and fine-tuning does not consistently increase effective pool size at 33rd-percentile thresholds recomputed for each training recipe. A smaller between-group gap can coexist with greater dependence. Pool evaluation should therefore report absolute dependence and threshold-specific effective size alongside accuracy and nominal pool size. The CoFailMed release provides aligned per-case traces, analysis scripts and study code to reproduce the central audit results.
Corresponding author.
Code and data: https://github.com/eunseobchoi/CoFailMed
Introduction
Medical-segmentation pipelines [38, 18] and pathology foundation-model ensembles [59] combine architectures or backbones for complementary predictions. However, diversity measures need not track accuracy gains [54]. Mean Dice does not show whether pool members perform poorly on the same cases. We compare per-case Dice and thresholded failures across frozen and adapted encoder pools on matched test cases (Figure 1).
Figure 1. Shared failures reduce effective pool size. Letters A–K index the 11 frozen encoder checkpoints in the bundle order of Figure 2 (A: DINOv2-B, . . . , K: ResNet-18); numbers index head-training seeds 42–45. Panel (b) compares predictions on the same cases (labels give member Dice) and defines failures by thresholding full-mask Dice. Panel (c) summarises the enumeration of all pools of four distinct retained encoders, taking one head per encoder over all seed combinations; two example pools are shown, followed by an ellipsis. For each pool, positive mean pairwise failure correlation gives an effective size below four. Bars show task-specific mean effective sizes over supported pools at each task’s pooled , on a common 0–4 scale (support counts: Figure 6). Crops show false positives (orange) and false negatives (blue).
Models with different names can share checkpoints, pretraining, input normalisation and decoder training. We find a hierarchy of dependence: shared-checkpoint seed pairs have the highest mean Dice correlation, distinct checkpoints within a family form a weaker tier, and cross-family dependence varies across tasks. Correlation gaps need not track absolute levels: richer readouts and fine-tuning narrow the shared-checkpoint/release gap while cross-encoder correlation stays high or rises.
We contribute (i) a case-aligned co-failure audit with paired resampling (§ 3), (ii) evidence on how checkpoint sharing, encoder family and adaptation relate to dependence on five tasks (§ 5–§ 6), (iii) threshold-specific effective pool sizes (§ 7) and (iv) the released traces, splits and code (§ 8).
Related work
Monoculture and correlated errors. Research on algorithmic monoculture examines risks from shared upstream components [52, 10, 77] and foundation-model homogenisation [9 Sec. 1.1]. Error consistency measures chance-corrected trial-level agreement [23]; other work quantifies error and performance correlations among models sharing algorithms, data or foundation-model components [57]. LLM studies measure correlated errors, output homogeneity and prediction diversity [44, 49, 27, 25], while null-model choices affect how agreement is interpreted [45]. In medical segmentation, pairwise Dice between ensemble predictions supports failure detection [73, 87]. Medical studies evaluate cross-centre robustness [16], clinical benchmarks [11, 67], and complementary domain-specialised features [59]. We instead measure case-level co-failure under checkpoint sharing across segmentation pools.
Representation similarity and ensemble diversity. Checkpoint similarity varies across initialisations and weight-space basins [20, 83]. Transfer studies analyse feature reuse and shared prediction errors [68, 26], and pretraining as a source of diversity [66]. Representation-similarity studies document alignment across initialisations of the same architecture [53, 51]. Medical ensembles can promote diversity [24]; in classification, diversity-promoting interventions can reduce deep-ensemble gains [1]. Diversity metrics and theory motivate our analysis [54, 82]: we correlate per-case Dice scores to characterise shared variation in segmentation quality.
Medical segmentation benchmarks and foundation-model evaluations. Segmentation benchmarks assess Dice [38, 39], while uncertainty studies assess reliability and calibration [64, 46]; related ensembles and uncertainty methods address classification and anomaly detection [56, 21] (Mühllematter et al. 2026; Gu et al. 2024). Failure-detection benchmarks ask whether a method identifies poor segmentations [87]. We compare failures across models, following case-level evaluation principles [62]. Medical segmentation benchmarks [55] and feature-transfer studies [7, 86, 48] assess model suitability and performance. SAM-based, volumetric, video and in-context models broaden the candidate pool [50, 36, 60, 13, 6, 28, 91, 61, 33, 22]. Specification-based evaluation assesses robustness against task-specific criteria [84]. Our audit adds measurements of shared failures in frozen-encoder pools.
Dependence measures
Bundles and traces. Figure 1 illustrates the failure-event audit. Let be the pretrained-encoder bundles, each comprising a concrete checkpoint, wrapper, pretraining, input normalisation and feature-extraction path. A pool member combines a bundle with a separately trained decoder head (readout). Trace denotes the member with decoder seed and records Dice on test case of a task. For distinct traces , is computed within each task.
Pair groups. The same-bundle/release group contains pairs of decoders trained with different seeds on the same frozen checkpoint and all DINOv2-B × DINOv2-S pairs. The latter variants share an upstream release, pretraining recipe and ViT-g/14 distillation teacher [69 Sec. 5]. All remaining pairs form the cross-bundle reference. With group means and over pairs with defined correlations, the primary gap is . We apply the mean-Dice floor defined in § 4; denotes the gap at floor , with unless stated. § 5 separately analyses shared and distinct checkpoints.
Failure and spatial metrics. At Dice threshold , failure is ; ties are passes. The primary threshold is the empirical 33rd percentile of all model-case scores after floor filtering. Failure correlation is the Pearson correlation of the binary indicators; is their chance-corrected agreement, following the error-consistency convention [23]. For members whose pairwise values have mean , the threshold-specific effective size is (Appendix § 15); we report it for supported pools, in which every member has at least five failures and five passes. We also measure spatial overlap by averaging pairwise Dice between prediction-error masks over test cases.
Averaging over decoder seeds. For each pair of distinct bundles, we enumerate all 16 seed combinations (four seeds per bundle). The same-bundle/release group includes six seed pairs per and all 16 DINOv2-BDINOv2-S pairs. Group means weight all defined trace pairs equally.
Uncertainty. The primary paired bootstrap () fixes the floor-selected bundle set, resamples aligned test units and recomputes both group means on each draw. We resample images for Kvasir and RIGA, and patients with all their associated slices for ACDC and BraTS. A hierarchical bootstrap sensitivity analysis also resamples bundle/release groups and traces within groups (Appendix § 13.§ 1). For the primary estimates, we report percentile 95% confidence intervals for the evaluated pools and cohorts. Captions and Appendix § 11 specify undefined-correlation handling and auxiliary resampling units.
Experimental design
Datasets. Table 1 summarises the targets, splits and evaluation units for Kvasir-SEG [40], ACDC [8], BraTS 2024 GLI [de Verdier et al., 2024] and RIGA+ [Hu et al., 2022a,b, Almazroa et al., 2018, Almazroa, 2018, Decencière et al., 2014]. The split files accompany the code, and additional data attributions are given in Appendix B.
| Task | Target and train/test partition | Test units |
| Kvasir polyp | Polyp; official 880/120 training/validation split, using validation for testing [41]. | 120 images |
| ACDC LV | LV-positive end-diastolic and end-systolic (ED/ES) slices; official training patients 001–100, test patients 101–150. | 977 slices; 50 patients |
| BraTS foreground (FG) | 2D FLAIR, ; patient-disjoint 80/20 split, with all sessions kept together and up to 10 eligible slices per session. | 3365 slices; 146 patients, 339 sessions |
| RIGA Cup/Disc | Optic cup/disc; 4-of-6 rater consensus. Train: BinRushed+MESSIDOR; test: Magrabia. | 95 fundus images |
Table 1. Datasets, train/test partitions and test units for the five primary tasks.
Kvasir has no patient/video IDs; all three detected near-duplicate groups are in training; a second group-disjoint split tests partition sensitivity (Appendix C.4). ACDC and BraTS use patient-disjoint partitions and enter as 2D slice derivatives (BraTS: FLAIR slices with all annotated foreground, including resection cavities), so the primary study audits a 2D encoder supply chain rather than native-3D segmentation (Section 6). RIGA provides six annotations per image [4, 42]; we use their 4-of-6 consensus of unedited rater masks (Appendix B.1). Dice uses this consensus reference.
Encoder bundles. The 11-bundle primary pool combines foundation-model encoders with conventional pretrained backbones: DINOv2-B/S [69], BiomedCLIP [88], CLIP ViT-L/14 and ViT-B/16 (CLIP-L/14, CLIP-B/16) [71], MAE ViT-B/16 (MAE-B) [32], DeiT ViT-B/16 (DeiT-B) [78], ConvNeXt-T [58], EfficientNet-B0 [75], and ResNet-50/18 [He et al., 2016]. Table B.1 identifies each wrapper and checkpoint. We evaluate four additional medically pretrained encoders—RETFound [90], Phikon [19], RAD-DINO [70] and BiomedParse [89]—under the same frozen protocol on RIGA Cup/Disc and Kvasir. These form an encoder-only extension (Section 6).
Protocol. Images and masks are resized to using bilinear and nearest-neighbour interpolation, respectively; images then receive each encoder’s upstream normalisation. We cache FP32 frozen features once per bundle and reuse them across decoder seeds, with TF32 disabled. Each convolutional decoder is trained separately using Adam (learning rate ), BCE loss, 30 epochs, batch size 16 and seeds 42–45. Evaluation uses the final-epoch weights; pixels are foreground when sigmoid(logit) . Dice has no smoothing term, and empty-prediction/empty-target pairs score one, following MONAI’s DiceMetric with ignore_empty=False [12]. All primary targets contain foreground at ; empty targets occur in the held-out Medical Segmentation Decathlon (MSD) Spleen task [5], described in Appendix I. Appendix B details normalisation, feature taps and mask alignment.
Mean-Dice filtering. A mean-Dice floor of 0.30 retains 11 bundles on Kvasir, 10 on ACDC, 10 on BraTS and 11 on both RIGA tasks. On ACDC and BraTS, the floor removes only ResNet-18, raising cross-bundle correlation and reducing the gap. The gap stays positive across all tested floors (Appendix D); Table C.1 gives each bundle’s unfiltered mean Dice.
Results
The same-bundle/release group has substantially higher mean per-case Dice correlation than the cross-bundle reference across all five primary tasks (Table 2): – versus –, giving –. Every primary interval excludes zero (case-level for Kvasir and
RIGA; patient-clustered for ACDC and BraTS); the narrower slice-level intervals for ACDC and BraTS are sensitivity analyses. Figure 2 shows the underlying block structure.
Figure 2. Heads sharing a checkpoint have strongly correlated Dice. Pearson correlations of per-case Dice use a shared scale from −1 to 1, centred at zero. Each bundle (letters A–K, as in Figure 1) contributes four adjacent rows and columns, one per decoder seed; the key appears below the matrices. Grey denotes bundles excluded by the mean-Dice floor.
Composition of the same-bundle/release group. Same-bundle decoder-seed pairs account for 78.9–80.5% of this group (Kvasir 66/82; ACDC 60/76) and share identical frozen encoder weights. The remaining pairs are DINOv2-BDINOv2-S. Moving them to the cross group changes the gap to 0.639 on RIGA Cup and 0.300 on Kvasir (Appendix § 16.§ 2, Table F.2).
Distinct checkpoints. We examine distinct checkpoints separately in a 17-bundle pool that adds DINOv2-L, CLIP-B/32, ResNet-34/101 and ConvNeXt-S/B (Table F.2, Figure 3a). Mean same-bundle correlations range from 0.959 to 0.989. Same-family pairs with distinct checkpoints have mean correlations of –0.745, compared with 0.194–0.650 across families. Their gaps relative to cross-family pairs, –0.213, are positive but smaller than those of same-bundle pairs (per-task intervals in Table F.2). This cross-family reference excludes all same-family pairs, unlike the primary cross-bundle reference, which includes them and uses a smaller pool.
Figure 3. Checkpoint hierarchy and sensitivity to the dependence measure. (a) Correlation levels on the 17-bundle pool (point estimates only). (b) Distinct-checkpoint gaps with 95% bootstrap CIs. (c) Same/cross gaps under alternative metrics and pool definitions; hollow markers have no CI. Calibration and intervals resample slices on ACDC/BraTS; their patient-clustered sensitivities are in Appendix § 14. All gaps are positive, but magnitudes across metrics are not directly comparable.
Within the 11-bundle primary pool, ranges from 0.045 [−0.044, 0.126] on RIGA Cup, the only interval that includes zero (the expanded-pool Cup interval excludes it), to 0.165 [0.016, 0.281] on RIGA Disc. The expanded pool shows the highest mean correlation for shared-checkpoint pairs, followed by distinct checkpoints within a family and then cross-family pairs.
Sensitivity to metrics and pool selection. Correlation and agreement gaps remain positive across the examined variants; disagreement gaps are correspondingly negative. In Table D.1, the Pearson gap stays positive with no floor and at a 0.40 floor on every task, and the ICC(A,1) [63] gap exceeds the Pearson gap by 0.038–0.095. Leave-one-bundle-out ranges remain positive (RIGA Cup 0.535–0.653; Kvasir 0.225–0.308), as do patient-aggregated gaps on the two slice-level tasks (Appendix § 13.§ 1). RIGA Cup illustrates sensitivity to pool membership: ResNet-18 narrowly clears the 0.30 floor (mean Dice 0.304, ), but drops out at 0.35 (), leaving a positive gap of 0.541. Failure-event gaps are also positive: gaps are 0.290–0.576 for and 0.329–0.582 for error-consistency , with all intervals excluding zero (Table D.2). Table D.1 also reports disjoint-case calibration selection (Figure 3c) and pair-quality adjustment.
Consensus ground truth. A shared consensus boundary could make cases uniformly hard. Against each individual RIGA rater, the gap stays positive, and with pools fixed by the consensus floor, the consensus gap lies within the single-rater range on Cup and Disc (Table C.2). The consensus reference alone therefore does not produce the gap, although shared annotation boundaries may still contribute.
Spatial overlap. Shared-checkpoint heads also make spatially overlapping errors (Figure 4). Error masks are the prediction–ground-truth XOR at , and overlap is pairwise error-mask Dice averaged over cases. The same-bundle/release-minus-cross gap in error-mask Dice is 0.355 on RIGA Cup, 0.443 on RIGA Disc, 0.343 on official Kvasir and 0.394 on ACDC LV; all gap intervals exclude zero. BraTS FG appears only in the examples; Appendix § 13.§ 3 details the cohort analysis and example selection.
Figure 4. Spatial errors recur across shared-checkpoint heads. Rows show three of the eight area-selected candidate cases (C1, C4, C7; Appendix § 13.§ 3) for RIGA Cup (a) and BraTS FG (b). Shared: DINOv2-B seeds 42–45; distinct: ResNet-50, EfficientNet-B0, CLIP-L/14 and BiomedCLIP at seed 42. At each pixel, colour intensity counts heads with a false-positive (orange) or false-negative (blue) error; green marks the reference boundary in each panel’s first column. Each case uses a common crop. Figure C.1 and Table C.3 give individual examples and cohort-level overlap estimates.
Item-difficulty sensitivity. Co-failure statistics depend on the null model [45]. For each of the 70 partitions of the eight upstream families into four reference and four evaluation families, we regress the evaluation traces on the reference families’ mean Dice, a common case-difficulty proxy. Averaged over partitions, the residual correlation gap exceeds the unadjusted gap of the same evaluation families on every task (by 0.071–0.417) and is positive in every partition, so this difficulty proxy does not explain the gap (exploratory; Appendix § 14.§ 4).
Controls and extensions
These controls test how correlation gaps relate to absolute dependence and effective size.
Shared architecture family versus shared checkpoint. Motivated by representation-similarity work [53], we compare five ViT-B-class bundles—DINOv2-B, BiomedCLIP, MAE-B, DeiT-B and CLIP-B/16—using the same frozen-feature readout architecture. All five pass the mean-Dice floor on every task and share a broad architecture class, while their pretraining differs (Table 3). On RIGA, cross-pretraining correlation is close to the cross-bundle reference and far below the within-bundle seed correlation; on ACDC, Kvasir and BraTS it lies between the two. Training ConvNeXt-T and ResNet-50 from scratch also gives positive gaps: [0.089, 0.179] on ACDC LV and 0.317 [0.193, 0.430] on RIGA Cup. This control uses two backbones and eight seeds per backbone (16 models per task), showing that same-architecture pairs can be more correlated than cross-architecture pairs even without pretraining. With two backbones on two tasks, it is not a dose-matched control for the 11-bundle pool, but it indicates that pretrained-bundle dependence builds on a positive architecture-and-recipe baseline. The contrasts in this section therefore locate dependence without causally separating architecture, pretraining corpus, checkpoint lineage, normalisation, optimisation and decoder recipe.
Table 3: ViT-B-class bundles with different pretraining are less correlated than heads sharing a checkpoint. DINOv2-B, BiomedCLIP, MAE-B, DeiT-B and CLIP-B/16 share the frozen readout. Seed correlation: mean cross-seed correlation within each of the five bundles; ViT cross-pretrain: mean correlation between different ViT bundles; cross-bundle ref.: from Table 2; : seed correlation minus ViT cross-pretrain, with case-bootstrap 95% CI (Table F.1: clustered intervals, pair counts).
| Statistic | Kvasir | ACDC LV | BraTS FG | RIGA Cup | RIGA Disc |
| Seed correlation | 0.981 | 0.993 | 0.988 | 0.960 | 0.971 |
| ViT cross-pretrain | 0.774 | 0.779 | 0.709 | 0.358 | 0.242 |
| Cross-bundle ref. | 0.675 | 0.638 | 0.587 | 0.310 | 0.230 |
| [95% CI] | 0.207 [0.158, 0.270] | 0.215 [0.192, 0.240] | 0.279 [0.261, 0.297] | 0.601 [0.492, 0.711] | 0.729 [0.623, 0.804] |
Table 3.
Decoder capacity. A multi-scale UNet-skip decoder [72] narrows the same/cross gap relative to the frozen readout. Across all 11 source bundles, the gaps are 0.158 [0.115, 0.208] on Kvasir, 0.084 [0.068, 0.100] on ACDC, 0.169 [0.136, 0.209] on RIGA Cup and 0.252 [0.169, 0.339] on RIGA Disc (BraTS not run; Figure 5a; Appendix § 17.§ 1). The primary reference uses the same source bundles (Table 2); ACDC LV retains one additional bundle under UNet-skip. Table G.1 also reports matched nine-bundle comparisons excluding both OpenAI CLIP bundles. The proportional gap reduction is largest on ACDC and smallest on Kvasir. Richer readouts can thus narrow the gap while raising cross-bundle Dice correlation.
Table 2: Heads sharing a bundle or release are more correlated than cross-bundle pairs on every primary task. : mean per-case Dice correlation of the same-bundle/release group () minus that of the cross-bundle reference () after the mean-Dice floor; : retained bundles (four seeds each); : test units. Case and Cluster CIs: percentile 95% intervals () from paired case (image or slice) and patient-clustered bootstraps; the latter is primary for ACDC and BraTS, and – marks image-level tasks. Hier. interval: 2.5th–97.5th percentiles of the descriptive case/patientgrouptrace resampling sensitivity. Shading marks each primary and its primary interval.
| Task | Case CI | Cluster CI | Hier. interval | |||||
| Kvasir | 11 | 120 | 0.940 | 0.675 | 0.265 | [0.217, 0.317] | – | [0.175, 0.395] |
| ACDC LV | 10 | 977 | 0.955 | 0.638 | 0.317 | [0.297, 0.339] | [0.289, 0.344] | [0.188, 0.449] |
| BraTS FG | 10 | 3365 | 0.927 | 0.587 | 0.340 | [0.327, 0.353] | [0.296, 0.388] | [0.234, 0.475] |
| RIGA Cup | 11 | 95 | 0.867 | 0.310 | 0.557 | [0.500, 0.609] | – | [0.444, 0.746] |
| RIGA Disc | 11 | 95 | 0.870 | 0.230 | 0.640 | [0.570, 0.689] | – | [0.495, 0.838] |
Table 2.
Figure 5. Richer readouts reduce the correlation gap. (a) Frozen, UNet-skip, fine-tuned and random-initialisation controls with 95% bootstrap CIs. (b) The nnU-Net fold gap (same-fold minus different-fold pairs) is a different estimand, shown on a separate axis scale. Hollow diamonds mark the random-initialisation CNN (no pretrained weights); absent markers indicate controls not run; absolute correlation levels are needed to interpret a smaller gap.
End-to-end fine-tuning. We unfreeze the same 11 encoders at learning rates and , retaining FP32, the readout architecture, epochs, batch size, seeds and splits. The RIGA Cup gap falls to 0.214 [0.171, 0.259] and 0.284 [0.225, 0.347]; both paired intervals for the gap reduction exclude zero. On official Kvasir, the gaps are 0.182 [0.133, 0.237] and 0.238 [0.172, 0.306], with only the paired interval for the gap reduction excluding zero (Figure 5a; Appendix § 17.§ 2).
The smaller Cup gap accompanies higher cross-bundle dependence. Same/cross correlations are 0.867/0.310 for the frozen readout, 0.806/0.637 for UNet-skip, and 0.871/0.657 and 0.864/0.581 after fine-tuning at and . At the pooled computed separately for each training configuration, mean four-member effective size on Kvasir is 1.559 for the frozen readout versus 1.630 and 1.552 after fine-tuning; on Cup it falls from 2.420 to 1.649 and 1.747 (Table G.4).
Medically pretrained encoders. Mean correlation between the four added medical encoders and the primary pool is lower than the primary cross-bundle reference on Disc and Kvasir; the Cup difference interval includes zero (Table H.2). Adding RETFound, Phikon, RAD-DINO and BiomedParse also lowers the extended pool’s cross-bundle correlation on Disc (0.230→0.197) and Kvasir (0.675→0.649), but raises it on Cup (0.310→0.338). The gap increases on Disc and Kvasir and remains close to its primary value on Cup (Table H.1): lower cross-bundle dependence can accompany a larger gap. These are comparisons of encoder bundles, which differ in architecture, objective and resolution. RAD-DINO shares DINOv2-B lineage, and BiomedParse uses related optic-disc/cup and polyp training data.
nnU-Net pipeline. We train nnU-Net v2 [38] on the RIGA Cup training sources using its self-configuring 2D pipeline, the primary inputs and the cup target (100 epochs; five folds two seeds, 10 models). Every model is evaluated on the same 95 Magrabia images with the primary per-case Dice convention. Mean Dice is 0.863. Mean per-case Dice correlation is 0.825 within folds and 0.758 across folds, giving a fold gap [0.036, 0.119] (case bootstrap); the five fold models of seed 13 have at their own . The ACDC 3D counterpart gives correlations of 0.957 within folds and 0.894 across folds, with [0.032, 0.094] (patient-cluster bootstrap). These fold gaps estimate a different contrast from the same/cross gap in Table 2 (Figure 5b; Appendix § 17.§ 4).
Volumetric abdominal segmentation. On 60 CT and 60 MRI volumes from AMOS22 [Ji et al., 2022], we compare models from two MRSegmentator folds [Häntze et al., 2025] with the modality-specific TotalSegmentator model [80, 2]. Same-system/cross-system correlations are 0.988/0.976 on CT and 0.975/0.918 on MRI. Despite small gaps ( [0.001, 0.029]; 0.057 [0.026, 0.105]), three-member effective sizes at are 1.071 and 1.268 (Appendix § 17.§ 6). The comparison covers two released systems within the nnU-Net family, and MRSegmentator training partly uses TotalSegmentator-assisted labels. Failure-event gaps vary by metric, including a negative MRI double-fault (joint-failure) gap. By contrast, in the primary pool and in UNet-skip Cup/Disc and nnU-Net Cup on the same test cases, , Yule’s and double-fault gaps are positive and disagreement gaps are negative (Tables D.2 and G.6; definitions in Appendix § 11).
Effective pool size
Enumerated pools. The equicorrelation effective size (§ 3; Appendix § 15) is a variance-equivalent size for equal-variance or standardised failure indicators, not a literal count of independent models. We average its defined values over supported pools of four distinct bundles and one seed per bundle. At pooled , the task means range from 1.56 (Kvasir) to 2.92 (RIGA Disc), substantially below four (Figure 6a).
Figure 6. Mean effective size is below nominal size in the evaluated pools. (a) Means over enumerated four-distinct-bundle pools, with 95% CIs conditional on the observed and retained bundles; curves show the formula for and, for reference, the full 11-bundle pool. Points average pool-specific effective sizes, so they can lie above the formula evaluated at mean failure correlation. (b) Calibration-selected pools: frequency of across splits, with 95% Clopper–Pearson intervals over random splits (conditional on the fixed cohort), low/total counts and unsupported counts (U; counted in the total but not as low); all intervals are in Table I.Table 1; †: held-out task. In (a), RIGA Disc supports 48,384 of 84,480 pools; the other tasks support all pools.
Calibration-selected pools. Substantial failure dependence remains in many pools selected from distinct encoder families. We select four family representatives on calibration data and evaluate them on the evaluation halves of the five primary tasks, ISIC-2018 [14, 79, 14] and MSD Hippocampus [5, 74], using 200 random splits per task. Evaluation-half occurs in every split on Kvasir, ACDC, BraTS and ISIC-2018, in 51.5% on Cup, in 2.0% on Disc and in 1.5% on Hippocampus, where 60 of 200 splits are unsupported. Overall, 910/1400 splits have low effective size (65.0%, counting 61 unsupported splits as not low; 68.0% among supported splits). Spleen and Heart are excluded because Spleen scores are dominated by empty targets and no Heart bundle passes the floor (Appendix § 19).
Selection rule. Within each calibration split, we retain bundles passing the 0.30 mean-Dice floor, choose each upstream family’s best member by calibration mean Dice, and take the top four representatives. The families are DINOv2, OpenAI CLIP, BiomedCLIP, MAE, DeiT, ResNet, ConvNeXt and EfficientNet, so a selected pool can contain several ViTs. The failure threshold is the calibration-best member’s , held fixed on the evaluation half. This differs from the pooled model-case used in the primary analysis. ACDC and BraTS are split by patient but retain slice-level failure indicators; Hippocampus is split by volume, without established patient independence.
Discussion
Using the audit. The protocol is intended for retrospective local quality assurance and procurement-style redundancy checks: before a pool is credited with complementary coverage, its candidates are scored on aligned local cases. Select candidates that meet a stated quality criterion on calibration data, then report same-bundle/release and cross-bundle correlation levels on held-out cases, alongside joint-failure rates, threshold-specific and support counts. A small gap between two correlations near one indicates redundancy; the same gap between correlations near zero indicates little linear association. Family diversity guides pool construction; the audit measures remaining dependence. Whether diverse pools keep mean member Dice at or above more often than same-bundle pools when a member fails depends strongly on (Appendix § 14.§ 3).
Scope and limitations. The 11 bundles are a controlled sample, chosen for public checkpoints, compatible dense features and complete four-seed traces, not a census of available encoders. Effect sizes and intervals are therefore conditional on these pools, cohorts and readouts; with eight upstream families in the primary pool, the intervals describe case-level uncertainty, not variation over future encoders. Compared bundles also differ in pretraining, normalisation and training recipe (§ 6), and overlap between test images and pretraining corpora is untested. Kvasir lacks patient/video IDs, Hippocampus is split by volume, RIGA Cup/Disc share images, and BraTS uses 2D slices with a merged foreground target. Per-task intervals do not constitute a joint significance test. Fine-tuning and nnU-Net use reduced budgets. These retrospective benchmark results do not validate clinical safety or outcomes, and we do not assess demographic fairness. Correlation in continuous Dice and agreement on thresholded failures capture different aspects of dependence, and both are needed to interpret the results.
Code and data availability. The CoFailMed repository provides aligned per-case Dice scores for the reported analyses, split identifiers, analysis scripts with reference outputs, a SHA-256 manifest, Croissant metadata and the study code. Its guide maps each table and figure to a command and lists the few values that require masks, predictions or run logs. Values derived from Phikon outputs are withheld under Okwin’s licence; datasets, masks, predictions, feature caches and checkpoints are not redistributed (Appendix § 20).
Conclusion. In the evaluated pools, shared-checkpoint pairs have the highest mean Dice correlation; distinct checkpoints within a family remain more correlated than cross-family pairs. Richer readouts and fine-tuning narrow the primary gap, yet fine-tuning does not consistently increase effective pool size at recipe-specific pooled thresholds. Nominal model diversity therefore does not imply independent failures, and pool audits should measure co-failure directly on held-out cases.
Acknowledgments and Disclosure of Funding
This work was supported by Artificial intelligence industrial convergence cluster development project funded by the Ministry of Science and ICT(MSIT, Korea)&Gwangju Metropolitan City and the IITP AI Semiconductor Program and the ANCHOR Program, funded by the Korean government and the Seoul Metropolitan Government (IITP-2023-RS-2023-00256081; 2026-ANCHOR-01-110).
Competing interests. The authors declare no competing interests.
References
Notation and conventions
Tasks Kvasir (Kvasir-SEG polyp, official split), ACDC LV (left ventricle, official split, slice level), BraTS FG (BraTS 2024 GLI 2D foreground, slice level), RIGA Cup and RIGA Disc (optic cup and disc, Magrabia test source), listed in this order in tables and figures. The held-out tasks are ISIC-2018 and the Hippocampus, Spleen and Heart derivatives of the Medical Segmentation Decathlon (MSD). “(grouped)” marks use of the content-group-disjoint Kvasir split in place of the official split.
Bundle A frozen encoder checkpoint with its wrapper, pretraining, normalisation and feature path (§ 3); the primary pool has four decoder seeds per bundle.
Decoder The separately trained segmentation head on encoder features, also called the readout or head; the primary decoder is a convolution, a linear probe.
The number of bundles retained by the mean-Dice floor ( traces in the primary pool). We call a bundle “functional” when it passes this floor. “/avail.” gives retained / available bundles; “/members” gives retained bundles / retained heads.
, Mean pairwise Pearson correlation of per-case Dice within the same-bundle/release group (same-bundle seed pairs plus all DINOv2-BDINOv2-S seed pairs) and within the cross-bundle reference (all remaining pairs); the pixel-overlap table summarises the same strata by error-mask Dice.
, the primary gap; is the gap at floor , and unless stated. is the same-family/distinct-checkpoint mean minus the cross-family mean (Table F.2); is the mean correlation of medical encoders with primary bundles minus (Tables H.1 and H.2; Table H.1(a) excludes BiomedCLIP from both primary sets); neither is a same/cross gap. (Table 3 and F.1) is within-bundle seed correlation minus ViT cross-pretraining correlation. (nnU-Net, Table G.5) and (AMOS22, Table G.7) are same-fold minus cross-fold and same-system minus cross-system gaps, which estimate different contrasts from .
Intervals Unless stated otherwise, paired 95% percentile intervals use bootstrap draws, recomputing both strata on each resample. The resampling unit is the test image for Kvasir and RIGA and the patient (cluster bootstrap) for ACDC LV and BraTS FG; “slice-level” or “case” resampling uses slices there instead (cases are images for Kvasir and RIGA). “hier.” denotes the descriptive hierarchical case/patientgrouptrace bootstrap (Appendix § 13.§ 1). Cells show estimates above or beside bracketed intervals; “–” means not applicable.
The empirical 33rd percentile of model-case Dice after floor filtering, used as the failure threshold in the primary analyses; pool selection instead uses a calibration-best-model threshold (Appendix § 19.§ 2).
The Pearson correlation of two traces’ binary failure indicators.
Geirhos error consistency [23], with both-never-fail pairs scored as in the official implementation (wichmann-lab/error-consistency, commit eb53bdb).
, disagreement, double fault For two traces on cases, let and count cases on which both pass and both fail, respectively, and , cases on which exactly one fails. Yule’s is undefined when its denominator is zero; the disagreement rate is and the double-fault rate is [54].
, the equicorrelation effective pool size at the stated threshold, defined for supported pools with a positive, numerically nonsingular denominator.
Supported A pool in which every member has at least five failures and five passes at the stated threshold. A pair is defined when both members have nonconstant failure indicators; this is a different criterion.
Joint-fail ratio The ratio of pairwise joint-failure probability in the same-bundle/release group to that in the cross-bundle reference.
Seed correlation The mean same-bundle cross-seed correlation of a bundle set.
17-bundle pool The primary pool plus DINOv2-L, CLIP-B/32, ResNet-34/101 and ConvNeXt-S/B. Its cross-family stratum excludes same-family/distinct-checkpoint pairs and therefore differs from the cross-bundle reference. Shorthand “Same/release” and “cross” abbreviate the same-bundle/release group and the cross-bundle reference, so the “same/cross gap” is ; “DINOv2-BS” denotes the DINOv2-BDINOv2-S seed pairs.
Pearson correlations are undefined when either trace has standard deviation below ; stratum means use finite pairs. Gap intervals omit bootstrap draws without a finite mean in both strata. All five primary gap intervals retain 2000/2000 draws; the degenerate MSD Spleen gap retains 1987/2000.
Implementation details
Datasets, attributions and preprocessing
Both RIGA tasks use the 4-of-6 rater consensus as ground truth: cup is raw label 128 and disc is raw label . Rater masks are used unedited. The training set holds the 649 BinRushed and MESSIDOR images; the 95 Magrabia images form the test set. In one of the 570 Magrabia test rater masks (95 images six raters), the disc rim (label 255) does not fully enclose the cup; because the targets are defined by label values, this mask needs no repair. Appendix § 13.§ 2 re-scores the same predictions against each individual rater. The RIGA+ training images from MESSIDOR [Decencière et al., 2014] are used under the Messidor terms of use: “Kindly provided by the Messidor program partners (see https://www.adcis.net/en/third-party/messidor/).”
The BraTS FG task is a BraTS 2024 GLI 2D FLAIR derivative with 3365 test slices from 339 sessions of 146 patients under a patient-disjoint split (585 training and 146 test patients; 584 training patients have eligible slices). Each session contributes up to 10 axial slices with the largest native-grid foreground areas (seg ), each requiring at least 64 foreground pixels before resizing. The BraTS data-use terms require citing MedPerf [47] and the following attribution: “Data used in this publication were obtained as part of the Brain Tumor Segmentation (BraTS) Challenge project through Synapse ID: syn53708249.” The portal provides data access and GLI label definitions.
ACDC LV uses the official challenge split of Bernard et al. [2018]: heads are trained on patients 001–100 and evaluated on the official test patients 101–150, restricted to the LV-positive end-diastolic and end-systolic (ED/ES) slices (977 slices of 50 patients). The held-out ISIC-2018 task likewise uses the official Task 1 training/test split [14, 79, 14] (1000 test images); Appendix § 19 lists the split units of all held-out tasks.
For ACDC and BraTS, NIfTI arrays are read with NiBabel and sliced along their native third axis, without affine reorientation, cropping, padding or intensity windowing. Each selected slice is converted to float32, min–max scaled over all pixels (including background) to [0, 255] and truncated to uint8; constant slices become zero. The slice is replicated into three channels, bilinearly resized to 224224 with align_corners=False and no antialiasing, then clamped and truncated to uint8. Encoder input conversion divides by 255 before applying the bundle-specific mean and standard deviation. The release implements these loaders and the feature-extraction path in code/src/fmpool/ (§ 8).
Encoder wrappers and head conventions
Table B.1 lists checkpoints and feature tensors for DINOv2 [69], CLIP [71], BiomedCLIP [88], MAE [32], DeiT [78], ResNet [He et al., 2016], ConvNeXt [58], EfficientNet [75], RETFound [90], Phikon [19], RAD-DINO [70] and BiomedParse [89]. The implementations use open_clip [37], timm [81] and torchvision [76], with the shared normalisation and resizing conventions described below.
All bundles use the same 224224 inputs, with bilinear image downsampling without antialiasing and nearest-neighbour mask resizing. Official model presets use antialiasing (torchvision) or bicubic PIL resampling (open_clip, timm, Hugging Face processors), so absolute Dice may differ from the vendor pipelines. The three OpenAI CLIP wrappers read the last transformer block’s patch tokens before CLIP’s final LayerNorm (ln_post); all other ViT wrappers read LayerNorm-normalised tokens. Because the 11 head has no BatchNorm, CLIP features enter it under a different scaling convention.
The head is a simplified DINOv2-style linear segmentation probe (no BatchNorm, no augmentation, inputs, Adam with a constant learning rate). Linear-probe rankings can differ from fine-tuning rankings [48]. The convolution produces one-channel logits on the native feature grid. These are bilinearly upsampled to with align_corners=False before BCE-with-logits training and sigmoid thresholding at evaluation. Features are cached once per bundle and reused by every seed. Head weights are trained separately for each bundle and seed, using a common head architecture and training recipe. The Dice convention of Section 4 follows DiceMetric in MONAI 1.6.0 [12], implemented in monai/metrics/meandice.py.
RAD-DINO (518 px) and BiomedParse (1024 px) have larger native input sizes but, like all bundles, receive inputs. Bundles use ImageNet normalisation except CLIP/BiomedCLIP (OpenAI CLIP normalisation) and RAD-DINO (its processor).
| Bundle | Checkpoint / identifier | |
| DINOv2-S/B/L | torch.hub facebookresearch/dinov2; pinned revision 7764ea0f | |
| CLIP-B/16, B/32, L/14 | open_clip ViT-B-16-quickgelu, ViT-B-32-quickgelu, ViT-L-14-quickgelu; openai weights | |
| BiomedCLIP | open_clip hf-hub microsoft/BiomedCLIP-PubMedBERT_256-vit_base_patch16_224 | |
| MAE-B | timm vit_base_patch16_224.mae | |
| DeiT-B | timm deit_base_patch16_224.fb_in1k | |
| ResNet, ConvNeXt, EfficientNet | torchvision IMAGENET1K_V1 enums: ResNet-18/34/50/101, ConvNeXt-T/S/B, EfficientNet-B0 | |
| RETFound | Hugging Face YukunZhou/RETFound_mae_natureCFP | |
| Phikon | Hugging Face owkin/phikon | |
| RAD-DINO | Hugging Face microsoft/rad-dino (DINOv2 ViT-B/14) | |
| BiomedParse v1 | Hugging Face microsoft/BiomedParse, biomedparse_v1.pt; backbone keys only | |
| (b) Features passed to the decoder | ||
| Bundle | Feature tensor | Grid |
| DINOv2-S/B/L | Final-block patch tokens after LN (x_norm_patchtokens) | |
| CLIP-B/16 | Last-block patch tokens before ln_post; no LN | |
| CLIP-B/32 | Last-block patch tokens before ln_post; no LN | |
| CLIP-L/14 | Last-block patch tokens before ln_post; no LN | |
| BiomedCLIP | Final-norm patch tokens, LN | |
| MAE-B | Final-norm patch tokens, LN | |
| DeiT-B | Final-norm patch tokens, LN; CLS dropped | |
| ResNet | Last stage, layer4 | |
| ConvNeXt | features, without classifier LayerNorm | |
| EfficientNet | features, including head convolution | |
| RETFound | Final encoder norm, CLS dropped, LN | |
| Phikon | Final-norm patch tokens, LN with official | |
| RAD-DINO | Final-block patch tokens, LN | |
| BiomedParse v1 | FocalNet res5, after stage LayerNorm |
Table B.1. Encoder wrappers. Checkpoint identities and decoder inputs. LN denotes final LayerNorm.
Mask resize convention
Table B.2 compares default and exact mask alignment on four tasks. The gaps change by at most 0.0043, is unchanged, and all same/cross gap intervals exclude zero.
Images use bilinear resizing (align_corners=False); masks use PyTorch’s default nearest mode, which floors the scaled destination index rather than aligning pixel centres and shifts foreground centroids by about 0.47 output pixels relative to nearest-exact (task range 0.44–0.49; absolute mean shift per axis, averaged over masks with foreground under both resizes, pooling training and test masks for Kvasir and RIGA and the 1808 official training slices for ACDC). The sensitivity retrains all heads for RIGA Cup, RIGA Disc, ACDC LV (official 100/50-patient split) and official-split Kvasir with nearest-exact masks under the same protocol; BraTS is not evaluated. All bundles, seeds and partitions share the resizing rule, cached features are independent of the masks, and results use the default nearest convention unless labelled nearest-exact.
Table B.2: Mask resize sensitivity. under default nearest and nearest-exact mask resizing, with their paired difference (exact minus nearest) computed on shared bootstrap draws.
| Task | (nearest/exact) | [95% CI] nearest | [95% CI] nearest-exact | Difference [95% CI] (exact − nearest) |
| Kvasir | 11/11 | 0.265 [0.217, 0.317] | 0.267 [0.219, 0.318] | 0.0018 [0.0003, 0.0033] |
| ACDC LV | 10/10 | 0.317 [0.289, 0.344] | 0.322 [0.292, 0.349] | 0.0043 [0.0019, 0.0068] |
| RIGA Cup | 11/11 | 0.557 [0.500, 0.609] | 0.554 [0.498, 0.606] | −0.0029 [−0.0075, 0.0021] |
| RIGA Disc | 11/11 | 0.640 [0.570, 0.689] | 0.639 [0.569, 0.689] | −0.0008 [−0.0048, 0.0038] |
Table B.2. Mask resize sensitivity.
Compute
The released scalar analyses can be recomputed on CPU; the documented reproduction commands complete in about seven minutes in total on a desktop CPU. Feature extraction and decoder training ran on 7 NVIDIA A100-SXM4-80GB GPUs of a shared academic cluster. Table B.3 lists the recorded GPU occupancy by experiment family, measured as the wall-clock union of run intervals per GPU (elapsed run times where per-interval timing was not recorded).
Table B.3: Recorded GPU occupancy (GPU-h) by experiment family. The official-split Kvasir family covers the primary, distinct-checkpoint and medical-encoder pools; the official-split ACDC family covers the primary, distinct-checkpoint, UNet-skip and random-initialisation pools. Family totals also include the Kvasir content-group-split sensitivity runs and unreported runs on an earlier internal ACDC split that the official-split runs replaced, including 10 nnU-Net training runs. Feature extraction is included; nnU-Net preprocessing and inference, native-3D AMOS inference and other preliminary validation runs are excluded. The sum is computed before rounding; distinct occupancy counts overlapping intervals once because inference runs co-ran with training.
| Experiment family | GPU-h | Experiment family | GPU-h |
| Primary 1×1 grid | 8.6 | Per-rater inference | 0.2 |
| Distinct-checkpoint tier | 4.4 | Pixel-overlap inference | 0.4 |
| Medical encoders | 0.3 | Held-out tasks | 0.6 |
| UNet-skip readouts | 25.7 | Mask-alignment sensitivity | 1.2 |
| End-to-end fine-tuning | 29.3 | Official-split Kvasir | 0.7 |
| nnU-Net training | 24.2 | Official-split ACDC and ISIC-2018 | 15.3 |
| Random-initialisation controls | 3.5 | ||
| Sum over families | 114.3 | Distinct occupancy | 113.3 |
Table B.3. Recorded GPU occupancy (GPU-h) by experiment family.
For the official splits, recorded times per frozen-feature encoder cell (feature extraction and four trained heads) were 2.4–10.7 min on Kvasir, 3.9–9.1 min on ACDC LV and 15.0–32.8 min on ISIC-2018 (11 cells per task; cells co-ran on shared GPUs). The 100-epoch nnU-Net training runs took 33.16–34.98 min each on RIGA Cup and 53.73–56.08 min each on official ACDC (10 runs per task). These shared-cluster elapsed times are not inference latency measurements.
Additional results on the primary tasks
Table C.1 reports mean Dice for all 11 primary bundles before mean-Dice filtering. Each value averages per-case Dice across all four decoder seeds under the primary protocol, showing the quality basis for pool selection.
| Bundle | Kvasir | ACDC LV | BraTS FG | RIGA Cup | RIGA Disc |
| DINOv2-B | 0.729 | 0.705 | 0.736 | 0.698 | 0.920 |
| DINOv2-S | 0.729 | 0.686 | 0.711 | 0.592 | 0.911 |
| BiomedCLIP | 0.642 | 0.698 | 0.693 | 0.700 | 0.933 |
| CLIP-L/14 | 0.682 | 0.662 | 0.690 | 0.646 | 0.899 |
| CLIP-B/16 | 0.649 | 0.575 | 0.645 | 0.499 | 0.884 |
| MAE-B | 0.653 | 0.672 | 0.730 | 0.609 | 0.869 |
| DeiT-B | 0.681 | 0.663 | 0.686 | 0.713 | 0.905 |
| ConvNeXt-T | 0.568 | 0.450 | 0.489 | 0.459 | 0.783 |
| EfficientNet-B0 | 0.538 | 0.382 | 0.420 | 0.566 | 0.840 |
| ResNet-50 | 0.496 | 0.367 | 0.442 | 0.391 | 0.789 |
| ResNet-18 | 0.356 | 0.263 | 0.296 | 0.304 | 0.729 |
Table C.1. Primary-bundle segmentation quality before the mean-Dice floor. Mean Dice over test cases and four decoder seeds; ACDC LV and BraTS FG are slice-level means. All 11 bundles are shown, including those below the 0.30 floor. ResNet-18 falls below the floor on ACDC LV and BraTS FG.
Bootstrap intervals and patient clustering
The intervals for the patient-clustered, slice-level, hierarchical and patient-aggregated estimates below all exclude zero.
The case interval resamples test units with replacement and recomputes both strata on each draw. For ACDC LV and BraTS FG, the primary interval resamples patients, retaining all slices of each sampled patient. Slice-level intervals are narrower; Table 2 reports both.
The hierarchical interval resamples cases (patients for ACDC LV and BraTS FG), bundle/release groups and traces within groups, all with replacement. DINOv2-B/S form one eight-trace release group; each other retained bundle forms a separate four-trace group. Same-group pairs come from one sampled occurrence; cross-group pairs have different source groups, and pairs between separate occurrences of the same source group are excluded. Duplicate trace draws within an occurrence retain their self-correlations; excluding them lowers the lower endpoints by at most 0.043 (RIGA Cup 0.444→0.409, RIGA Disc 0.495→0.452).
ACDC LV evaluates 977 LV-positive slices from 50 patients, identified by the released split files. Averaging each head’s per-slice Dice within patient before computing Pearson gives a different estimand from the clustered slice-level statistic: ACDC =0.493 [0.391, 0.594] (50 patients) and BraTS =0.204 [0.160, 0.253] (146 patients). Patient aggregation in the distinct-checkpoint tier of Table F.2 gives ACDC 0.315 [0.224, 0.411] and BraTS 0.103 [0.070, 0.138].
Per-rater RIGA re-scoring
The correlation gap remains positive against every individual RIGA rater (Table C.2).
We score the same primary predictions for both RIGA+ [34 b] tasks against each of the six rater masks and the 4-of-6 consensus used in Table 2. The GT-specific pool applies the Dice floor separately for each ground truth, so can vary by rater. The fixed pool retains the bundles selected using the consensus.
Section 5 compares the consensus gaps with the single-rater ranges. On Cup, the consensus gap exceeds the GT-specific single-rater range by 0.012; the consensus pool retains ResNet-18, whereas four rater-specific pools exclude it. These comparisons therefore reflect both annotation and pool-composition differences.
(a) RIGA Cup
| GT | /members | [95% CI] | /members | [95% CI] | ||
| Rater 1 | 10/40 | 0.876 0.331 | 0.545 [0.487, 0.598] | 11/44 | 0.876 0.313 | 0.563 [0.506, 0.614] |
| Rater 2 | 10/40 | 0.879 0.370 | 0.509 [0.447, 0.567] | 11/44 | 0.878 0.342 | 0.536 [0.481, 0.590] |
| Rater 3 | 10/40 | 0.882 0.342 | 0.539 [0.479, 0.593] | 11/44 | 0.880 0.330 | 0.550 [0.490, 0.605] |
| Rater 4 | 10/40 | 0.883 0.372 | 0.511 [0.451, 0.570] | 11/44 | 0.882 0.344 | 0.537 [0.480, 0.592] |
| Rater 5 | 11/44 | 0.870 0.344 | 0.526 [0.472, 0.577] | 11/44 | 0.870 0.344 | 0.526 [0.472, 0.577] |
| Rater 6 | 11/44 | 0.879 0.364 | 0.516 [0.454, 0.574] | 11/44 | 0.879 0.364 | 0.516 [0.454, 0.574] |
| Consensus | 11/44 | 0.867 0.310 | 0.557 [0.500, 0.609] | 11/44 | 0.867 0.310 | 0.557 [0.500, 0.609] |
(b) RIGA Disc
| GT | /members | [95% CI] | |
| Rater 1 | 11/44 | 0.863 0.216 | 0.647 [0.582, 0.695] |
| Rater 2 | 11/44 | 0.902 0.275 | 0.627 [0.526, 0.704] |
| Rater 3 | 11/44 | 0.902 0.278 | 0.623 [0.553, 0.685] |
| Rater 4 | 11/44 | 0.880 0.230 | 0.651 [0.580, 0.702] |
| Rater 5 | 11/44 | 0.906 0.279 | 0.627 [0.553, 0.682] |
| Rater 6 | 11/44 | 0.888 0.267 | 0.621 [0.564, 0.665] |
| Consensus | 11/44 | 0.870 0.230 | 0.640 [0.570, 0.689] |
Table C.2. Per-rater re-scoring of the RIGA heads. GT-specific pool: floor applied per ground truth; fixed pool: bundles retained on the consensus; /members: retained bundles / retained heads. On RIGA Disc every rater-specific floor retains all 11 bundles, so the two pools coincide and one block is shown.
Pixel-level error overlap, example cases and per-case scatter
Table C.3 reports the error-mask Dice strata and error-type breakdown; the same-bundle/release group has greater overlap for both false positives and false negatives. Figure C.1 shows individual-head errors on two example cases, and Figure C.2 illustrates the stronger per-case Dice correlation of a same-bundle pair relative to a cross-bundle pair.
Cohort overlap. For primary heads from bundles retained by the floor, error masks are the prediction XOR the ground truth at . We compute Dice between each pair of heads’ error masks and average over test cases. Strata follow Table 2; the DINOv2-B×S column isolates the only distinct-checkpoint pairs in the same-bundle/release group. BraTS FG appears only in the fixed-case examples; the cohort-level overlap covers the other four primary tasks. The patient-resampled ACDC LV gap interval is [0.380, 0.409]. Kvasir uses the official-split heads; on the content-group split, the gap is 0.390 [0.370, 0.412]. The gap persists for both false-positive-only and false-negative-only masks, so spatial co-failure is not driven by a single error type.
Empty error masks. When both error masks are empty, the unconditional Dice is one. No observed pair has two empty full error masks (false positives false negatives) on the same case; this can occur for FP-only and FN-only masks. Table C.4 reports these counts and recomputes the mean after excluding comparisons with both masks empty. Within each stratum, we average defined pairs per case, then average these case means, omitting cases with no defined comparisons. All conditional gap intervals remain positive; they resample cases (slices for ACDC), not patients.
Example selection. Figure C.1 shows, for each task, the test case whose mean Dice over the eight shown heads is closest to the mean of the other seven panel cases (RIGA Cup: MagrabiFemale image 5; BraTS FG: BraTS-GLI-02193-104, slice 112). Crops are centred on the union of ground truth and the eight predictions. Selection is confined to these eight panel candidates (the middle eight test cases with positive ground-truth area, sorted by area and case identifier; C1–C8 follow this stored order and denote different cases across tasks), which are not a random cohort sample. Figure 4 displays fixed candidate indices C1, C4 and C7 per task, including those two examples; each case uses the same union-based crop for both head groups.
Error-count maps. For a displayed four-head group with binary predictions and ground truth , Figure 4 counts false positives and false negatives at each pixel:
The common discrete colour scales run from one to four; pixels with zero errors retain the greyscale image. These fixed-case counts describe error recurrence; the cohort estimates in Table C.3 use pairwise error-mask Dice.
Kvasir official split and content-group sensitivity
Table C.5 compares the official split with the content-group-disjoint split; both gap intervals exclude zero.
The primary Kvasir task uses the official Kvasir-SEG training/validation split of Iha et al. [2021] (880/120), with validation used for testing. All three detected near-duplicate groups lie in its training set. The sensitivity uses a second content-group-disjoint split (880/120), which also places every detected near-duplicate group in training; it therefore tests sensitivity to the test partition rather than correcting leakage. The test sets share 12 of 120 images and are not fully case-matched. The cross-split bootstrap resamples them separately; it is descriptive because it omits dependence from shared test images. Content groups combine exact image-file hash matches, one audited near-duplicate pair, and pairs with greyscale-image correlation and mask Dice at . Masks are nearest-sized and thresholded at 127; overlapping groups merge transitively. The grouped partition uses SHA-256 ordering of group IDs and deterministic allocation of 120 test images. The release records the audited pair, group memberships and final partitions.
Figure C.1. Individual-head errors on two example cases. RIGA Cup C1 and BraTS FG C7 from Figure 4, with false positives, false negatives and ground-truth boundaries; labels give per-head Dice. The left group shares DINOv2-B across decoder seeds 42–45; the right uses ResNet-50, EfficientNet-B0, CLIP-L/14 and BiomedCLIP at seed 42. Cohort-level overlap estimates are given in Table C.3.
Table C.3: Pixel-level error-mask Dice. Panel (a) gives mean overlap per primary stratum and the same-bundle/release-minus-cross-bundle gap. Panel (b) separates pair types and false-positive/false-negative errors. Its final column compares same-bundle pairs with all distinct-bundle pairs, with the DINOv2-B×S pairs in the cross reference. Intervals are case-level (slice-level for ACDC LV, whose patient-clustered gap interval is given in the text).
(a) Primary strata and gap
| Task | Same/release group | Cross-bundle reference | [95% CI] | ||
| Kvasir | 120 | 11 | 0.842 [0.831, 0.854] | 0.499 [0.467, 0.534] | 0.343 [0.319, 0.366] |
| ACDC LV | 977 | 10 | 0.860 [0.856, 0.863] | 0.465 [0.455, 0.476] | 0.394 [0.386, 0.403] |
| RIGA Cup | 95 | 11 | 0.842 [0.830, 0.853] | 0.486 [0.458, 0.514] | 0.355 [0.334, 0.378] |
| RIGA Disc | 95 | 11 | 0.840 [0.830, 0.848] | 0.397 [0.377, 0.417] | 0.443 [0.428, 0.457] |
(b) Pair types and error types
| Task | Same-bundle pairs | DINOv2-B×S | FP-only | FN-only | Same minus distinct bundle |
| Kvasir | 0.896 [0.889, 0.904] | 0.619 [0.586, 0.653] | 0.467 [0.450, 0.484] | 0.319 [0.292, 0.345] | 0.395 [0.368, 0.420] |
| ACDC LV | 0.925 [0.924, 0.927] | 0.614 [0.600, 0.627] | 0.503 [0.495, 0.511] | 0.365 [0.356, 0.375] | 0.457 [0.448, 0.466] |
| RIGA Cup | 0.904 [0.898, 0.910] | 0.584 [0.541, 0.625] | 0.447 [0.424, 0.471] | 0.340 [0.317, 0.363] | 0.416 [0.393, 0.440] |
| RIGA Disc | 0.907 [0.903, 0.911] | 0.562 [0.523, 0.596] | 0.514 [0.497, 0.530] | 0.459 [0.438, 0.481] | 0.507 [0.489, 0.525] |
Table C.3. Pixel-level error-mask Dice.
Figure C.2. Per-case Dice correlation within and across bundles. DINOv2-B seed 42 is compared with seed 43 of the same bundle or ResNet-50 seed 42 on Kvasir and RIGA Cup. Panels report Pearson and case count; dashed lines indicate identical Dice.
| Task | Error | Both-empty/total same/release | Both-empty/total cross-bundle | Conditional [95% CI] |
| Kvasir | FP | 905/9840 | 4524/103,680 | 0.465 [0.450, 0.480] |
| Kvasir | FN | 92/9840 | 160/103,680 | 0.317 [0.291, 0.343] |
| ACDC LV | FP | 12,356/74,252 | 59,994/687,808 | 0.519 [0.513, 0.526] |
| ACDC LV | FN | 2110/74,252 | 4563/687,808 | 0.363 [0.354, 0.372] |
| RIGA Cup | FP | 2194/7790 | 12,688/82,080 | 0.466 [0.444, 0.489] |
| RIGA Cup | FN | 372/7790 | 1004/82,080 | 0.334 [0.313, 0.358] |
| RIGA Disc | FP | 708/7790 | 1807/82,080 | 0.504 [0.487, 0.519] |
| RIGA Disc | FN | 424/7790 | 1176/82,080 | 0.453 [0.433, 0.473] |
Table C.4. Empty-error-mask sensitivity. Counts of head-pair comparisons on individual cases in which both error masks are empty, and the same/release-minus-cross gap after excluding those comparisons.
in code/data/splits/kvasir_grouped_split.json and kvasir_official_split.json; code/src/fmpool/datasets/kvasir.py implements the grouping rule.
ResNet-18 passes the mean-Dice floor on the official split (mean Dice 0.356). At a 0.40 floor, it is excluded (), giving a gap of 0.225. The Kvasir pixel-overlap values in Appendix § 13.§ 3 also use the official-split heads.
| Analysis | Official split | Content-group split |
| Primary 1×1: | 11 | 11 |
| Primary 1×1: / | 0.940 / 0.675 | 0.933 / 0.610 |
| Primary 1×1: | 0.265 [0.217, 0.317] | 0.323 [0.257, 0.403] |
| Distinct-checkpoint tier: | 0.086 [0.047, 0.126] | 0.117 [0.066, 0.169] |
| Medical encoders: pooled | −0.058 [−0.106, −0.012] | −0.050 [−0.099, −0.006] |
| Medical encoders: 15-bundle | 0.302 [0.254, 0.357] | 0.360 [0.288, 0.442] |
| UNet-skip: | 0.158 [0.115, 0.208] | 0.173 [0.108, 0.251] |
Table C.5. Kvasir on the official split and on the content-group split; cells give estimates with 95% CIs. The two test sets share 12 of 120 images, so the columns are not fully paired. The distinct-checkpoint tier uses the 17-bundle pool; the pooled medical-encoder uses the full primary reference (Table H.2); the content-group UNet-skip pool retains .
Sensitivity to metrics and analysis choices
The following diagnostics use the same traces to examine different mean-Dice floors, correlation statistics, pool definitions, pair-quality adjustments, failure thresholds and item-difficulty nulls (Tables D.1–D.5). Some change the metric or comparison population rather than estimating the same . Correlation and chance-corrected agreement gaps remain positive; disagreement-rate gaps are negative (Table D.2). Magnitudes are metric-specific. All analyses in this appendix are descriptive sensitivity analyses of the evaluated pools: regression coefficients are associational, and permutation tail fractions are not calibrated -values.
Floor, correlation statistic, pool definition and pair-quality controls
Table D.1 reports with no floor and at floors from 0.20 to 0.40, with ICC(A,1), with each bundle or task left out, and on bundles selected using disjoint calibration data. It also reports the same-bundle/release coefficient from a pair-quality regression and the fit’s ; its caption gives the pair and refit counts. Every variant remains positive, and every reported interval excludes zero.
The absolute-agreement, single-measurement intraclass correlation ICC(A,1) [63] gives a larger gap than Pearson on every task. The leave-one-bundle-out range covers omission of each retained bundle.
The five-task mean Pearson gap at the 0.30 floor is 0.424 (median 0.340, range 0.265–0.640). A bundle-label permutation diagnostic, which reapplies the release grouping after each shuffle, compares the observed gap with gaps under randomly reassigned labels (Table D.5; Appendix § 14.§ 4).
The calibration split assigns 35% of each task’s aligned units (images for Kvasir and RIGA, patients for ACDC LV and BraTS FG) to calibration, taking units in SHA-256 order of their task-prefixed identifiers; the remaining units form the evaluation part. Bundles are retained when their calibration mean Dice reaches the 0.30 floor, and is evaluated on the evaluation part with case-bootstrap intervals. This single fixed split differs from the 200 random half-splits used for pool selection (Appendix § 19.§ 2). It is a leakage-sensitivity analysis, not a prospective validation protocol.
The pair-quality regression is fitted by ordinary least squares over all pairs of traces retained in Table 2 that have a defined correlation:
where and are the mean and variance over cases, and and are their pair averages. The absolute differences control for within-pair quality and variance imbalance. The indicator follows the primary grouping rule (same-bundle pairs plus all DINOv2-BS seed pairs). The positive coefficient indicates that the association persists after these controls. Its interval uses case-bootstrap refits (Table D.1(c)).
For ACDC LV and BraTS FG, the calibration-split and columns resample slices. The corresponding patient-resampled intervals are [0.234, 0.305] and [0.191, 0.241] on ACDC LV, and [0.313, 0.430] and [0.194, 0.285] on BraTS FG. These are 1.2–3.9 wider than the slice-level intervals; all exclude zero.
Table D.1: Primary-gap sensitivity to metrics and pool selection. at the 0.30 floor with its primary interval, with no floor and at other floors (retained in parentheses), with ICC(A,1), over leave-one-bundle-out pools (LOO range), and as the mean over the other four tasks when each task is left out. Calibration-selected bundles (calibration/evaluation units and selected count) and the pair-quality coefficient appear in panel (c), which uses image/slice-bootstrap intervals, including slice resampling for ACDC and BraTS; (Eq. (1)) is fitted over all retained trace pairs of Table 2 (82 same and 864 cross pairs with 11 bundles; 76 and 704 with 10) with refits, and is that fit’s coefficient of determination. Patient-clustered sensitivities are given in the text.
(a) Mean-Dice floor sensitivity
| Task | [95% CI] | No floor () | 0.20 () | 0.25 () | 0.35 () | 0.40 () |
| Kvasir | 0.265 [0.217, 0.317] | 0.265 | 0.265 (11) | 0.265 (11) | 0.265 (11) | 0.225 (10) |
| ACDC LV | 0.317 [0.289, 0.344] | 0.351 | 0.351 (11) | 0.351 (11) | 0.317 (10) | 0.227 (8) |
| BraTS FG | 0.340 [0.296, 0.388] | 0.382 | 0.382 (11) | 0.382 (11) | 0.340 (10) | 0.340 (10) |
| RIGA Cup | 0.557 [0.500, 0.609] | 0.557 | 0.557 (11) | 0.557 (11) | 0.541 (10) | 0.531 (9) |
| RIGA Disc | 0.640 [0.570, 0.689] | 0.640 | 0.640 (11) | 0.640 (11) | 0.640 (11) | 0.640 (11) |
(b) Correlation and leave-out analyses
| Task | ICC(A,1) | LOO range | Leave-task-out mean |
| Kvasir | 0.334 | 0.225–0.308 | 0.464 |
| ACDC LV | 0.397 | 0.278–0.361 | 0.451 |
| BraTS FG | 0.435 | 0.314–0.399 | 0.445 |
| RIGA Cup | 0.596 | 0.535–0.653 | 0.391 |
| RIGA Disc | 0.716 | 0.602–0.750 | 0.370 |
(c) Calibration selection and pair-quality adjustment
| Task | Cal./eval. units | Selected | Cal.-split [95% CI] | [95% CI] | |
| Kvasir | 42/78 | 10 | 0.210 [0.161, 0.273] | 0.153 [0.115, 0.201] | 0.836 |
| ACDC LV | 18/32 | 9 | 0.269 [0.243, 0.294] | 0.221 [0.200, 0.241] | 0.917 |
| BraTS FG | 51/95 | 11 | 0.366 [0.352, 0.382] | 0.233 [0.220, 0.247] | 0.901 |
| RIGA Cup | 33/62 | 10 | 0.546 [0.473, 0.616] | 0.544 [0.476, 0.608] | 0.713 |
| RIGA Disc | 33/62 | 11 | 0.630 [0.534, 0.695] | 0.572 [0.493, 0.631] | 0.687 |
Table D.1.
Failure thresholds and failure-event statistics
Table D.2: Failure-event gaps and effective pool size at . (a) Same/release minus cross-bundle gaps in , , Yule’s , disagreement rate and double-fault rate. (b) Joint-fail ratio (same/release over cross-bundle pairwise joint-failure probability) and equicorrelation of nominal four-member pools drawn from distinct bundles (mean over supported pools with a defined ratio, with supported/total counts), also at the fixed , whose support is lower on Kvasir and RIGA Cup (Table E.1(a)). Threshold handling and resampling units are given in the text below.
| (a) Same/release minus cross-bundle failure-event gaps | |||||
| Task | Yule’s | Disagreement | Double fault | ||
| Kvasir | 0.290 [0.243, 0.356] | 0.329 [0.273, 0.388] | 0.072 [0.039, 0.113] | −0.176 [−0.209, −0.147] | 0.055 [0.043, 0.067] |
| ACDC LV | 0.405 [0.378, 0.436] | 0.449 [0.428, 0.471] | 0.128 [0.107, 0.151] | −0.233 [−0.244, −0.221] | 0.084 [0.079, 0.088] |
| BraTS FG | 0.425 [0.391, 0.460] | 0.469 [0.458, 0.480] | 0.169 [0.154, 0.183] | −0.252 [−0.258, −0.246] | 0.088 [0.086, 0.091] |
| RIGA Cup | 0.537 [0.469, 0.590] | 0.558 [0.502, 0.613] | 0.438 [0.341, 0.559] | −0.282 [−0.313, −0.252] | 0.117 [0.103, 0.131] |
| RIGA Disc | 0.576 [0.512, 0.652] | 0.582 [0.517, 0.680] | 0.547 [0.291, 0.658] | −0.362 [−0.388, −0.336] | 0.128 [0.120, 0.136] |
| (b) Joint failures and effective pool size | |||||
| Task | Joint-fail ratio [95% CI] | [95% CI] | Supported/total pools at | ||
| Kvasir | 1.259 [1.190, 1.353] | 1.559 [1.454, 1.704] | 84,480/84,480 | 1.865 | |
| ACDC LV | 1.427 [1.373, 1.491] | 1.645 [1.573, 1.728] | 53,760/53,760 | 2.360 | |
| BraTS FG | 1.498 [1.421, 1.583] | 1.828 [1.714, 1.966] | 53,760/53,760 | 2.154 | |
| RIGA Cup | 1.800 [1.651, 1.944] | 2.420 [2.182, 2.749] | 84,480/84,480 | 2.309 | |
| RIGA Disc | 2.071 [1.968, 2.212] | 2.918 [2.626, 3.308] | 48,384/84,480 | 3.001 |
Table D.2.
Table D.2 reports the gaps in , , Yule’s , disagreement and double-fault rate, together with the joint-fail ratio and the effective size of nominal four-member pools: the , , and double-fault gaps are positive with intervals excluding zero, the disagreement gap is negative, and all reported pool-mean values lie below four. Table D.3 scores the traces at the empirical 25th, 33rd and 50th percentiles of model-case Dice after floor filtering.
These empirical thresholds define failure relative to each task’s score distribution. Both and the joint-fail ratio use draws with the percentile threshold re-estimated in each draw. The threshold and the retained bundles are held fixed for , , disagreement, double fault and . Intervals for , , disagreement and double fault resample cases on every task; the , joint-fail-ratio and intervals are patient-clustered for ACDC LV and BraTS FG. For ACDC LV and BraTS FG, the gap resamples slices; its patient-resampled intervals [0.416, 0.481] and [0.435, 0.503] exclude zero.
On RIGA Disc, 2 of the 44 heads (BiomedCLIP seeds 43 and 45) never fail at . A pair with exactly one never-failing head has by construction because observed agreement equals chance agreement; these pairs are retained. The single pair in which both heads never fail uses the convention in Appendix A.
The Disc gap is 0.582 under this convention and remains positive when the both-never-failing pair is excluded. Excluding every pair involving a never-failing head gives 0.604. Both and exclude pairs with a never-failing head. The five-failure/five-pass rule excludes five heads: all four BiomedCLIP heads (1, 0, 1 and 0 failures for seeds 42–45) and DINOv2-B seed 45 (4 failures). Thus, 48,384 of 84,480 four-member Disc pools have a defined ; its mean is conditional on these supported pools.
| Task | Percentile | gap [95% CI] | Joint-fail ratio [95% CI] | Defined/total pairs (same, cross) | ||
| Kvasir | 0.461 | 0.833 0.526 | 0.307 [0.250, 0.376] | 1.343 [1.246, 1.471] | 82/82, 864/864 | |
| 0.569 | 0.816 0.525 | 0.290 [0.243, 0.356] | 1.259 [1.190, 1.353] | 82/82, 864/864 | ||
| 0.715 | 0.852 0.520 | 0.333 [0.283, 0.392] | 1.174 [1.135, 1.223] | 82/82, 864/864 | ||
| ACDC LV | 0.390 | 0.881 0.508 | 0.373 [0.347, 0.399] | 1.465 [1.398, 1.540] | 76/76, 704/704 | |
| 0.547 | 0.885 0.480 | 0.405 [0.378, 0.436] | 1.427 [1.373, 1.491] | 76/76, 704/704 | ||
| 0.715 | 0.849 0.397 | 0.452 [0.420, 0.481] | 1.295 [1.264, 1.328] | 76/76, 704/704 | ||
| BraTS FG | 0.540 | 0.821 0.399 | 0.422 [0.384, 0.463] | 1.645 [1.531, 1.790] | 76/76, 704/704 | |
| 0.616 | 0.821 0.396 | 0.425 [0.391, 0.460] | 1.498 [1.421, 1.583] | 76/76, 704/704 | ||
| 0.713 | 0.826 0.387 | 0.439 [0.408, 0.467] | 1.286 [1.254, 1.315] | 76/76, 704/704 | ||
| RIGA Cup | 0.388 | 0.742 0.182 | 0.560 [0.505, 0.609] | 2.178 [1.977, 2.386] | 82/82, 864/864 | |
| 0.521 | 0.756 0.219 | 0.537 [0.469, 0.590] | 1.800 [1.651, 1.944] | 82/82, 864/864 | ||
| 0.655 | 0.700 0.203 | 0.497 [0.450, 0.546] | 1.371 [1.317, 1.431] | 82/82, 864/864 | ||
| RIGA Disc | 0.835 | 0.717 0.105 | 0.612 [0.532, 0.713] | 2.676 [2.478, 2.874] | 76/82, 704/864 | |
| 0.858 | 0.698 0.123 | 0.576 [0.512, 0.652] | 2.071 [1.968, 2.212] | 77/82, 784/864 | ||
| 0.892 | 0.724 0.111 | 0.613 [0.542, 0.662] | 1.497 [1.440, 1.557] | 82/82, 864/864 |
Table D.3. Binary failure-event diagnostic across thresholds. per stratum, the gap and the joint-fail ratio at each percentile threshold ; defined/total: pairs with a finite ; joint-failure rates average all pairs.
Conditional mean-score threshold crossing
Table D.4 varies the threshold . The recovery advantage , the excess rate at which diverse rather than same-bundle pools keep mean member Dice at or above when a member fails, is largest at the lowest examined threshold () on four tasks and is strongly threshold-dependent.
For each pool, we compute , then average across pools. Same-bundle pools contain four seeds of one functional bundle; diverse pools enumerate all four-bundle functional compositions and seed tuples. The difference is the diverse-pool rate minus the same-bundle rate. At , the difference is small and negative on Kvasir, ACDC LV, BraTS FG and RIGA Cup; RIGA Disc retains a positive difference, with its peak at (Table D.4). Conditioning is performed separately within each pool. Pools with no member failures have undefined conditional rates and are omitted from the pool-mean summary.
| Task | Rate | 0.70 | 0.75 | 0.80 | 0.85 | 0.90 |
| Kvasir | Same-bundle | 0.075 | 0.081 | 0.067 | 0.043 | 0.026 |
| Diverse | 0.278 | 0.242 | 0.182 | 0.100 | 0.023 | |
| 0.203 | 0.162 | 0.114 | 0.057 | |||
| ACDC LV | Same-bundle | 0.068 | 0.061 | 0.056 | 0.039 | 0.019 |
| Diverse | 0.310 | 0.241 | 0.157 | 0.060 | 0.006 | |
| 0.242 | 0.180 | 0.101 | 0.021 | |||
| BraTS FG | Same-bundle | 0.087 | 0.069 | 0.042 | 0.020 | 0.003 |
| Diverse | 0.289 | 0.192 | 0.106 | 0.034 | 0.001 | |
| 0.203 | 0.123 | 0.065 | 0.014 | |||
| RIGA Cup | Same-bundle | 0.087 | 0.083 | 0.054 | 0.028 | 0.005 |
| Diverse | 0.223 | 0.132 | 0.056 | 0.015 | 0.001 | |
| 0.136 | 0.049 | 0.002 | ||||
| RIGA Disc | Same-bundle | 0.426 | 0.162 | 0.196 | 0.264 | 0.110 |
| Diverse | 0.945 | 0.886 | 0.781 | 0.565 | 0.225 | |
| 0.519 | 0.725 | 0.585 | 0.302 | 0.114 |
Table D.4. Conditional mean-score threshold crossing. Same-bundle and diverse rates and their difference (diverse minus same-bundle) at each ; a member-score diagnostic, not the Dice of an aggregated ensemble prediction.
Item-difficulty sensitivity with a common reference panel
For each four-family reference subset of the eight upstream families, define the common difficulty regressor
where counts the retained traces in family . Thus traces are averaged within each family before the family means receive equal weight. The complementary families form the evaluation panel on the same cases. For each evaluation trace, fit by OLS with an intercept and compute the gap on the residuals. The regressor is common to all evaluation traces within a panel; the gaps from all complementary panels receive equal weight. Each paired case/patient bootstrap draw reconstructs the family means and common regressor and refits every OLS model. The retained bundles and panel definitions remain fixed.
The residual gap exceeds the raw gap on every task and is positive on every eligible panel. The matched raw gap is the equally weighted mean over the 70 complementary evaluation panels (e.g. ACDC LV 0.328 vs. 0.317 in Table 2). This panel weighting differs from the weighting of the full-pool gap in Table 2.
Table D.5 gives the raw Pearson gap’s bundle-label permutation null, with tail fraction . The raw-gap null shuffles the multiset of bundle labels across individual complete traces, preserving label counts but breaking the original checkpoint/seed clusters, then reapplies the same-bundle/release grouping. Its upper-tail fraction is , where counts valid permutations; because bundle labels are not exchangeable sampling units, this fraction is descriptive. This is not a null for the residualised gap. A separate residualised-gap null (200 bundle-label permutations, with the family reference and evaluation panels rebuilt from the permuted labels) has task means between and and a maximum of 0.169, below every observed residualised gap.
Table D.5: Common-panel item-difficulty sensitivity. Raw and residualised , their difference and residual-to-raw ratio; eligible/total reference panels (positive residual-gap count in parentheses); and the range and SD of residual gaps across panels. The raw-gap permutation null gives 2.5th/97.5th percentiles; over 1000 permutations, every task has tail fraction , which is not a calibrated -value. For slice-level tasks, we also report the residual-minus-raw gap after patient aggregation.
(a) Raw and residualised gaps
| Task | Raw [95% CI] | Residual [95% CI] | Residual − raw [95% CI] | Residual/raw [95% CI] |
| Kvasir | 0.266 [0.219, 0.317] | 0.683 [0.645, 0.715] | 0.417 [0.364, 0.467] | 2.568 [2.163, 3.078] |
| ACDC LV | 0.328 [0.298, 0.356] | 0.698 [0.682, 0.713] | 0.370 [0.339, 0.403] | 2.127 [1.956, 2.341] |
| BraTS FG | 0.351 [0.305, 0.399] | 0.686 [0.671, 0.700] | 0.335 [0.292, 0.375] | 1.955 [1.738, 2.217] |
| RIGA Cup | 0.570 [0.514, 0.624] | 0.725 [0.696, 0.751] | 0.155 [0.107, 0.205] | 1.271 [1.175, 1.395] |
| RIGA Disc | 0.655 [0.592, 0.701] | 0.725 [0.672, 0.761] | 0.071 [0.041, 0.106] | 1.108 [1.061, 1.172] |
(b) Panel coverage, variation and permutation diagnostic
| Task | Eligible/total (positive) | Panel range | Panel SD | Permutation null [2.5th, 97.5th] | Patient-level residual − raw |
| Kvasir | 70/70 (70) | 0.333–0.864 | 0.129 | [−0.025, 0.031] | – |
| ACDC LV | 70/70 (70) | 0.299–0.895 | 0.140 | [−0.030, 0.040] | 0.235 |
| BraTS FG | 70/70 (70) | 0.327–0.897 | 0.134 | [−0.029, 0.039] | 0.484 |
| RIGA Cup | 70/70 (70) | 0.518–0.921 | 0.111 | [−0.040, 0.048] | – |
| RIGA Disc | 70/70 (70) | 0.529–0.940 | 0.115 | [−0.048, 0.055] | – |
Table D.5.
Effective pool size
Table E.1 gives the equicorrelation effective size of nominal four-member pools formed from distinct bundles, with one seed per bundle. At , task-level means range from 1.559 (Kvasir) to 2.918 (RIGA Disc).
Consider a pool of members whose per-case failure indicators at threshold share a common marginal variance and have mean pairwise Pearson correlation . The variance of the per-case mean indicator is then . Equating this with the variance of a hypothetical independent pool gives
Equation (2) is therefore exact under equal marginal variances and does not require equal pairwise correlations. With unequal variances, it summarises the variance of the mean after standardising each failure indicator. It is a threshold-specific variance summary, not a literal independent-member count. A nonpositive or numerically singular denominator makes this statistic undefined.
For the empirical covariance matrix of a pool’s binary failure indicators, the exact-covariance variant is
Equation (3) compares the variance of the pool mean with that of independent members having the observed average marginal variance. Table E.1 averages this ratio over supported pools at and , applying the same five-failure/five-pass rule and excluding zero or numerically singular mean variance. At , both variants are defined on the same supported pools. The covariance normalisation cancels in the ratio.
Table E.1 uses binary failure indicators. Unsupported pools are counted rather than imputed; each mean includes only supported pools with a defined ratio. At , task-level mean effective sizes range from 1.865 (Kvasir) to 3.001 (RIGA Disc). Many RIGA Cup compositions fail the support rule (13,440 of 84,480 supported), so is the primary threshold.
| Task | Exact covariance | Supported/total pools | Unsupported | |
| Kvasir | 1.865 | 1.864 | 53,760/84,480 | 30,720 |
| ACDC LV | 2.360 | 2.313 | 53,760/53,760 | 0 |
| BraTS FG | 2.154 | 2.178 | 53,760/53,760 | 0 |
| RIGA Cup | 2.309 | 2.317 | 13,440/84,480 | 71,040 |
| RIGA Disc | 3.001 | 3.055 | 48,384/84,480 | 36,096 |
| Task | [95% CI] | Exact covariance | Supported/total pools | Unsupported |
| Kvasir | 1.559 [1.454, 1.704] | 1.572 | 84,480/84,480 | 0 |
| ACDC LV | 1.645 [1.573, 1.728] | 1.672 | 53,760/53,760 | 0 |
| BraTS FG | 1.828 [1.714, 1.966] | 1.857 | 53,760/53,760 | 0 |
| RIGA Cup | 2.420 [2.182, 2.749] | 2.460 | 84,480/84,480 | 0 |
| RIGA Disc | 2.918 [2.626, 3.308] | 2.956 | 48,384/84,480 | 36,096 |
Table E.1. Effective size of nominal four-member diverse pools. at the fixed and empirical thresholds (mean over supported pools with a defined ratio, with a 95% bootstrap CI at ), the exact-covariance variant, supported pools over enumerated bundle-and-seed compositions, and unsupported pools; exact-covariance values are point estimates at both thresholds. The CI fixes the observed threshold and retained bundles, while rechecking support and ratio eligibility on each draw. (a) Fixed failure threshold . (b) Empirical failure threshold .
Architecture and checkpoint diagnostics
Shared ViT-family diagnostic across pretraining
Table F.1 adds patient-clustered intervals and pair counts to Table 3. On RIGA, cross-pretraining correlation is close to the cross-bundle reference. On ACDC LV, Kvasir and BraTS, it lies between that reference and within-bundle seed correlation.
All five ViT-B-class bundles (DINOv2-B, BiomedCLIP, MAE-B, DeiT-B, CLIP-B/16) pass the mean-Dice floor on every task. They share a broad architecture class but differ in pretraining, feature extraction and normalisation. Within-bundle seed correlation is computed within this five-bundle subset. We report with the case interval and, for slice-level tasks, the patient-clustered interval.
Distinct-checkpoint tier and primary-pool decomposition
Table F.2 separates same-bundle seed pairs, same-family/distinct-checkpoint pairs and cross-family pairs. In the 17-bundle expanded pool, mean per-case Dice correlation is highest for same-bundle seed pairs, followed by distinct checkpoints within a family and then cross-family pairs.
The same-bundle/release group is dominated by same-bundle, different-seed pairs. The 17-bundle pool groups distinct checkpoints by broad upstream family (DINOv2-S/B/L, OpenAI CLIP B/16, B/32 and L/14, ConvNeXt-T/S/B and ResNet-18/34/50/101 where functional). BiomedCLIP remains separate from OpenAI CLIP because its pretraining corpus, text encoder and training configuration differ. The cross-family stratum differs from the cross-bundle reference as stated in Appendix § 11. Table F.2(b) reports the corresponding statistics within the 11-bundle primary pool.
| Task | /avail. | Seed correlation | ViT cross-pretrain | [95% CI] | Pairs | |
| Kvasir | 5/5 | 0.981 [0.975, 0.985] | 0.774 [0.708, 0.826] | 0.675 | 0.207 [0.158, 0.270] | 30 / 160 |
| ACDC LV | 5/5 | 0.993 [0.992, 0.994] | 0.779 [0.752, 0.802] | 0.638 | 0.215 [0.192, 0.240] [0.187, 0.240] | 30 / 160 |
| BraTS FG | 5/5 | 0.988 [0.988, 0.989] | 0.709 [0.691, 0.728] | 0.587 | 0.279 [0.261, 0.297] [0.231, 0.337] | 30 / 160 |
| RIGA Cup | 5/5 | 0.960 [0.943, 0.972] | 0.358 [0.244, 0.473] | 0.310 | 0.601 [0.492, 0.711] | 30 / 160 |
| RIGA Disc | 5/5 | 0.971 [0.965, 0.976] | 0.242 [0.168, 0.348] | 0.230 | 0.729 [0.623, 0.804] | 30 / 160 |
Table F.1. Shared ViT-family diagnostic across pretraining. Seed correlation and ViT cross-pretraining correlation with case-bootstrap intervals. is within-bundle seed correlation minus cross-pretraining correlation, with a case interval and, for slice-level tasks, a patient-clustered interval below it. Pair counts are seed pairs / cross-pretraining pairs.
Adaptation and pipeline controls
These controls separate the same/cross Dice-correlation gap from absolute dependence.
Decoder-capacity control
Table G.1 compares the UNet-skip and frozen readouts on the same source bundles. The proportional gap reduction is largest on ACDC LV and smallest on Kvasir; the gap remains positive on every measured task.
The UNet-skip decoder replaces the readout on all 11 bundles, using frozen multi-scale features computed online, FP32 and the primary seeds, epochs and batch size. CLIP-B/16 and CLIP-L/14 use block taps from their QuickGELU wrappers. The nine-bundle rows omit these two separately wrapped encoders, and the gap reduction is similar without them; the frozen gap is recomputed on the same nine source bundles, applying the mean-Dice floor separately to each readout. For ViT bundles, the four levels are transformer-block taps (ViT-B blocks 3, 6, 9, 11, 0-indexed as in the MONAI implementation of UNETR [31]; CLIP-L/14 blocks 5, 11, 17, 23). These retain a single token resolution rather than forming a pyramid. Tap normalisation follows each wrapper (DINOv2: all taps; timm: last tap only; CLIP: none). The CNNs supply a true stride pyramid. Each of the four feature levels is projected to 128 channels by a biased convolution. Three top-down stages bilinearly resize the deeper map to the next skip resolution, add the projected skip, and apply two bias-free convolutions, each followed by BatchNorm and ReLU. After resizing to , one further Conv–BatchNorm–ReLU block and a biased classifier produce the logits. All interpolations use align_corners=False. Decoder parameter counts range from 1,157,505 to 1,558,913 across bundles; the per-bundle input channels are listed in the released decoder code (code/src/fmpool/heads_unet.py and code/src/fmpool/multiscale.py).
End-to-end fine-tuning control
Training uses encoder learning rates of or , a head learning rate of , FP32, 30 epochs, batch size 16 and seeds 42–45. Evaluation uses the final-epoch checkpoint with the primary splits and per-case Dice convention. Pair grouping follows the starting encoder bundle/release; encoder weights can diverge across seeds during fine-tuning. Encoders run in training mode: BatchNorm statistics are re-estimated for ResNet-18/50 and EfficientNet-B0, and stochastic depth is active for ConvNeXt-T and EfficientNet-B0, while training mode adds no stochastic depth to the ViT bundles. Optimisation uses Adam without weight decay and BCE loss, with a constant learning rate, no warm-up and no augmentation.
| Task | /avail. | Same bundle | Same family, distinct ckpt | Cross family | [95% CI] | , 11-bundle [95% CI] |
| Kvasir | 17/17 | 0.975 (102) | 0.736 (240) | 0.650 (1936) | 0.086 [0.047, 0.126] | 0.061 [0.003, 0.114] |
| ACDC LV | 15/17 | 0.989 (90) | 0.745 (160) | 0.613 (1520) | 0.131 [0.111, 0.148] | 0.150 [0.122, 0.176] |
| BraTS FG | 16/17 | 0.978 (96) | 0.670 (192) | 0.560 (1728) | 0.110 [0.085, 0.139] | 0.120 [0.086, 0.158] |
| RIGA Cup | 16/17 | 0.959 (96) | 0.400 (192) | 0.281 (1728) | 0.119 [0.058, 0.179] | 0.045 [, 0.126] |
| RIGA Disc | 17/17 | 0.970 (102) | 0.407 (240) | 0.194 (1936) | 0.213 [0.139, 0.280] | 0.165 [0.016, 0.281] |
| Task | , same-bundle only | , DINOv2 as cross | Distinct-ckpt (pairs) | Cross ref., excl. family | vs. cross excl. | vs. cross excl. |
| Kvasir | 0.302 | 0.300 | 0.735 (48) | 0.674 | 0.061 | 0.303 |
| ACDC LV | 0.350 | 0.346 | 0.786 (32) | 0.636 | 0.150 | 0.352 |
| BraTS FG | 0.394 | 0.391 | 0.705 (32) | 0.585 | 0.120 | 0.396 |
| RIGA Cup | 0.643 | 0.639 | 0.356 (48) | 0.311 | 0.045 | 0.642 |
| RIGA Disc | 0.736 | 0.731 | 0.390 (48) | 0.225 | 0.165 | 0.740 |
Table F.2. Table F.2: Distinct-checkpoint tier and grouping sensitivity. (a) 17-bundle pool after the floor: mean pairwise correlation (pair counts in parentheses) of same-bundle seed pairs, same-family/distinct-checkpoint pairs and cross-family pairs; is same-family minus cross-family, also restricted to the primary pool. (b) 11-bundle primary pool: with DINOv2-B×S pairs dropped from the same-bundle/release group or moved to the cross-bundle reference; distinct-checkpoint (pairs); and, against a cross reference that excludes all same-family pairs, and the same-bundle gap .
One fine-tuned head on the content-group Kvasir split (CLIP-L/14, encoder , seed 42) collapsed to near-zero Dice and remains in the pool. Excluding it gives 0.177 [0.114, 0.255] on that split. No head collapsed on the official split at either learning rate. The paired fine-tuned-minus-frozen intervals (last column of Table G.3) exclude zero on RIGA Cup and the content-group split at both learning rates. On the official split, they exclude zero at but include it at .
Fine-tuning does not uniformly raise per-bundle Dice. At an encoder learning rate of , Dice falls below the frozen-readout value for DINOv2-S and DINOv2-B on the official Kvasir split and for DINOv2-S, DINOv2-B and CLIP-L/14 on the content-group split. On RIGA Cup, Dice is lower than the frozen-readout value at an encoder learning rate of for EfficientNet-B0 (0.473 vs. 0.566 frozen).
Failure dependence under adaptation. Table G.4 applies the effective-size definition and four-distinct-bundle enumeration of Appendix E to frozen and fine-tuned traces. Each pool uses one seed per bundle. The threshold is computed from each recipe’s pooled model-case scores and held fixed during the paired case bootstrap (, seed 0). Every member must have at least five failures and five passes; unsupported pools are omitted within each draw. Recipes therefore share a threshold rule, not a numerical Dice threshold. The Kvasir difference intervals include zero at both learning rates. Cup effective size decreases at both rates despite a smaller Pearson gap.
| Task | Readout | [95% CI] | Frozen [95% CI] () | |||
| Kvasir | UNet-skip (11) | 11 | 0.851 | 0.693 | 0.158 [0.115, 0.208] | 0.265 [0.217, 0.317] () |
| ACDC LV | UNet-skip (11) | 11 | 0.908 | 0.824 | 0.084 [0.068, 0.100] | 0.317 [0.289, 0.344] () |
| RIGA Cup | UNet-skip (11) | 11 | 0.806 | 0.637 | 0.169 [0.136, 0.209] | 0.557 [0.500, 0.609] () |
| RIGA Disc | UNet-skip (11) | 11 | 0.811 | 0.559 | 0.252 [0.169, 0.339] | 0.640 [0.570, 0.689] () |
| Kvasir | UNet-skip (9) | 9 | 0.845 | 0.672 | 0.173 [0.125, 0.229] | 0.277 [0.223, 0.335] () |
| ACDC LV | UNet-skip (9) | 9 | 0.902 | 0.816 | 0.086 [0.068, 0.103] | 0.325 [0.294, 0.355] () |
| RIGA Cup | UNet-skip (9) | 9 | 0.802 | 0.631 | 0.171 [0.131, 0.219] | 0.521 [0.468, 0.576] () |
| RIGA Disc | UNet-skip (9) | 9 | 0.804 | 0.540 | 0.264 [0.174, 0.352] | 0.635 [0.552, 0.699] () |
Table G.1. Decoder-capacity controls. UNet-skip (11/9) denotes the source-bundle count; the nine-bundle pool excludes the two OpenAI CLIP bundles. The frozen reference uses the same source-bundle set and split, with the mean-Dice floor applied separately, so retained may differ on ACDC LV. Kvasir denotes the official split.
Random-initialisation and medical-encoder controls
The random-initialisation CNN control jointly trains randomly initialised ConvNeXt-T or ResNet-50 encoders and their heads on ACDC LV and RIGA Cup. Training uses FP32 Adam (learning rate for both encoder and head), BCE loss, 30 epochs and batch size 16, with eight seeds per architecture (seeds 42–49; 16 models per task). Here, same/cross compares pairs of runs using the same backbone architecture with pairs using different architectures. Its positive gap shows that this structure also occurs without pretrained weights (Table G.2). The 15-bundle frozen medical-encoder pool (Appendix § 18, Table H.1(b)) has a gap close to the primary gap on RIGA Cup and larger on RIGA Disc and Kvasir. Both controls differ from the primary analysis in pool composition and size.
nnU-Net evaluated on the same test cases
The nnU-Net v2 control in Table G.5 uses five folds and two seeds, scoring every model on identical test cases. Same-fold pairs have higher mean correlation than different-fold pairs; this fold gap, , is a different estimand from .
For RIGA Cup, nnU-Net v2 uses its self-configuring 2D configuration on the same 224 × 224 resized images and consensus cup masks (raw label 128) as the primary protocol. Training runs for 100 epochs, fewer than the default 1000 because of the compute budget, with five folds × seeds {13, 37} (10 models). Each seed fixes only network initialisation; augmentation randomness is not fixed. The seeds therefore label replicate runs, which are not bitwise reproducible.
| Task | Control | [95% CI] | |||
| Kvasir | FT, | 11 | 0.796 | 0.615 | 0.182 [0.133, 0.237] |
| Kvasir | FT, | 11 | 0.890 | 0.652 | 0.238 [0.172, 0.306] |
| Kvasir (grouped) | FT, | 11 | 0.776 | 0.601 | 0.175 [0.114, 0.250] |
| Kvasir (grouped) | FT, | 11 | 0.884 | 0.657 | 0.227 [0.142, 0.328] |
| RIGA Cup | FT, | 11 | 0.871 | 0.657 | 0.214 [0.171, 0.259] |
| RIGA Cup | FT, | 11 | 0.864 | 0.581 | 0.284 [0.225, 0.347] |
| ACDC LV | Random-init CNN | 2 | 0.909 | 0.776 | 0.133 [0.089, 0.179] |
| RIGA Cup | Random-init CNN | 2 | 0.840 | 0.523 | 0.317 [0.193, 0.430] |
Table G.2. Fine-tuning and other pool controls. FT, denotes end-to-end fine-tuning at encoder learning rate ; for the random-initialisation CNN, same/cross compares same-architecture with cross-architecture pairs. Rows show the retained bundles, correlation levels and same/cross gap. Kvasir denotes the official split unless marked grouped; paired fine-tuned-minus-frozen comparisons appear in Table G.3.
The ACDC counterpart uses the 3d_fullres configuration with 3D frame volumes as cases and nnU-Net’s SimpleITKIO geometry convention. Its five folds partition official training patients 001–100 at the patient level. Every model is scored on the same 100 frame volumes: the ED and ES frames of each of the 50 official test patients. Both tasks use 100 epochs and identical test cases across models. Test predictions (the 95 Magrabia images for RIGA Cup) use the primary per-case Dice convention: empty prediction and empty target = 1, with no smoothing. nnU-Net’s own evaluator instead returns NaN for empty/empty cases and likewise uses no smoothing term. Same-fold pairs are the two seeds of one fold; all other pairs are different-fold pairs.
Alternative failure-event metrics for stronger pipelines
Table G.6 applies , , disagreement and double-fault rates to UNet-skip Cup/Disc traces and nnU-Net Cup traces evaluated on the same test cases. This diagnostic covers only the RIGA tasks (UNet-skip on Cup and Disc; nnU-Net on Cup, its only 2D task); it is not computed for UNet-skip on Kvasir or ACDC LV. The mean-Dice floor is applied by bundle. The pooled model-case is held fixed during the paired case bootstrap (, seed 0); the convention follows Appendix D.2. UNet-skip uses the primary same-bundle/release grouping; nnU-Net compares same-fold with different-fold pairs. The and gaps support the same ordering as the Pearson gaps. At the stated threshold, same-group pairs also have more joint failures and fewer disagreements, but these two rates additionally depend on the members’ marginal failure probabilities. All reported gap intervals exclude zero. Every member must have at least five failures and five passes on the observed cohort and each resample. All resamples meet this requirement, and all pairs have finite values here.
Table G.3: Matched fine-tuning comparisons. Frozen gaps use the same source-bundle set and split; the final column gives the paired fine-tuned-minus-frozen gap with its 95% interval. FT estimates are in Table G.2.
| Task | Recipe | Frozen [95% CI] () | FT − frozen (paired) [95% CI] |
| Kvasir | FT, | 0.265 [0.217, 0.317] () | −0.083 [−0.147, −0.018] |
| Kvasir | FT, | 0.265 [0.217, 0.317] () | −0.027 [−0.102, 0.048] |
| Kvasir (grouped) | FT, | 0.323 [0.257, 0.403] () | −0.148 [−0.223, −0.071] |
| Kvasir (grouped) | FT, | 0.323 [0.257, 0.403] () | −0.096 [−0.175, −0.006] |
| RIGA Cup | FT, | 0.557 [0.500, 0.609] () | −0.344 [−0.391, −0.288] |
| RIGA Cup | FT, | 0.557 [0.500, 0.609] () | −0.274 [−0.335, −0.208] |
Table G.3.
Table G.4: Effective size under fine-tuning. Nominal four-member pools at each recipe’s pooled ; paired FT-minus-frozen intervals hold both observed thresholds fixed. Frozen marginal intervals are recomputed using these paired draws, so the RIGA Cup frozen interval differs slightly from Table E.1(b). Support counts refer to the observed cohort. Pale blue marks the paired changes.
| Task | Recipe | [95% CI] | FT − frozen [95% CI] | Supported/total pools | |
| Kvasir | Frozen | 0.569 | 1.559 [1.454, 1.704] | – | 84,480/84,480 |
| Kvasir | FT, | 0.760 | 1.630 [1.497, 1.801] | 0.071 [−0.082, 0.224] | 84,480/84,480 |
| Kvasir | FT, | 0.799 | 1.552 [1.437, 1.685] | −0.008 [−0.164, 0.141] | 84,480/84,480 |
| RIGA Cup | Frozen | 0.521 | 2.420 [2.182, 2.737] | – | 84,480/84,480 |
| RIGA Cup | FT, | 0.781 | 1.649 [1.513, 1.853] | −0.771 [−1.090, −0.508] | 84,480/84,480 |
| RIGA Cup | FT, | 0.764 | 1.747 [1.605, 1.939] | −0.673 [−0.986, −0.411] | 84,480/84,480 |
Table G.4.
Table G.5: nnU-Net evaluated on the same test cases. Both tasks use 100 epochs and 10 models (five folds, two seeds). counts test frame volumes for ACDC LV and test images for RIGA Cup. and are mean correlations within and between folds, and is their difference. uses one model per fold at training seed 13, with pooled over their case scores.
| Task | Mean Dice | [95% CI] | [95% CI] | ||
| ACDC LV | 100 | 0.942 | 0.957 0.894 | 0.063 [0.032, 0.094] | 1.244 [1.148, 1.373] |
| RIGA Cup | 95 | 0.863 | 0.825 0.758 | 0.067 [0.036, 0.119] | 1.264 [1.150, 1.417] |
Table G.5.
Table G.6: Failure-event gaps for stronger pipelines at pooled . Each cell gives the gap and its paired case-bootstrap 95% interval: same-bundle/release minus cross-bundle for UNet-skip, same-fold minus cross-fold for nnU-Net.
| Task | Pipeline | Disagreement | Double fault | ||
| RIGA Cup | UNet-skip | 0.148<br>[0.111, 0.187] | 0.097<br>[0.059, 0.150] | <br>[] | 0.025<br>[0.017, 0.034] |
| RIGA Disc | UNet-skip | 0.226<br>[0.177, 0.273] | 0.165<br>[0.101, 0.258] | <br>[] | 0.054<br>[0.041, 0.068] |
| RIGA Cup | nnU-Net | 0.099<br>[0.043, 0.159] | 0.028<br>[0.008, 0.060] | <br>[] | 0.022<br>[0.009, 0.036] |
Table G.6. Failure-event gaps for stronger pipelines at pooled . Each cell gives the gap and its paired case-bootstrap 95% interval: same-bundle/release minus cross-bundle for UNet-skip, same-fold minus cross-fold for nnU-Net.
Native-3D abdominal comparison
We evaluate released MRSegmentator folds 0 and 1 (release tag v1.2.0) and TotalSegmentator 2.4.0 (total for CT, total_mr for MRI) on AMOS22. The convenience sample comprises the first 60 labelled CT cases in sorted case-ID order and all 60 labelled MRI cases. Inference uses each system’s native 3D pipeline. Prediction and ground-truth labels are resampled with nearest-neighbour interpolation to a mm grid for volumetric scoring, so evaluation is not on the original acquisition grid.
Table G.7: Native-3D AMOS22 comparison. Two MRSegmentator folds and one modality-specific TotalSegmentator model; nominal pool size , with evaluated at pooled ; is same-system minus cross-system .
| Modality | Same-system<br> [95% CI] | Cross-system<br> [95% CI] | [95% CI] | [95% CI] | |
| CT | 60 | 0.988<br>[0.980, 0.994] | 0.976<br>[0.960, 0.987] | 0.012<br>[0.001, 0.029] | 1.071<br>[1.016, 1.154] |
| MRI | 60 | 0.975<br>[0.953, 0.987] | 0.918<br>[0.861, 0.951] | 0.057<br>[0.026, 0.105] | 1.268<br>[1.144, 1.466] |
Table G.7. Native-3D AMOS22 comparison. Two MRSegmentator folds and one modality-specific TotalSegmentator model; nominal pool size , with evaluated at pooled ; is same-system minus cross-system .
Table G.8: AMOS22 failure-event gaps at pooled . Each cell gives the same-system-minus-cross-system gap and its paired volume-bootstrap 95% interval.
| Modality | Disagreement | Double fault | ||
| CT | 0.038<br>[] | 0.004<br>[] | <br>[] | 0.008<br>[] |
| MRI | 0.143<br>[] | 0.009<br>[] | <br>[] | <br>[] |
Table G.8. AMOS22 failure-event gaps at pooled . Each cell gives the same-system-minus-cross-system gap and its paired volume-bootstrap 95% interval.
The 13 common targets are spleen, right/left kidney, gallbladder, esophagus, liver, stomach, aorta, inferior vena cava, pancreas, right/left adrenal gland and duodenum. MONAI 1.5.0 Dice with ignore_empty=True assigns undefined scores to empty ground-truth organs. Within each modality the three models share the same missingness pattern (8 undefined case-organ cells per model on CT and 6 on MRI), leaving 11–13 finite organ scores per case. The case score averages only those finite organ scores. Every case mean is finite.
The same-system statistic correlates the two MRSegmentator score traces; the cross-system statistic averages each trace’s correlation with the TotalSegmentator trace. Pearson intervals jointly resample volumes (). Failure metrics use the pooled (CT 0.798, MRI 0.748), held fixed during bootstrapping. Equation (2) applies to the three fixed members, each requiring at least five failures and passes; support is rechecked on each resample. Table G.7 reports the absolute levels, gap and effective size.
These instances represent two released systems within the nnU-Net family. MRSegmentator training partly uses TotalSegmentator-assisted labels, so the systems also share upstream lineage. The comparison illustrates that a small gap can coexist with high absolute dependence; it does not identify the cause or generalise to diverse 3D model families. At P33, all CT alternative-metric gap intervals include zero. MRI , and disagreement intervals also include zero, while the double-fault gap is negative (Table G.8). Thus, the Pearson ordering does not hold across all failure metrics. Assessing broader 3D scope first requires a larger cohort and a system outside the nnU-Net family.
Medical-domain encoders
Table H.1 summarises RETFound, Phikon, RAD-DINO and BiomedParse as a pool, Table H.2 compares each with the 11 primary bundles, and Table H.1(b) reports the 15-bundle extended pool. Mean mutual correlations among the medical encoders range from 0.136 to 0.647. Their pooled mean correlation with primary bundles is near the cross-bundle reference on RIGA Cup and slightly below it on RIGA Disc and Kvasir. Against the full cross-bundle reference, pooled is 0.047 on RIGA Cup, −0.064 on RIGA Disc and −0.058 on Kvasir (Table H.2). The extended-pool gap remains close to the primary value on RIGA Cup and increases on RIGA Disc and Kvasir. Values that involve Phikon cannot be recomputed from the release, because Owkin’s licence restricts sharing its outputs (Appendix § 20); the RETFound, RAD-DINO and BiomedParse rows of Table H.2 can.
Table H.1: Medical-encoder pooled summary. (a) Mutual : mean per-case Dice correlation over seed pairs from distinct retained medical bundles. Pooled : mean medical-to-primary correlation minus the primary cross-bundle mean (not a same/cross gap), with BiomedCLIP, the primary pool’s medically pretrained encoder, excluded from both primary sets; Table H.2 gives the full-reference difference. (b) 15-bundle extended pool ( after the floor) and 11-bundle primary gap.
| Task | Mutual [95% CI] | Pooled [95% CI] (excl. BiomedCLIP) |
| Kvasir | 0.647 [0.568, 0.720] | −0.059 [−0.113, −0.010] |
| RIGA Cup | 0.449 [0.366, 0.535] | 0.044 [−0.012, 0.096] |
| RIGA Disc | 0.136 [0.021, 0.209] | −0.065 [−0.112, −0.007] |
| Task | 15-bundle | 15-bundle | 15-bundle [95% CI] | 11-bundle [95% CI] |
| Kvasir | 0.950 | 0.649 | 0.302 [0.254, 0.357] | 0.265 [0.217, 0.317] |
| RIGA Cup | 0.889 | 0.338 | 0.551 [0.498, 0.602] | 0.557 [0.500, 0.609] |
| RIGA Disc | 0.896 | 0.197 | 0.698 [0.635, 0.741] | 0.640 [0.570, 0.689] |
Table H.1. Medical-encoder pooled summary. (a) Dependence among medical encoders and with the primary pool. (b) Extended and primary pools.
The four encoders use the same frozen FP32 protocol, with four decoder seeds and the same epochs, batch size and optimiser. RETFound is a ViT-L/16 MAE pretrained on colour fundus photographs; its CLS token is dropped and its patch tokens are reshaped to a spatial map. Phikon is pretrained on histopathology and is used with its official LayerNorm ε. RAD-DINO is pretrained on chest radiographs. BiomedParse uses a FocalNet backbone (Yang et al., [85]), whose final-stage features are used. Each added encoder is scored against the 11 primary bundles under the primary grouping. This encoder-only probe differs from the official fine-tuning or prompting pipelines. All four use 224×224 inputs, below the native input sizes of RAD-DINO and BiomedParse.
RAD-DINO is initialised from DINOv2-B, a primary-pool member, so its pairs with DINOv2-B share lineage. DINOv2-B is RAD-DINO’s most correlated primary bundle on RIGA Cup, but not on RIGA Disc or Kvasir. BiomedParse’s training corpus includes optic disc/cup and polyp segmentation datasets, as well as ACDC, MSD and ISIC-2018. Its RIGA and Kvasir tasks may therefore involve training exposure to related images. BiomedParse is not run on the ACDC, MSD or ISIC tasks, and overlap between its corpus and the test images is untested.
Table H.2: Per-encoder medical probe. to primary bundles: mean per-case Dice correlation between the encoder’s four heads and all retained primary-bundle heads; : that value minus , not a same/cross gap.
| Task | Encoder | Mean Dice | to primary bundles [95% CI] | [95% CI] | |
| Kvasir | RETFound | 0.515 | 0.597 [0.505, 0.677] | 0.675 | −0.078 [−0.164, −0.002] |
| Kvasir | Phikon | 0.616 | 0.685 [0.591, 0.759] | 0.675 | 0.010 [−0.050, 0.059] |
| Kvasir | RAD-DINO | 0.571 | 0.593 [0.488, 0.681] | 0.675 | −0.082 [−0.167, −0.007] |
| Kvasir | BiomedParse | 0.715 | 0.592 [0.487, 0.689] | 0.675 | −0.083 [−0.176, 0.002] |
| Kvasir | pooled medical | – | 0.616 [0.541, 0.681] | 0.675 | −0.058 [−0.106, −0.012] |
| RIGA Cup | RETFound | 0.740 | 0.357 [0.249, 0.459] | 0.310 | 0.048 [−0.047, 0.127] |
| RIGA Cup | Phikon | 0.673 | 0.410 [0.316, 0.497] | 0.310 | 0.100 [0.017, 0.168] |
| RIGA Cup | RAD-DINO | 0.525 | 0.339 [0.231, 0.436] | 0.310 | 0.029 [−0.061, 0.106] |
| RIGA Cup | BiomedParse | 0.743 | 0.323 [0.238, 0.408] | 0.310 | 0.013 [−0.068, 0.090] |
| RIGA Cup | pooled medical | – | 0.357 [0.284, 0.427] | 0.310 | 0.047 [−0.005, 0.096] |
| RIGA Disc | RETFound | 0.936 | 0.040 [−0.056, 0.139] | 0.230 | −0.190 [−0.302, −0.087] |
| RIGA Disc | Phikon | 0.925 | 0.280 [0.206, 0.377] | 0.230 | 0.051 [−0.010, 0.126] |
| RIGA Disc | RAD-DINO | 0.895 | 0.212 [0.159, 0.336] | 0.230 | −0.018 [−0.076, 0.096] |
| RIGA Disc | BiomedParse | 0.900 | 0.131 [0.019, 0.241] | 0.230 | −0.099 [−0.214, 0.008] |
| RIGA Disc | pooled medical | – | 0.166 [0.122, 0.229] | 0.230 | −0.064 [−0.112, −0.007] |
Table H.2. Per-encoder medical probe. to primary bundles: mean per-case Dice correlation between the encoder’s four heads and all retained primary-bundle heads; : that value minus , not a same/cross gap.
Held-out tasks and pool selection
Held-out tasks
Table I.1(a) reports four held-out tasks using the 11 bundles and four decoder seeds under the same protocol. ISIC-2018 and MSD Hippocampus yield functional pools with positive gaps; MSD Spleen is degenerate, and MSD Heart has no functional pool. The held-out tasks are 2D binary derivatives evaluated under the primary protocol, rather than official challenge evaluations. ISIC-2018 uses the official Task 1 training/test split; the MSD tasks use task-specific splits.
Each MSD volume contributes one central slice at along its third spatial axis. Intensities are min–max scaled within that slice to uint8, replicated into three channels and resized to . Masks use nearest-neighbour resizing, with as the target. Hippocampus combines its head/body labels, Heart targets the left atrium, and Spleen targets spleen foreground. Thus, counts single central slices (one per volume), not whole-volume Dice evaluations. The slice-selection rule can yield empty targets even when the volume contains foreground. The split units are:
ISIC-2018 [14, 79, 14]: image-level analysis of the official ISIC-2018 Task 1 training (2594) / test (1000) split; partial patient identifiers are available.
MSD Hippocampus [5, 74]: volume-level, not patient-level, 208/52 over the 260 training volumes: the 390 MSD volumes are left/right crops of 195 subjects and no subject mapping is released.
MSD Spleen: volume-level random split into 33 training and 8 test volumes (seed 42).
MSD Heart: volume-level random split into 16 training and 4 test volumes (seed 42).
The official ISIC Archive metadata (accessed 25 October 2026) match all 1000 test-image IDs in the released traces. Patient IDs are available for 890 test images from 711 patients and for 390 training images from 350 patients. The partitions share 48 known patient IDs, involving 53 test images; 47 of these 53 test images also share a known lesion ID with training images.2 The reported ISIC results retain the official image benchmark: bootstrap draws and calibration/evaluation splits use images. These analyses do not establish patient-disjoint generalisation, and unknown patient IDs prevent a complete overlap assessment.
The Hippocampus case bootstrap treats the 52 test volumes as independent. However, MSD releases left/right crops of the same subject without a mapping, so crops from one subject may appear in both training and test partitions or together in the test set. Patient independence is not established, and the case-bootstrap interval may understate uncertainty.
MSD Spleen passes the floor only through empty targets: 4 of its 8 test targets are empty. Of its 44 heads, 38 predict no foreground on any test slice (6 of 352 case-head predictions are non-empty). Per-case Dice is 1 for 174 model–case pairs with empty targets and 0 for 176 model–case pairs with non-empty targets. Its near-zero gap is therefore degenerate (the gap interval retains 1987/2000 bootstrap draws), and Spleen is excluded from pool selection.
MSD Heart gives no functional pool (0 of 11 bundles pass the mean-Dice floor) and is excluded from selection.
Encoder-family pool selection
Table I.1(b) scores family-selected four-member pools on the evaluation halves of seven tasks, without assuming that different encoder families supply independent members.
For each of seven tasks (the five primary tasks with their stated splits, plus ISIC-2018 [14, 79, 14] and MSD Hippocampus [5, 74]), 200 random calibration/evaluation half-splits are drawn. On each split, the four-member pool selected by encoder-family diversity on the calibration half is scored on the evaluation half at a threshold fixed from calibration. A supported split counts as low when ; unsupported splits count as not low. ACDC LV and BraTS FG are split by patient, while is computed from slice-level failure indicators within each evaluation half. ISIC uses image-level splits; images of a known patient can occur in both halves.
Details beyond the rule stated in Section 7: calibration means weight patients equally on ACDC LV and BraTS FG; a bundle passes the 0.30 mean-Dice floor on the mean of its four heads, all of which remain candidates; heads are ranked by calibration mean, each family’s best head is selected and ties are resolved lexicographically; and the threshold is the empirical 33rd percentile of the best functional head’s calibration case scores (slice scores on ACDC/BraTS), not of patient-averaged scores. Each task’s frequency interval is a two-sided 95% Clopper–Pearson interval over independently randomised splits; it describes Monte Carlo uncertainty conditional on the observed cohort, not uncertainty over new patients or datasets.
| (a) Frozen readout | |||||
| Task | /avail. | [95% CI] | |||
| Kvasir | 11/11 | 0.940 | 0.675 | 0.265 [0.217, 0.317] | 120 |
| ACDC LV | 10/11 | 0.955 | 0.638 | 0.317 [0.289, 0.344] | 977 |
| BraTS FG | 10/11 | 0.927 | 0.587 | 0.340 [0.296, 0.388] | 3365 |
| RIGA Cup | 11/11 | 0.867 | 0.310 | 0.557 [0.500, 0.609] | 95 |
| RIGA Disc | 11/11 | 0.870 | 0.230 | 0.640 [0.570, 0.689] | 95 |
| ISIC-2018 | 11/11 | 0.930 | 0.679 | 0.250 [0.226, 0.276] | 1000 |
| MSD Hippocampus | 11/11 | 0.908 | 0.441 | 0.467 [0.326, 0.648] | 52 |
| MSD Spleen | 11/11 | 0.983 | 0.979 | 0.003 [0.000, 0.007] | 8 |
| MSD Heart | 0/11 | – | – | – | 4 |
| (b) Encoder-family pool selection | |||||
| Task | Low/splits | Share (%) [95% CI] | Unsupported | Supported-only share (%) | |
| Kvasir | 200/200 | 100.0 [98.2, 100.0] | 0 | 100.0 | |
| ACDC LV | 200/200 | 100.0 [98.2, 100.0] | 0 | 100.0 | |
| BraTS FG | 200/200 | 100.0 [98.2, 100.0] | 0 | 100.0 | |
| RIGA Cup | 103/200 | 51.5 [44.3, 58.6] | 0 | 51.5 | |
| RIGA Disc | 4/200 | 2.0 [0.5, 5.0] | 1 | 2.0 | |
| ISIC-2018 | 200/200 | 100.0 [98.2, 100.0] | 0 | 100.0 | |
| MSD Hippocampus | 3/200 | 1.5 [0.3, 4.3] | 60 | 2.1 | |
| MSD Spleen | – | – | – | – | |
| MSD Heart | – | – | – | – |
Table I.1. Held-out tasks and encoder-family pool selection. (a) Frozen analysis on the five primary tasks and four held-out task derivatives. (b) Over 200 calibration/evaluation half-splits per task: the count and share of splits whose four-member pool, selected by encoder family and calibration Dice, has evaluation-half (95% Clopper–Pearson intervals over random splits, conditional on the fixed cohort); unsupported-split counts; and the share among supported splits. MSD Spleen (degenerate) and MSD Heart (no functional pool) do not enter pool selection; Appendix § 19.§ 1 gives the reasons and the image-level resampling used for ISIC.
Broader impacts and responsible AI
Societal impact. This is a retrospective analysis of evaluation evidence; it does not propose, deploy or recommend any clinical AI system. Its intended positive impact is better-informed evaluation of model pools for medical image segmentation: reporting absolute dependence and threshold-specific effective size alongside accuracy exposes shared failures that mean accuracy and nominal pool size do not reveal. The main risk is over-interpretation: low measured dependence on these retrospective cohorts could be mistaken for evidence that a pool is safe to deploy, although all statistics are conditional on the evaluated pools, cohorts, readouts and thresholds (§ 8) and are not validated for clinical deployment decisions. Because we perform no subgroup analysis, pools with low overall dependence may still share failures within patient subgroups.
Release and intended use. The release (§ 8) contains aligned per-case scores for the reported analyses, split identifiers, panel metadata, illustrative figures, scripts with reference outputs, a SHA-256 manifest and Croissant metadata, and a reproduction guide that maps commands and output columns to the tables and figure panels. It also contains the study’s preprocessing, feature-extraction, training and evaluation code.
Per-case Dice traces retain upstream de-identified subject/patient/slice IDs and deterministic hashed RIGA alignment IDs solely for case alignment and subject-level resampling. The released traces contain no machine paths, run-environment identifiers or embedded copies of source code, and no images, masks, per-pixel predictions, feature caches, checkpoints or additional demographic annotations. Users must obtain original datasets and models under their upstream terms for retraining or image-level re-evaluation.
The intended use is to recompute case-level dependence statistics from the aligned traces, inspect the aggregate evidence and reuse the protocol in evaluation research and in retrospective local redundancy checks of candidate pools (§ 8). Training and image-level analyses require the upstream datasets and models, and Phikon-derived values are withheld, as explained below. The traces are not intended for individual profiling, re-identification or clinical diagnosis; stable alignment identifiers do not guarantee anonymity.
All upstream datasets (Kvasir-SEG, ACDC, BraTS 2024 GLI, RIGA+, ISIC-2018, MSD and AMOS22) are publicly released under their respective licences and de-identification procedures. Demographic annotations are not harmonised across the evaluated datasets; we perform no subgroup or fairness analysis and claim no subgroup robustness. This analysis diagnoses redundancy in fixed pools, not fairness.
Asset licences. Table J.1 lists the terms under which the datasets are used for training and evaluation and under which the backbones, checkpoints and software are used; backbone, checkpoint and software licences follow the named repositories or model cards. Data attributions are given in Appendix § 12.§ 1, and software releases are specified in Appendices § 12.§ 2 and § 17.§ 6, Table J.1 and the repository’s pinned requirement files. BraTS image-derived examples are confined to the illustrative figures.
Authored code is provided under Apache-2.0; upstream assets and model-derived data retain their own terms. Phikon-derived outputs are subject to the Owkin licence, which restricts use and sharing to qualifying non-commercial research; they are not included in the release. Included scalar traces retain applicable attribution and use conditions, documented alongside the data.
| Asset | Licence or terms of use |
| Datasets (training and evaluation) | |
| RIGA+ | CC BY 4.0 (Zenodo record doi:10.5281/zenodo.6325549); upstream terms also apply: RIGA CC BY-NC 4.0 (Deep Blue Data) and MESSIDOR (research and educational use only; no copying or redistribution) |
| ACDC | CC BY-NC-SA 4.0; non-commercial scientific research only (licence terms distributed with the dataset) |
| Kvasir-SEG | Research and educational use only; commercial use requires prior written permission (Simula) |
| BraTS 2024 | Post-challenge CC BY-NC terms, via Synapse |
| MSD | CC BY-SA 4.0 |
| ISIC-2018 Task 1 | CC0 for the challenge bulk distribution; current Archive metadata list per-image terms |
| AMOS22 | CC BY 4.0 |
| Backbones and checkpoints | |
| DINOv2 | Apache-2.0 (code and weights) |
| CLIP ViT-B/16, ViT-B/32, ViT-L/14 | Weights MIT; open_clip 3.3.0 code MIT |
| BiomedCLIP | Checkpoint MIT |
| MAE-B | CC BY-NC 4.0 |
| DeiT-B | Apache-2.0 |
| ConvNeXt-T/S/B, EfficientNet-B0, ResNet-18/34/50/101 | torchvision IMAGENET1K_V1 weights; torchvision code BSD-3-Clause; pretrained weights may carry dataset-derived terms; original ConvNeXt code MIT, EfficientNet code Apache-2.0 |
| RETFound | Checkpoint gated; CC BY-NC 4.0 |
| Phikon | Owkin non-commercial licence |
| RAD-DINO | MIT; research use only per its model card |
| BiomedParse v1 | Weights gated; CC BY-NC-SA 4.0; code Apache-2.0; research and development use only |
| Software | |
| timm 1.0.27 | Code Apache-2.0; checkpoint-specific terms apply to pretrained weights |
| FocalNet | Code MIT |
| MONAI 1.5.0 (AMOS22 scoring) | Apache-2.0 |
| NiBabel 5.4.2 (5.3.2 for AMOS22 scoring) | Code MIT |
| nnU-Net v2.6.2 | Code Apache-2.0 (licence file) |
| MRSegmentator | Code Apache-2.0 (licence file) |
| TotalSegmentator 2.4.0 | Code Apache-2.0 (licence file); the release documentation states that the total and total_mr tasks are free for any use |
Table J.1. Licences and terms of use of third-party assets. Backbone, checkpoint and software terms follow the named repositories or model cards.
References
- [1]Taiga Abe, E. Kelly Buchanan, Geoff Pleiss, and John Patrick Cunningham. Pathologies of predictive diversity in deep ensembles. Transactions on Machine Learning Research, 2024. URL https://openreview.net/forum?id=TQfQuksaC8.arxiv.org/abs/2302.00704
- [2]Tugba Akinci D’Antonoli, Lucas K. Berger, Ashraya K. Indrakanti, Nathan Vishwanathan, Jakob Weiss, Matthias Jung, Zeynep Berkarda, Alexander Rau, Marco Reisert, Thomas Küstner, Alexandra Walter, Elmar M. Merkle, Daniel T. Boll, Hanns-Christian Breit, Andrew Phillip Nicoli, Martin Segeroth, Joshy Cyriac, Shan Yang, and Jakob Wasserthal. TotalSegmentator MRI: Robust sequence-independent segmentation of multiple anatomic structures in MRI. Radiology, 314(2):e241613, 2025. doi: 10.1148/radiol.241613.DOI
- [4]Ahmed Almazroa, Sami Alodhayb, Essameldin Osman, Eslam Ramadan, Mohammed Hummadi, Mohammed Dlaim, Muhammad Alkateе, Kaamran Raheмifar, and Vasudevan Lakshminarayanan. Retinal fundus images for glaucoma analysis: the RIGA dataset. In Medical Imaging 2018: Imaging Informatics for Healthcare, Research, and Applications, volume 10579, page 105790B. SPIE, 2018. doi: 10.1117/12.2293584.DOI
- [5]Michela Antonelli, Annika Reinke, Spyridon Bakas, Keyvan Farahani, Annette Kopp-Schneider, Bennett A. Landman, Geert Litjens, Bjoern Menze, Olaf Ronneberger, Ronald M. Summers, Bram van Ginneken, Michel Bilello, Patrick Bilic, Patrick F. Christ, Richard K. G. Do, Marc J. Gollub, Stephan H. Heckers, Henkjan Huisman, William R. Jagnin, Maureen K. McHugo, Sandy Napel, Jennifer S. Golia Pernicka, Kawal Rhode, Catalina Tobon-Gomez, Eugene Vorontsov, James A. Meakin, Sebastien Ourselin, Manuel Wiesenfarth, Pablo Arbelaez, Byeonguk Bae, Sihong Chen, Laura Daza, Jianjiang Feng, Baochun He, Fabian Isensee, Yuanfeng Ji, Fucang Jia, Ildoo Kim, Klaus Maier-Hein, Dorit Merhof, Akshay Pai, Beomhee Park, Mathias Perslev, Ramin Rezaiifar, Oliver Rippel, Ignacio Sarasua, Wei Shen, Jaemin Son, Christian Wachinger, Liansheng Wang, Yan Wang, Yingda Xia, Dagang Xu, Zhanwei Xu, Yefeng Zheng, Amber L. Simpson, Lena Maier-Hein, and M. Jorge Cardoso. The Medical Segmentation Decathlon. Nature Communications, 13 (1):4128, 2022. doi: 10.1038/s41467-022-30695-9.DOI
- [6]Anwai Archit, Luca Freckmann, and Constantin Pape. MedicoSAM: Robust improvement of SAM for medical imaging. IEEE Transactions on Medical Imaging, 45(5):1974–1985, 2026. doi: 10.1109/TMI.2025.3644811.DOI
- [7]Mohammed Baharoon, Waseem Qureshi, Jiaһong Ouyang, Yanwu Xu, Abdulrhman Aljouie, and Wei Peng. Evaluating general purpose vision foundation models for medical image analysis: An experimental study of DINOv2 on radiology benchmarks. arXiv preprint arXiv:2312.02366, 2023.
- [8]Olivier Bernard, Alain Lalande, Clement Zotti, Frederick Cervenansky, Xin Yang, Pheng-Ann Heng, Irem Cetin, Karim Lekadir, Oscar Camara, Miguel Angel Gonzalez Ballester, Gerard Sanroma, Sandy Napel, Steffen Petersen, Georgios Tziritas, Elias Grinias, Mahendra Khened, Varghese Alex Kollerathu, Ganapathy Krishnamurthi, Marc-Michel Rohé, Xavier Pennec, Maxime Sermesant, Fabian Isensee, Paul Jäger, Klaus H. Maier-Hein, Peter M. Full, Ivo Wolf, Sandy Engelhardt, Christian F. Baumgartner, Lisa M. Koch, Jelmer M. Wolterink, Ivana Išgum, Yeonggul Jang, Yoomni Hong, Jay Patravali, Shubham Jain, Olivier Humbert, and Pierre-Marc Jodoin. Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: Is the problem solved? IEEE Transactions on Medical Imaging, 37(11):2514–2525, 2018. doi: 10.1109/TMI.2018.2837502.
- [9]Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, S. Buch, Dallas Card, Rodrigo Castellon, Niladri S. Chatterji, Annie S. Chen, Kathleen A. Creel, Jared Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren E. Gillespie, Karan Goel, Noah D. Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas F. Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, O. Khattab, Pang Wei Koh, Mark S. Krass, Ranjay Krishna, Rohith Kudtipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Levent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D. Manning, Suvir P. Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Benjamin Newman, Allen Nie, Juan Carlos Niebles, Hamed Nilforoshan, J. F. Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, Aditi Raghunathan, Robert Reich, Hongyu Ren, Frieda Rong, Yusuf H. Roohani, Camilo Ruiz, Jack Ryan, Christopher Ré, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishnan Srinivasan, Alex Tamkin, Rohan Taori, Armin W. Thomas, Florian Tramèr, Rose E. Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro Yasunaga, Jiaxuan You, Matei A. Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, and Percy Liang. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021.arxiv.org/abs/2108.07258
- [10]Rishi Bommasani, Kathleen A. Creel, Ananya Kumar, Dan Jurafsky, and Percy S. Liang. Picking on the same person: Does algorithmic monoculture lead to outcome homogenization? In Advances in Neural Information Processing Systems, volume 35, pages 3663–3678. Curran Associates, Inc., 2022. doi: 10.52202/068431-0265.DOI
- [11]Gabriele Campanella, Shengjia Chen, Manbir Singh, Ruchika Verma, Silke Muehlstedt, Jennifer Zeng, Aryeh Stock, Matt Croken, Brandon Veremis, Abdulkadir Elmas, Ivan Shujski, Noora Neittaanmäki, Kuan-lin Huang, Ricky Kwan, Jane Houldsworth, Adam J. Schoenfeld, and Chad Vanderbilt. A clinical benchmark of public self-supervised pathology foundation models. Nature Communications, 16(1):3640, 2025. doi: 10.1038/s41467-025-58796-1.DOI
- [12]M. Jorge Cardoso, Wengqi Li, Richard Brown, Nic Ma, Eric Kerfoot, Yiheng Wang, Benjamin Murray, Andriy Myronenko, Can Zhao, Dong Yang, Vishwesh Nath, Yufan He, Ziyue Xu, Ali Hatamizadeh, Wentao Zhu, Yun Liu, Mingxin Zheng, Yucheng Tang, Isaac Yang, Michael Zephyr, Behrooz Hashemian, Sachidanand Alle, Mohammad Zalbagi Darestani, Charlie Budd, Marc Modat, Tom Vercauteren, Guotai Wang, Yiwen Li, Yipeng Hu, Yunguan Fu, Benjamin Gorman, Hans Johnson, Brad Genereaux, Barbaros S. Erdal, Vikash Gupta, Andres Diaz-Pinto, Andre Dourson, Lena Maier-Hein, Paul F. Jaeger, Michael Baumgartner, Jayashree Kalpathy-Cramer, Mona Flores, Justin Kirby, Lee A.D. Cooper, Holger R. Roth, Daguang Xu, David Bericat, Ralf Floca, S. Kevin Zhou, Haris Shuaib, Keyvan Farahani, Klaus H. Maier-Hein, Stephen Aylward, Prerna Dogra, Sebastien Ourselin, and Andrew Feng. MONAI: An open-source framework for deep learning in healthcare. arXiv preprint arXiv:2211.02701, 2022.arxiv.org/abs/2211.02701
- [13]Junlong Cheng, Jin Ye, Zhongying Deng, Jianpin Chen, Tianbin Li, Haoyu Wang, Yanzhou Su, Ziyan Huang, Jilong Chen, Lei Jiang, Hui Sun, Junjun He, Shaoting Zhang, Min Zhu, and Yu Qiao. SAM-Med2D. arXiv preprint arXiv:2308.16184, 2023.arxiv.org/abs/2308.16184
- [14]Noel Codella, Veronica Rotemberg, Philipp Tschandl, M. Emre Celebi, Stephen Dusza, David Gutman, Brian Helba, Aadi Kalloo, Konstantinos Liopyris, Michael Marchetti, Harald Kittler, and Allan Halpern. Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the International Skin Imaging Collaboration (ISIC). arXiv preprint arXiv:1902.03368, 2019.arxiv.org/abs/1902.03368
- [16]Edwin D. de Jong, Eric Marcus, and Jonas Teuwen. Current pathology foundation models are unrobust to medical center differences. arXiv preprint arXiv:2501.18055, 2025.
- [18]André Ferreira, Naida Solak, Jianning Li, Philipp Dammann, Jens Kleesiek, Victor Alves, and Jan Egger. How we won BraTS 2023 adult glioma challenge? Just faking it! Enhanced synthetic data augmentation and model ensemble for brain tumour segmentation. arXiv preprint arXiv:2402.17317, 2024.
- [19]Alexandre Filiot, Ridouane Ghermi, Antoine Olivier, Paul Jacob, Lucas Fidon, Axel Camara, Alice Mac Kain, Charlie Saillard, and Jean-Baptiste Schiratti. Scaling self-supervised learning for histopathology with masked image modeling. medRxiv, 2023. doi: 10.1101/2023.07.21.23292757.DOI
- [20]Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanan. Deep ensembles: A loss landscape perspective. arXiv preprint arXiv:1912.02757, 2019.arxiv.org/abs/1912.02757
- [21]Yarin Gal and Zoubin Ghahramani. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of the 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pages 1050–1059. PMLR, 2016. URL https://proceedings.mlr.press/v48/gal16.html.arxiv.org/abs/1506.02142
- [22]Yunhe Gao, Di Liu, Zhuowei Li, Yunsheng Li, Dongdong Chen, Mu Zhou, and Dimitris N. Metaxas. Show and segment: Universal medical image segmentation via in-context learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20830–20840, 2025.arxiv.org/abs/2503.19359
- [23]Robert Geirhos, Kristof Meding, and Felix A. Wichmann. Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency. In Advances in Neural Information Processing Systems, volume 33, pages 13890–13902. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/2020/file/9f6992964c3c3ea0162a056cb45fe5-Paper.pdf.arxiv.org/abs/2006.16736
- [24]Mariana-Juliana Georgescu, Radu Tudor Ionescu, and Andreea Juliana Miron. Diversity-promoting ensemble for medical image segmentation. In Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing, pages 599–606. ACM, 2023. doi: 10.1145/3555776.3577682.DOI
- [25]Shashwat Goel, Joschka Strüber, Ilze Amanda Auzina, Karuna K. Chandra, Ponnurangam Kumaraguru, Douwe Kiela, Ameya Prabhu, Matthias Bethge, and Jonas Geiping. Great models think alike and this undermines AI oversight. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 19621–19678. PMLR, 2025. URL https://proceedings.mlr.press/v267/goel25b.html.arxiv.org/abs/2502.04313
- [26]Raphael Gontijo-Lopes, Yann Dauphin, and Ekin D. Cubuk. No one representation to rule them all: Overlapping features of training methods. In International Conference on Learning Representations (ICLR), 2022. URL https://openreview.net/forum?id=BK-4qbGgIE3.
- [27]Mila Gorecki and Moritz Hardt. Monoculture or multiplicity: Which is it? In Advances in Neural Information Processing Systems, volume 38, pages 181332–181392. Curran Associates, Inc., 2025. doi: 10.52202/085713-5456.DOI
- [28]Hanxue Gu, Haoyu Dong, Jichen Yang, and Maciej A. Mazurowski. How to build the best medical image segmentation algorithm using foundation models: a comprehensive empirical study with Segment Anything Model. Machine Learning for Biomedical Imaging, pages 88–120, 2025. doi: 10.59275/j.mlba.2025-86a6.DOI
- [31]Ali Hatamizadeh, Yucheng Tang, Vishwesh Nath, Dong Yang, Andriy Myronenko, Bennett Landman, Holger R. Roth, and Daguang Xu. UNETR: Transformers for 3D medical image segmentation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 574–584, 2022. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016. doi: 10.1109/CVPR.2016.90.
- [32]Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16000–16009, 2022.arxiv.org/abs/2111.06377
- [33]Yufan He, Pengfei Guo, Yucheng Tang, Andriy Myronenko, Vishwesh Nath, Ziyue Xu, Dong Yang, Can Zhao, Benjamin Simon, Mason Belue, Stephanie Harmon, Baris Turkbey, Daguang Xu, and Wenqi Li. VISTA3D: A unified segmentation foundation model for 3D medical imaging. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20863–20873, 2025.arxiv.org/abs/2406.05285
- [34]Shishuai Hu, Zehui Liao, and Yong Xia. Domain specific convolution and high frequency reconstruction based unsupervised domain adaptation for medical image segmentation. In Medical Image Computing and Computer Assisted Intervention – MICCAI 2022, volume 13437 of Lecture Notes in Computer Science, pages 650–659. Springer Nature Switzerland, 2022a. doi: 10.1007/978-3-031-16449-1_62.DOI
- [36]Yuhao Huang, Xin Yang, Lian Liu, Han Zhou, Ao Chang, Xinrui Zhou, Rusi Chen, Junxuan Yu, Jiongquan Chen, Chaoyou Chen, Sijing Liu, Haozhe Chi, Xindi Hu, Kejuan Yue, Lei Li, Vicente Grau, Deng-Ping Fan, Fajin Dong, and Dong Ni. Segment anything model for medical images? Medical Image Analysis, 92:103061, 2024. doi: 10.1016/j.media.2023.103061.DOI
- [37]Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hanna Hajishirzi, Ali Farhadi, and Ludwig Schmidt. OpenCLIP. Zenodo, version 0.1, 2021. URL https://doi.org/10.5281/zenodo.5143773.
- [38]Fabian Isensee, Paul F. Jaeger, Simon A. A. Kohl, Jens Petersen, and Klaus H. Maier-Hein. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods, 18(2): 203–211, 2021. doi: 10.1038/s41592-020-01008-z.
- [39]Fabian Isensee, Tassilo Wald, Constantin Ulrich, Michael Baumgartner, Saikat Roy, Klaus Maier-Hein, and Paul F. Jäger. nnU-Net revisited: A call for rigorous validation in 3D medical image segmentation. In Medical Image Computing and Computer Assisted Intervention – MICCAI 2024, volume 15009 of Lecture Notes in Computer Science, pages 488–498. Springer Nature Switzerland, 2024. doi: 10.1007/978-3-031-72114-4_47.
- [40]Debesh Jha, Pia H. Smedsrud, Michael A. Riegler, Pål Halvorsen, Thomas de Lange, Dag Johansen, and Håvard D. Johansen. Kvasir-SEG: A segmented polyp dataset. In MultiMedia Modeling, volume 11962 of Lecture Notes in Computer Science, pages 451–462. Springer International Publishing, 2020. doi: 10.1007/978-3-030-73734-2_37.arxiv.org/abs/1911.07069
- [41]Debesh Jha, Sharib Ali, Nikhil Kumar Tomar, Håvard D. Johansen, Dag Johansen, Jens Rittscher, Michael A. Riegler, and Pål Halvorsen. Real-time polyp detection, localization and segmentation in colonoscopy using deep learning. IEEE Access, 9:40496–40510, 2021. doi: 10.1109/ACCESS.2021.3063716.
- [42]Wei Ji, Shuang Yu, Junde Wu, Kai Ma, Cheng Bian, Qi Bi, Jingjing Li, Hanruo Liu, Li Cheng, and Yefeng Zheng. Learning calibrated medical image segmentation via multi-rater agreement modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12341–12351, 2021.
- [44]Liwei Jiang, Yuanjun Chai, Margaret Li, Mickel Liu, Raymond Fok, Nouha Dziri, Yulia Tsvetkov, Maarten Sap, and Yejin Choi. Artificial hivemind: The open-ended homogeneity of language models (and beyond). In Advances in Neural Information Processing Systems, volume 38, pages 90705–90774. Curran Associates, Inc., 2025. doi: 10.52202/085713-2732.DOI
- [45]Nathanael Jo, Nikhil Garg, and Manish Raghavan. The subjectivity of monoculture. arXiv preprint arXiv:2602.24086, 2026.arxiv.org/abs/2602.24086
- [46]Alain Jungo and Mauricio Reyes. Assessing reliability and challenges of uncertainty estimations for medical image segmentation. In Medical Image Computing and Computer Assisted Intervention – MICCAI 2019, volume 11765 of Lecture Notes in Computer Science, pages 48–56. Springer International Publishing, 2019. doi: 10.1007/978-3-030-32245-8_6.DOI
- [47]Alexandros Karargyris, Renato Umeton, Micah J. Sheller, Alejandro Aristizabal, Johnu George, Anna Wuest, Sarthak Pati, Hasan Kassem, Maximilian Zenk, Ujjwal Baid, Prakash Narayana Moorthy, Alexander Chowdhury, Junyi Guo, Sahil Nalawade, Jacob Rosenthal, David Kanter, Maria Xenochristou, Daniel J. Beutel, Verena Chung, Timothy Bergquist, James Eddy, Abubakar Abid, Lewis Tunstall, Omar Sanseviero, Dimitrios Dimitriadis, Yiming Qian, Xinxing Xu, Yong Liu, Rick Siow Mong Goh, Srini Bala, Victor Bittorf, Srĕkar Reddy Puchala, Biagio Ricciuti, Soujanya Saminen, Eshna Sengupta, Akshay Chaudhari, Cody Coleman, Bala Desinghu, Gregory Diamos, Debo Dutta, Diane Feddema, Grigori Fursin, Xinyuan Huang, Satyananda Kashyap, Nicholas Lane, Indrani Mallick, FeTS Consortium, BraTS-2020 Consortium, AI4SafeChole Consortium, Pietro Mascagni, Virendra Mehta, Cassiano Ferro Moraes, Vivek Natarajan, Nikola Nikolov, Nicolas Padoy, Gennady Pekhimenko, Vijay Janapa Reddi, G. Anthony Reina, Pablo Ribalta, Abhishek Singh, Jayaraman J. Thiagarajan, Jacob Albrecht, Thomas Wolf, Geralyn Miller, Huazhu Fu, Prashant Shah, Daguang Xu, Poonam Yadav, David Talby, Mark M. Awad, Jeremy P. Howard, Michael Rosenthal, Luigi Marchionni, Massimo Loda, Jason M. Johnson, Spyridon Bakas, and Peter Mattson. Federated benchmarking of medical artificial intelligence with MedPerf. Nature Machine Intelligence, 5(7):799–810, 2023. doi: 10.1038/s42256-023-00652-2.DOI
- [48]Tommie Kerssies, Daan de Geus, and Gijs Dubbelman. How to benchmark vision foundation models for semantic segmentation? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 1162–1171, 2024.
- [50]Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick. Segment Anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 3992–4003, 2023. doi: 10.1109/ICCV51070.2023.00371.
- [51]Max Klabunde, Tobias Schumacher, Markus Strohmaier, and Florian Lemmerich. Similarity of neural network models: A survey of functional and representational measures. ACM Computing Surveys, 57(9):242, 2025. doi: 10.1145/3728458.DOI
- [52]Jon Kleinberg and Manish Raghavan. Algorithmic monoculture and social welfare. Proceedings of the National Academy of Sciences, 118(22):e2018340118, 2021. doi: 10.1073/pnas.2018340118.DOI
- [53]Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 3519–3529. PMLR, 2019. URL https://proceedings.mlr.press/v97/kornblith19a.html.arxiv.org/abs/1905.00414
- [54]Ludmila I. Kuncheva and Christopher J. Whitaker. Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine Learning, 51(2):181–207, 2003. doi: 10.1023/A:102859003006.DOI
- [55]Zeki Kuş and Musa Aydın. MedSegBench: A comprehensive benchmark for medical image segmentation in diverse data modalities. Scientific Data, 11(1):1283, 2024. doi: 10.1038/s41597-024-04159-2.DOI
- [56]Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems, volume 30, pages 6402–6413. Curran Associates, Inc., 2017. URL https://proceedings.neurips.cc/paper_files/paper/2017/file/9ef2ed4b7fd2c810847ffa5fa85bce38-Paper.pdf.arxiv.org/abs/1612.01474
- [57]Yuanyuan Li, Neeraj Sarna, and Yang Lin. Quantifying correlations of machine learning models. In Proceedings of the IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pages 410–417, 2025. doi: 10.1109/ICSTW64639.2025.10962482.DOI
- [58]Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A ConvNet for the 2020s. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11976–11986, 2022.
- [59]Xiangde Luo, Xiyue Wang, Feyisopе Eweje, Xiaoming Zhang, Juan Luis Gomez Marti, Sarah Cascarino, Sen Yang, Yuchen Li, Ryan Quinton, Jinxi Xiang, Yuanfeng Ji, Zhe Li, Yijian Chen, Colin Bergstrom, Cedric Kim, Francesca Maria Oliquin, Kelley Yuan, Matthew Abikenari, Andrew Heider, Sierra Willens, Sanjeeth Rajaram, Robert West, Joel Neal, Adam Schoenfeld, Maximilian Diehn, Chad Vanderbilt, and Ruijiang L. Ensemble learning of pathology foundation models for precision oncology. Cancer Cell, 44(10):1981–1996.e8, 2026. doi: 10.1016/j.ccell.2026.08.008.DOI
- [60]Jun Ma, Yuting He, Feifei Li, Lin Han, Chenyu You, and Bo Wang. Segment anything in medical images. Nature Communications, 15(1):654, 2024. doi: 10.1038/s41467-024-44824-z.DOI
- [61]Jun Ma, Zongxin Yang, Sumin Kim, Bihui Chen, Mohammed Baharoon, Adibvafa Fallahpour, Reza Asakereh, Hongwei Lyu, and Bo Wang. MedSAM2: Segment anything in 3D medical images and videos. arXiv preprint arXiv:2504.03600, 2025.arxiv.org/abs/2504.03600
- [62]Lena Maier-Hein, Annika Reinke, Patrick Godau, Minu D. Tizabi, Florian Buettner, Evangelia Christodoulou, Ben Glocker, Fabian Isensee, Jens Kleesiek, Michal Kozubek, Mauricio Reyes, Michael A. Riegler, Manuel Wiesenfarth, A. Emre Kavur, Carole H. Sudre, Michael Baumgartner, Matthias Eisenmann, Doreen Heckmann-Nötzel, Tim Rä̈dsch, Laura Acion, Michela Antonelli, Tal Arbel, Spyridon Bakas, Arriel Benis, Matthew B. Blaschko, M. Jorge Cardoso, Veronika Cheplygina, Beth A. Cimini, Gary S. Collins, Keyvan Farahani, Luciana Ferrer, Adrian Galdran, Bram van Ginneken, Robert Haase, Daniel A. Hashimoto, Michael M. Hoffman, Merel Huisman, Pierre Jannin, Charles E. Kahn, Dagmar Kainmueller, Bernhard Kainz, Alexandros Karargyris, Alan Karthikesalingam, Florian Kofler, Annette Kopp-Schneider, Anna Kreshuk, Tahsin Kurc, Bennett A. Landman, Geert Litjens, Amin Madani, Klaus Maier-Hein, Anne L. Martel, Peter Mattson, Erik Meijering, Bjoern Menze, Karel G. M. Moons, Henning Müller, Brennan Nichyporuk, Felix Nickel, Jens Petersen, Nasir Rajpoot, Nicola Rieke, Julio Saez-Rodriguez, Clara I. Sánchez, Shravya Shetty, Maarten van Smeden, Ronald M. Summers, Abdel A. Taha, Aleksei Tiulpin, Sotirios A. Tsaftaris, Ben Van Calster, Gaël Varoquaux, and Paul F. Jäger. Metrics reloaded: recommendations for image analysis validation. Nature Methods, 21(2):195–212, 2024. doi: 10.1038/s41592-023-02151-z.
- [63]Kenneth O. McGraw and S. P. Wong. Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1):30–46, 1996. doi: 10.1037/1082-989X.1.1.30.DOI
- [64]Alireza Mehrtash, William M. Wells, Clare M. Tempany, Purang Abolmaesumi, and Tina Kapur. Confidence calibration and predictive uncertainty estimation for deep medical image segmentation. IEEE Transactions on Medical Imaging, 39(12):3868–3878, 2020. doi: 10.1109/TMI.2020.3006437.DOI
- [66]Basil Mustafa, Carlos Riquelme, Joan Puigcerver, André Susano Pinto, Daniel Keysers, and Neil Houlsby. Deep ensembles for low-data transfer learning. arXiv preprint arXiv:2010.06866, 2020.arxiv.org/abs/2010.06866
- [67]Peter Neidlinger, Omar S. M. El Nahhas, Hannah Sophie Muti, Tim Lenz, Michael Hoffmeister, Hermann Brenner, Marko van Treeck, Rupert Langer, Bastian Dislich, Hans Michael Behrens, Christoph Röcken, Sebastian Foersch, Daniel Truhn, Antonio Marra, Oliver Lester Saldanha, and Jakob Nikolas Kather. Benchmarking foundation models as feature extractors for weakly supervised computational pathology. Nature Biomedical Engineering, 10(6):1113–1123, 2026. doi: 10.1038/s41551-025-01516-3.DOI
- [68]Behnam Neyshabur, Hanie Sedghi, and Chiyuan Zhang. What is being transferred in transfer learning? In Advances in Neural Information Processing Systems, volume 33, pages 512–523. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/2020/file/0607f4c705595b911a4f3e7a127b44e0-Paper.pdf.arxiv.org/abs/2008.11687
- [69]Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaeldin El-Nouby, Mahmoud Assran, Nicolas Balhas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jégou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. DINOv2: Learning robust visual features without supervision. Transactions on Machine Learning Research, 2024. URL https://openreview.net/forum?id=a68SUt6zFt.arxiv.org/abs/2304.07193
- [70]Fernando Pérez-García, Harshita Sharma, Sam Bond-Taylor, Kenza Bouzid, Valentina Salvatelli, Maximilian Ilse, Shruthi Bannur, Daniel C. Castro, Anton Schwaighofer, Matthew P. Lungren, Maria Teodora Wetscherеk, Noel Codella, Stephanie L. Hyland, Javier Alvarez-Valle, and Ozan Oktay. Exploring scalable medical image encoders beyond text supervision. Nature Machine Intelligence, 7(1):119–130, 2025. doi: 10.1038/s42256-024-00965-w.
- [71]Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 8748–8763. PMLR, 2021. URL https://proceedings.mlr.press/v139/radford21a.html.arxiv.org/abs/2103.00020
- [72]Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, volume 9351 of Lecture Notes in Computer Science, pages 234–241. Springer International Publishing, 2015. doi: 10.1007/978-3-319-24574-4_28.DOI
- [73]Abhijit Guha Roy, Sailesh Conjeti, Nassir Navab, and Christian Wachinger. Bayesian QuickNAT: Model uncertainty in deep whole-brain segmentation for structure-wise quality control. NeuroImage, 195:11–22, 2019. doi: 10.1016/j.neuroimage.2019.03.042.DOI
- [74]Amber L. Simpson, Michela Antonelli, Svoridon Bakas, Michel Bilello, Keyvan Farahani, Bram van Ginneken, Annette Kopp-Schneider, Bennett A. Landman, Geert Litjens, Bjoern Menze, Olaf Ronneberger, Ronald M. Summers, Patrick Bilic, Patrick F. Christ, Richard K. G. Do, Marc Gollub, Jennifer Golia-Pernicka, Stephan H. Heckers, William R. Jarnagin, Maureen K. McHugo, Sandy Napel, Eugene Vorontsov, Lena Maier-Hein, and M. Jorge Cardoso. A large annotated medical image dataset for the development and evaluation of segmentation algorithms. arXiv preprint arXiv:1902.09063, 2019.
- [75]Mingxing Tan and Quoc Le. EfficientNet: Rethinking model scaling for convolutional neural networks. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 6105–6114. PMLR, 2019. URL https://proceedings.mlr.press/v97/tan19a.html.arxiv.org/abs/1905.11946
- [76]TorchVision maintainers and contributors. TorchVision: PyTorch’s computer vision library. GitHub repository, 2016. URL https://github.com/pytorch/vision.
- [77]Connor Toups, Rishi Bommasani, Kathleen Creel, Sarah Bana, Dan Jurafsky, and Percy S. Liang. Ecosystem-level analysis of deployed machine learning reveals homogeneous outcomes. In Advances in Neural Information Processing Systems, volume 36, pages 51178–51201. Curran Associates, Inc., 2023. doi: 10.52202/075280-2228.DOI
- [78]Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 10347–10357. PMLR, 2021. URL https://proceedings.mlr.press/v139/touvron21a.html.arxiv.org/abs/2012.12877
- [79]Philipp Tschandl, Cliff Rosendahl, and Harald Kittler. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data, 5(1):180161, 2018. doi: 10.1038/sdata.2018.161.
- [80]Jakob Wasserthal, Hanns-Christian Breit, Manfred T. Meyer, Maurice Pradella, Daniel Hinck, Alexander W. Sauter, Tobias Heye, Daniel T. Boll, Joshy Cyriac, Shan Yang, Michael Bach, and Martin S. Segeroth. TotalSegmentator: Robust segmentation of 104 anatomic structures in CT images. Radiology: Artificial Intelligence, 5 (5):e230024, 2023. doi: 10.1148/ryai.230024.
- [81]Ross Wightman. PyTorch Image Models. Zenodo, 2019. URL https://doi.org/10.5281/zenodo.4414861.
- [82]Danny Wood, Tingting Mu, Andrew M. Webb, Henry W. J. Reeve, Mikel Luján, and Gavin Brown. A unified theory of diversity in ensemble learning. Journal of Machine Learning Research, 24(359):1–49, 2023. URL http://jmlr.org/papers/v24/23-0041.html.arxiv.org/abs/2301.03962
- [83]Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S. Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 23965–23998. PMLR, 2022. URL https://proceedings.mlr.press/v162/wortsman22a.html.
- [84]R. Patrick Xian, Noah R. Baker, Tom David, Qiming Cui, A. Jay Holmgren, Stefan Bauer, Madhumita Sushil, and Reza Abbasi-Asl. Robustness tests for biomedical foundation models should tailor to specifications. npj Digital Medicine, 8(1):557, 2025. doi: 10.1038/s41746-025-01926-2.DOI
- [85]Jianwei Yang, Chunyuan Li, Xiyang Dai, and Jianfeng Gao. Focal modulation networks. In Advances in Neural Information Processing Systems, volume 35, pages 4203–4217. Curran Associates, Inc., 2022. doi: 10.52202/068431-0304.DOI
- [86]Yuncheng Yang, Meng Wei, Junjun He, Jie Yang, Jin Ye, and Yun Gu. Pick the best pre-trained model: Towards transferability estimation for medical image segmentation. In Medical Image Computing and Computer Assisted Intervention – MICCAI 2023, volume 14220 of Lecture Notes in Computer Science, pages 674–683. Springer Nature Switzerland, 2023. doi: 10.1007/978-3-031-43907-0_64.DOI
- [87]Maximilian Zenk, David Zimmerer, Fabian Isensee, Jeremias Traub, Tobias Norajitra, Paul F. Jäger, and Klaus Maier-Hein. Comparative benchmarking of failure detection methods in medical image segmentation: Unveiling the role of confidence aggregation. Medical Image Analysis, 101:103392, 2025. doi: 10.1016/j.media.2024.103392.DOI
- [88]Sheng Zhang, Yanbo Xu, Naoto Usuyama, Hanwen Xu, Jaspreet Bagga, Robert Tinn, Sam Preston, Rajesh Rao, Mu Wei, Naeven Valluri, Cliff Wong, Andrea Tupini, Yu Wang, Matt Mazzola, Swadheen Shukla, Lars Liden, Jianfeng Gao, Angela Crabtree, Brian Piening, Carlo Bifulco, Matthew P. Lungren, Tristan Naumann, Sheng Wang, and Hoifung Poon. A multimodal biomedical foundation model trained from fifteen million image–text pairs. NEJM AI, 2(1), 2025. doi: 10.1056/AIoa2400640.DOI
- [89]Theodore Zhao, Yu Gu, Jianwei Yang, Naoto Usuyama, Ho Hin Lee, Sid Kiblawi, Tristan Naumann, Jianfeng Gao, Angela Crabtree, Jacob Abel, Christine Moung-Wen, Brian Piening, Carlo Bifulco, Mu Wei, Hoifung Poon, and Sheng Wang. A foundation model for joint segmentation, detection and recognition of biomedical objects across nine modalities. Nature Methods, 22(1):166–176, 2025. doi: 10.1038/s41592-024-02499-w.DOI
- [90]Yukun Zhou, Mark A. Chia, Siegfried K. Wagner, Murat S. Ayhan, Dominic J. Williamson, Robert R. Struyven, Timing Liu, Moucheng Xu, Mateo G. Lozano, Peter Woodward-Court, Yuka Kihara, UK Biobank Eye & Vision Consortium, Naomi Allen, John E. J. Gallacher, Thomas Littlejohns, Tariq Aslam, Paul Bishop, Graeme Black, Panagiotis Sergouniotis, Deniz Atan, Andrew D. Dick, Cathy Williams, Sarah Barman, Jenny H. Barrett, Sarah Mackie, Tasanee Braithwaite, Roxana O. Carare, Sarah Ennis, Jane Gibson, Andrew J. Lotery, Jay Self, Usha Chakravarthy, Ruth E. Hogg, Euan Paterson, Jayne Woodside, Tunde Peto, Gareth McKay, Bernadette McGuinness, Paul J. Foster, Konstantinos Balaskas, Anthony P. Khawaja, Nikolas Pontikos, Jugnoo S. Rahi, Gerassimos Lascaratos, Praveen J. Patel, Michelle Chan, Sharon Y. L. Chua, Alexander Day, Parul Desai, Cathy Egan, Marcus Fruttiger, David F. Garway-Heath, Alison Hardcastle, Peng T. Khaw, Tony Moore, Sobha Sivaprasad, Nicholas Strouthidis, Dhanes Thomas, Adnan Tufail, Ananth C. Viswanathan, Bal Dhillon, Tom Macgillivray, Cathie Sudlow, Veronique Vitart, Alexander Doney, Emanuele Trucco, Jeremy A. Guggenheim, James E. Morgan, Chris J. Hammond, Katie Williams, Pirro Hysi, Simon P. Harding, Yalin Zheng, Robert Luben, Phil Mather, Zihan Sun, Martin McKibbin, Eoin O’Sullivan, Richard Oram, Mike Weedon, Chris G. Owen, Alicja R. Rudnicka, Naveed Sattar, David Steel, Irene Stratton, Robyn Tapp, Max M. Yates, Axel Petzold, Savita Madhusudhan, Andre Altmann, Aaron Y. Lee, Eric J. Topol, Alastair K. Denniston, Daniel C. Alexander, and Pearse A. Keane. A foundation model for generalizable disease detection from retinal images. Nature, 622(7981):156–163, 2023. doi: 10.1038/s41586-023-06555-x.DOI
- [91]Jiayuan Zhu, Abdullah Hamdi, Yunli Qi, Yueming Jin, and Junde Wu. Medical SAM 2: Segment medical images as video via Segment Anything Model 2. arXiv preprint arXiv:2408.00874, 2024.arxiv.org/abs/2408.00874







