betaFeedback

WorksComputer Science

ReuScope: Auditing Data Reuse and Evaluation Sensitivity in Capsule Vision 2024

Abstract

We develop ReuScope, a protocol for auditing data reuse and evaluation sensitivity, using Capsule Vision 2024 as a case study. It combines measurements of data relationships with two comparisons: changing the evaluation population while holding predictions fixed, and changing training data while holding evaluation fixed. Byte-identical files cross the public training-validation boundary, and all 41 Kvasir-Capsule recordings that contribute frames occur in both partitions. Shared source recordings establish cross-split overlap; missing cross-recording patient identifiers prevent counting unique shared patients. Across ten paired seeds, removing all training rows attributed to Kvasir-Capsule, plus five rows from other sources flagged as near-matches to it, lowers public-validation balanced accuracy from 82.5% to 61.2%. Evaluating the same fitted models on the validation rows of the other sources reverses the comparison: removal improves balanced accuracy by 1.38 percentage points. The public-validation drop measures sensitivity to source removal, not an amount of leakage-induced inflation. Matched controls show that sample size and class counts alone do not explain the observed score differences. An exploratory experiment removes the training neighbors of validation frames that have close training matches, or equally many distant training rows, and yields a difference-in-differences estimate, relative to matched unflagged frames, of -0.355 percentage points in true-class probability; the paired-t interval for accuracy spans zero. The resulting procedure distinguishes observed reuse from conditional performance sensitivity and specifies the population, class support and uncertainty unit for each conclusion.

Introduction

A new filename does not imply new image content, a new source video or a new patient. In capsule endoscopy, neighboring frames can depict nearly the same scene, and identical files can occur under different names or labels. These relationships matter when public-validation scores are interpreted as evidence of generalization. Detecting reuse and identifying its effect on a score are nevertheless different tasks [1, 2]. Removing a source can reduce such reuse, but it also removes useful content and changes class support (rows per class), sample size and update count.

Capsule Vision 2024 (CV2024) was designed to support clinical frame review through ten-class abnormality classification and to encourage vendor-independent models [3]. Its public release combines four acquisition sources and exhibits exact file reuse, shared videos, disagreement between released labels and source-dependent performance. The public validation split also matters beyond the challenge: EndoExtend24 reports organizer-evaluated CV2024 public-validation results alongside official-test results [4]. We ask whether the score change from removing a source keeps its direction across evaluation populations, and which matched controls are needed to interpret it.

The case reveals three challenges for auditing: different tests detect different data relationships; training contrasts can reverse across evaluation populations; and narrower interventions support narrower conclusions, each with its own population and uncertainty. We contribute the empirical CV2024 audit and ReuScope, the comparison and reporting protocol derived from its findings. Section 2 defines the comparisons, and Section 3 presents a public-file census, fixed-prediction diagnostics, matched training controls and a local-removal experiment. Section 4 specifies the protocol’s comparison rules and reporting requirements.

Relationship to prior work. Leakage taxonomies and model information sheets relate data dependence to the population supporting a claim [1]; REFORMS extends this discipline to study design, uncertainty and reporting [5]. Visual audits characterize leakage by modality, coverage and degree [6], while ciFAIR connects detection, adjudication, test-set replacement and reevaluation [7]. Medical-image studies benchmark duplicate detection [8] and document repeated content and lost provenance across derivative datasets [9]. In endoscopy, EndoExtend24 propagates its revised 80/20 CV2024 split assignments to overlapping source images by filename [4]; content and video-prefix checks reveal relationships that filename matching alone cannot establish. Our audit makes matching rules, training changes and evaluation populations explicit, separating observed reuse from the score response to a specified change.

Case study and comparison design

Release and evaluation boundary. CV2024 contains 37,607 public training and 16,132 validation rows. The organizers describe a random 70–30 split and duplicate exclusion. They designate public validation as internal assessment and use a separate 4,385-image AIIMS test for external assessment and final ranking. Test images were available for prediction, but their labels were withheld until after evaluation. Final-test models could be trained on both public partitions [10 b] (Handa et al. 2024). We primarily audit the public training–validation boundary and analyze the official test separately.

By the organizers’ source labels, validation contains 11,581 KVASIR (frames from Kvasir-Capsule), 4,289 SEE-AI, 165 KID and 97 AIIMS rows; KVASIR contributes 71.8% of rows. Source totals match the challenge paper’s Table I; the other sources are the SEE-AI project [12], KID [13] and an AIIMS collection [14]. Forty-one of the 43 labeled Kvasir-Capsule reference videos contribute to CV2024 [15]. Their filename prefixes identify source recordings; patient identity across recordings is unavailable. We preserve full paths, labels and row multiplicities throughout.

Two comparisons. Figure 1 illustrates why training and evaluation changes must be separated. Let A(T;s)\mathcal{A}(T;s) denote the model produced by a specified training pipeline on list TT with seed ss, and let MC(f,V)\mathcal{M}_{\mathcal{C}}(f,V) evaluate model ff on list VV. The metric definition C\mathcal{C} fixes the included classes and their aggregation. We distinguish

Overview of data relationships and training comparisons across evaluation views

Figure 1. Overview of data relationships and training comparisons across evaluation views in ReuScope. The protocol audits file identity, shared videos and similarity candidates across data splits. Paired Full and Removal models are evaluated on three validation populations, separating training changes from evaluation changes. The resulting performance contrasts demonstrate evaluation-population sensitivity rather than establishing score inflation from data reuse.

Δview(f)=MC(f,V1)−MC(f,V0),(1)\Delta_{\mathrm{view}}(f)=\mathcal{M}_{\mathcal{C}}(f,V_1)-\mathcal{M}_{\mathcal{C}}(f,V_0), \tag*{(1)}
Δtrain(V)=Es[MC(A(T1;s),V)−MC(A(T0;s),V)].(2)\Delta_{\mathrm{train}}(V)=\mathbb{E}_s[\mathcal{M}_{\mathcal{C}}(\mathcal{A}(T_1;s),V)-\mathcal{M}_{\mathcal{C}}(\mathcal{A}(T_0;s),V)]. \tag*{(2)}

The first changes evaluation while holding the fitted model fixed; the second, a training contrast, changes training while holding evaluation fixed and pairing seeds. We call a training list a pool and an evaluation list a view; evaluating models trained on each pool across all views reveals how the training contrast depends on the evaluation population.

Relationship measurements. We compute SHA-256 for every released public image and compare the bytes within each matching group. We also use 64-bit pHash and dHash descriptors [16]; dpd_p and ddd_d denote their Hamming distances. The hash screen against Kvasir-Capsule flags an image whose minimum dpd_p and minimum ddd_d over all reference images are both ≤6\leq6; the two minima may come from different references. The joint rule instead requires a single matching image with dp≤6d_p \leq6 and dd≤6d_d \leq6, and the sum rule one with dp+dd≤6d_p + d_d \leq6. We measure pixel similarity with normalized cross-correlation (NCC) and structural similarity (SSIM) [17], after direct resizing to 96×9696 \times96 grayscale. The thresholds specify screening rules, not calibrated duplicate classifiers.

Training controls and uncertainty. The primary pipeline uses DINOv2-L/14 [18] with a 256-unit nonlinear head and rank-eight LoRA [19] on the final four blocks. The primary conditions use ten paired seeds, train for ten epochs and report the final epoch. The head trains throughout; LoRA updates begin after three head-only epochs. We resize images directly to 224×224224 \times224, apply ImageNet normalization and use batch size 128. AdamW retains its state across epochs, with head/LoRA learning rates 3×10−4/10−43 \times10^{-4}/10^{-4} and weight decay 10−410^{-4}. Class-weighted cross-entropy uses label smoothing 0.05, and the backbone remains in evaluation mode.

We define six main pools. Full uses all 37,607 rows. Source-only retains only non-KVASIR training data (10,596 rows); Removal further excludes five other-source rows passing the hash screen against Kvasir-Capsule, leaving 10,591. Random is a fixed 10,596-row sample from Full that matches Source-only’s size and nominal update count. Comp (class-count-matched) matches Source-only’s class counts; Class-match-R matches Removal’s counts at 10,591 rows. Both are drawn from Full, taking KVASIR rows first within each class. Dropping the final incomplete batch leaves 293 minibatches per epoch for Full and 82 for each reduced pool; equal epochs therefore do not imply equal update counts. Paired intervals quantify variation across seeds conditional on fixed lists and pipeline, not patient sampling; each table states whether ±\pm denotes a sample SD, an SE or the paired-tt 95% half-width h95h_{95}. The records only partly document how lists and hyperparameters were chosen. Appendix C gives the optimizer settings and ordered pool-construction rules.

Evaluation populations. We evaluate trained models on public validation (16,132 rows), non-KVASIR validation (4,551) and internal-filtered validation (4,326). The last view removes 225 rows (221 unique basenames) from non-KVASIR validation. A row is removed when its basename (filename without folder) or exact pHash matches a same-source training row; no same-class, dHash or NCC condition is required, and extending the removal to other rows sharing these basenames adds none. All ten classes occur in each view. Balanced accuracy (BA) is mean class recall; the organizer score averages class-macro AUC and BA.

What the case reveals

File identity and video overlap answer different questions

Byte-identical files cross the public training–validation boundary in 216 groups: 210 SEE-AI groups involving 215 validation files, and six AIIMS groups involving six files. A filename-only check would miss 45 of them, 39 SEE-AI groups and all six AIIMS groups, because their basenames differ across the boundary. No byte-identical files cross this boundary within KVASIR or KID, or across source labels (Figure 2). Group counts differ from counts of affected rows; Table 4 in Appendix § 7.§ 1 gives the complete census by source.

Two-panel chart showing file identity and video sharing across sources and KVASIR video prefixes

Figure 2. File identity and video sharing across the public training–validation boundary. (a) Byte-identity groups by source and released label relationship. The SEE-AI groups involve 215 validation files; the AIIMS groups involve six. (b) Training and validation row shares within each of the 41 contributing KVASIR video prefixes, sorted by identifier and normalized separately for each prefix. Every prefix occurs in both partitions despite the absence of byte-identical KVASIR crossings. The dashed line marks the announced 30% validation share.

SourceValidation rowsGroupsValidation filesRateLabels intersectLabels disjointRenamed groups
KVASIR11,581000.00%000
SEE-AI4,2892102155.01%3317739
KID165000.00%000
AIIMS97666.19%606

Table 4. Byte-identical public train–validation crossings.

Identical files need not have matching labels: 177 SEE-AI groups have disjoint released label sets, while 33 SEE-AI groups and all six AIIMS groups share a released label. KVASIR illustrates a different distinction: no byte-identical files cross the boundary, yet all 41 contributing video prefixes occur in both partitions. For each KVASIR validation row, we find the nearest training frame from the same video. The frame-index gap has median one and interquartile range 1–1. File identity, label relationships and video sharing thus provide complementary evidence of cross-split dependence.

The separate official-test byte scan finds zero matches against public training, one label-disjoint match against validation and eight within-test groups. Final-test training could use both public partitions, so the absence of training-folder matches does not cover every permitted final-training pool or establish which participants used matching files for training.

The counterpart rule changes the measured relationship

Exact-pHash matching links 1,381 of 11,581 KVASIR validation rows (11.92%) to same-source training rows in 6,285 pairs. Pairing each validation row with its lexicographically first matching training row, its counterpart, yields 540 rows with NCC ≥0.99\ge0.99; accepting any matching training row as the counterpart yields 799. The rates among all KVASIR validation rows are 4.66% and 6.90%. With the threshold fixed, changing the counterpart rule changes the question and numerator; the larger count does not establish higher detector recall. On the same 1,381 selected pairs, 256×256256 \times256 bilinear grayscale processing gives 387 NCC-positive pairs, compared with 540 at 96×9696 \times96 bicubic. Resolution, interpolation and the NCC normalization change together (Appendix § 7); the 799-row count uses the separate any-counterpart rule.

Same-modality controls put these similarity scores in context. The 1,381 selected exact-pHash pairs have median SSIM 0.880, compared with 0.647 for unflagged adjacent pairs from the same video. Joint video-prefix resampling gives a 95% interval of [0.186, 0.285] for the difference between these SSIM medians. Yet 68 adjacent comparison pairs also reach NCC 0.99, and their duplicate status is unknown. The 11.92% exact-pHash match rate is a census of all KVASIR validation rows, not a sample estimate. Resampling ten of the 41 prefixes per draw gives a [4.39%, 24.82%] percentile range that describes sensitivity to hypothetically sampling fewer prefixes, not uncertainty in the census or patient membership.

Source-to-reference screening compares CV2024 with upstream Kvasir-Capsule, a different boundary, using the hash screen. Pixel measurements use a pHash-minimizing counterpart, with ties broken by a fixed enumeration order. Among 38,592 selected KVASIR/reference pairs, 92.31% reach NCC 0.99; mean DINOv2 cosine similarity is 0.9658, compared with 0.5376 for random pairs. At the same NCC 0.99 cutoff, only 36.17% pass when the larger of the selected pair’s pHash and dHash distances is 3–6. The joint rule misses all tested 90% center crops in a 50-frame check (Appendix § 7).

Matched controls separate sample count from source composition

Table 1 summarizes the main comparisons, their uncertainty and evaluation populations. The central result is a sign reversal: Removal-minus-Full BA is −0.2132-0.2132 on public validation (95% pairedFiltered-view contrast: Removal−Full Δ\DeltaBA = +.0051±.0094+.0051 \pm.0094.

Poolntrainn_{\mathrm{train}}PublicNon-KVFilt.
(a) Training pools (DINOv2, BA)
Full37,607.825.745.773
Random10,596.779.695.707
Comp10,596.734.388.401
Class-match-R10,591.738.391.406
Removal10,591.612.759.778
Source-only10,596.611.757.776
ContrastPublicNon-KV
(b) Training contrasts (Δ\DeltaBA)
Rem.−Full−.2132±.0068-.2132 \pm.0068+.0138±.0096+.0138 \pm.0096
S−R−.1682±.0072-.1682 \pm.0072+.0624±.0108+.0624 \pm.0108
A−Comp−.0690±.0062-.0690 \pm.0062+.0978±.0079+.0978 \pm.0079
R50: S−R−.2002±.0023-.2002 \pm.0023+.0878±.0042+.0878 \pm.0042
CNX: S−R−.2096±.0041-.2096 \pm.0041+.0955±.0035+.0955 \pm.0035
PoolBAAUC
(c) Official test (μ±SE\mu\pm\mathrm{SE})
Full.2804±.0046.2804 \pm.0046.7177±.0015.7177 \pm.0015
Random.2786±.0038.2786 \pm.0038.7352±.0036.7352 \pm.0036
Source-only.2688±.0034.2688 \pm.0034.7533±.0013.7533 \pm.0013
Cohort / aggregationNon-KVFiltered
(d) Fixed predictions: Δ\Deltacombined score
Pooled 24−.0503-.0503−.0393-.0393
Equal-source 24−.0199-.0199−.0163-.0163
Pooled 19−.0624-.0624−.0488-.0488
Equal-source 19−.0220-.0220−.0177-.0177
EndpointΔ±h95\Delta\pm h_{95}Signs
(e) Local−distant, exposed−unflagged
True-class prob.−.00355±.00131-.00355 \pm.0013110/0/0
BA (6 classes)−.00194±.00291-.00194 \pm.002914/6/0
Accuracy−.00079±.00084-.00079 \pm.000844/6/0

Table 1. Training and evaluation sensitivity. Public/Non-KV/Filtered: 16,132/4,551/4,326 rows, ten classes each. Non-KV excludes KVASIR; Filtered additionally excludes internal-match candidates (definitions in § 2). BA: mean class recall; AUC: macro one-versus-rest.

Blue marks key comparisons, not significance. Absolute units; ten seeds in (a–c,e). (b,e) and footer report mean ±\pm paired-tt 95% half-width h95h_{95} (not SE), conditional on fixed lists. First three rows in (b): DINOv2-L/14; R50/CNX: frozen ResNet-50/ConvNeXt-Tiny. Rem.: Removal; S/R: Source-only/Random (the R of Class-match-R denotes Removal); A: 66 Ulcer KVASIR-to-AIIMS replacements. (c) Separate experiment evaluated on the 4,385-row, ten-class official test; mean ±\pm SE. (d) Combined-score change from the Public view under the same aggregation: pooled (BA + AUC)/2, or equal-source averages with represented-class support. Descriptive; entries may be related: 24 score-consistent entries (excluding only Team G) are primary; 19 are a fixed subset sensitivity (Appendix § 8.§ 1). (e) Exploratory exposed-minus-unflagged, local-minus-distant contrast on 588 exposed and 588 matched-unflagged rows; signs: negative/zero/positive seeds (§ 3.5).

tt interval [−0.2200,−0.2065][-0.2200,-0.2065]) but +0.0138+0.0138 ([+0.0043,+0.0234+0.0043,+0.0234]) on non-KVASIR. On internal-filtered validation, the contrast is +0.0051+0.0051 and its interval includes zero. Figure 3 follows the same model pairs across these views. The controls below distinguish explanations for this response.

Line chart comparing Full and Removal balanced accuracy across Public, Non-KVASIR, and Internal-filtered evaluation populations

Figure 3. A training comparison reverses across evaluation populations (DINOv2-L/14). Full uses 37,607 rows; Removal excludes KVASIR and five other-source rows passing the Kvasir-Capsule hash screen, retaining 10,591. Markers show ten-seed mean BA; faint paths follow each fitted model’s scores across views. Full leads on public validation; Removal has higher mean BA on non-KVASIR. Below each view are paired Removal-minus-Full changes in percentage points and pointwise 95% tt intervals conditional on fixed lists. The internal-filtered interval includes zero. Table 1 reports all six training pools.

Matching training size and updates. Random and Source-only both contain 10,596 rows and have equal nominal update counts. Their public BA values are 0.779 and 0.611, respectively. The Source-only-minus-Random contrast is −0.1682-0.1682 on public validation and +0.0624+0.0624 on non-KVASIR (Table 1(b)), so sample count alone cannot explain the public loss or its reversal. Removal produces a similar public loss to Source-only: the five additional exclusions change BA by +0.0011+0.0011 (95% interval [−0.0044,+0.0066][-0.0044,+0.0066]).

Class counts and within-class source replacement. Comp and Class-match-R preserve the corresponding reduced pool’s class counts, but 63% of their rows are KVASIR and the dominant source differs by class, so source appearance can predict class (Appendix § 9.§ 2). They reach public BA 0.734 and 0.738 but only 0.388 and 0.391 on non-KVASIR, compared with 0.745 for Full. Pool A replaces Comp’s 66 KVASIR Ulcer images with 66 AIIMS Ulcer images and retains every non-Ulcer row. Public BA falls by 0.069 (95% interval [−0.0752,−0.0628][-0.0752,-0.0628]), whereas non-KVASIR BA, whose Ulcer rows are all AIIMS, rises by 0.098 (95% interval [+0.0898,+0.1057][+0.0898,+0.1057]). This replacement changes the score even when sample and class counts are fixed, making composition essential to interpreting removal effects.

Comparisons across model families, sources and classes. Frozen ResNet-50 [20] and ConvNeXt-Tiny [21] with nonlinear heads show the same reversal: Source-only-minus-Random is −0.200-0.200 and −0.210-0.210 on public validation, respectively, and positive on non-KVASIR. For the primary pipeline, source-specific Removal-minus-Full BA changes are −0.473-0.473 (KVASIR), +0.013+0.013 (SEE-AI), −0.044-0.044 (KID) and +0.021+0.021 (AIIMS). These source-specific averages cover 9, 8, 5 and 2 classes, respectively, and are not additive components of pooled BA. Public-validation Ulcer recall falls from 0.9760.976 to 0.0960.096, and recall losses vary across classes. The matched-size reversal occurs in all three tested model families, alongside source- and class-specific variation.

Auxiliary analyses reinforce the composition interpretation. Removing only rows whose source-matched NCC reaches 0.990.99 or 0.900.90 retains 12,697 and 10,613 training rows (2,101 and 17 KVASIR rows), giving public BA 0.7440.744 and 0.6540.654. Broader removal changes both content and size. Permuting KVASIR labels while preserving their histogram gives BA 0.5040.504; this tests label sensitivity, not instance memorization. Removing SEE-AI lowers SEE-AI validation BA from 0.6970.697 to 0.2390.239. These auxiliary comparisons use different hardware and lack matched-size controls (Appendix § 10).

Metric and endpoint sensitivity. Full and Removal have mean public AUC 0.98790.9879 and 0.88120.8812, and combined scores 0.90660.9066 and 0.74660.7466 (change −0.1600-0.1600). Using each run’s highest public-validation BA instead of its epoch-ten value gives 0.83060.8306 and 0.61780.6178. The contrast is −0.2129-0.2129, close to the final-epoch contrast of −0.2132-0.2132. The public loss thus appears in both score components and under both epoch choices. Selecting an epoch using evaluation outcomes makes that comparison descriptive; the final epoch remains the common reporting point.

Fixed-prediction diagnostics expose metric and rank sensitivity

The preceding experiments change training; re-scoring the participants’ submitted predictions holds them fixed, giving an analogue of Eq. (1). Of the 27 released workbooks, 25 are scoreable; 24 of these agree with their reported full-list scores and form our primary, score-consistent cohort. Score agreement establishes neither that public validation was held out from model fitting and selection nor that the same checkpoint generated both the public-validation and official-test predictions. The primary cohort’s mean organizer-score change on non-KVASIR is −0.050-0.050, with 19 entries changing rank. A 19-entry subset additionally omits Teams U–Y (aliases in Appendix § 8.§ 1); it gives −0.062-0.062, with 14 entries changing rank. An earlier version of this work used this subset as its headline cohort, describing these 19 as teams that trained their own models; that description is unverified, so we retain the subset only as a sensitivity analysis.

For the same 24 entries, equal-source averaging gives −0.020-0.020, compared with the pooled change of −0.050-0.050 (Table 1(d)). Within each source, BA averages recall over represented classes and AUC averages the defined one-versus-rest comparisons. We then average the combined scores equally across sources: four in the public view and three in the non-KVASIR and filtered views. The contrast therefore depends on both source weighting and class support. Appendix § 8.§ 1 specifies cohort exclusions and resampling sensitivities.

Spearman correlation with official-test combined scores falls from 0.5570.557 on public validation to 0.4590.459 on non-KVASIR, and it also falls in each sensitivity cohort (Table 15). A separate DINOv2 official-test study shows that the two score components can respond differently (Table 1(c)). Relative to Full, Source-only training raises mean AUC by 0.03560.0356 (95% paired-tt interval [+0.0309+0.0309, +0.0403+0.0403]; all ten seeds), whereas the interval for its BA difference, −0.0117-0.0117 ([−0.0259-0.0259, +0.0026+0.0026]), includes zero. With the same training recipe, the separate official-test fits reach BA of only 0.270.27–0.280.28, against 0.610.61–0.830.83 for the primary fits on public validation. Reporting diagnostic views alongside the official result makes this dependence on population and metric explicit [22].

CohortnnPublicNon-KVASIRInternal-filtered
Score-consistent (primary)240.5570.4590.497
All scoreable250.5660.3550.390
19-entry subset sensitivity190.2440.0680.139
18-entry subset excl. A180.4630.2570.340

Table 15. Spearman correlation of public-view combined scores with organizer-reported official-test combined scores. The Public column uses the organizer-reported scores; the non-KVASIR and internal-filtered columns use recalculated scores. The primary 24-entry score-consistent cohort excludes only Team G, whose reported public-validation score disagrees with re-scoring. The fixed 19-entry subset sensitivity additionally omits Teams U–Y, without an independently justified exclusion rationale, and the 18-entry sensitivity also omits Team A. All four cohorts have lower observed agreement on both reduced views; possible dependence among entries limits population-level inference.

Local versus distant removal: an exploratory probability contrast

This experiment asks whether removing the training neighbors of exposed validation frames (frames with an exact-pHash training counterpart at NCC ≥0.99\ge0.99) lowers confidence on those frames more than removing equally many distant training rows does, relative to matched unflagged frames. We compare local and distant removals within the same video/class strata while retaining KVASIR. Every arm (training condition) first removes the 1,100 distinct training rows linked to the initial 799 exposed validation rows by exact-pHash pairs with NCC ≥0.99\ge0.99. A video/class stratum is eligible only if it has enough distant training rows to match its local removal; one stratum fails, which excludes its 211 exposed rows and leaves 588 exposed rows EE and 588 matched-unflagged rows UU. We sampled UU without replacement to match video/class counts before eligibility filtering. Unflagged rows have no exact-pHash counterpart, which does not verify absence of exposure. Each group contains 547 Normal rows and 41 rows from five abnormal classes.

Signs count negative/zero/positive seed contrasts (zero tolerance 10−1210^{-12}). Intervals use ten paired seed blocks after averaging distant arms, not 30 independent controls. Both strategies remove training rows; no unremoved baseline is compared. Design, seed set, controls and endpoint were selected with access to results. Intervals are exploratory, conditional on those choices and unadjusted for selection; they do not establish patient-level generalization.

Within eligible strata, each arm removes 876 additional rows: local neighbors in one arm, or distant rows matched by video/class in each of three fixed control arms. Eligibility filtering leaves the common removal unchanged; all arms retain 35,631 rows. Local removal selects each counterpart’s immediate predecessor and successor in its video/class list, sorted by frame index. Distant controls match the resulting run lengths, exclude the local set, stay at least 50 frame indices from exposed rows and apply additional hash-match exclusions (Appendix § 11). Adjacent entries in a list need not be consecutive source frames.

For endpoint gg, local arm LL, distant arms CjC_j and paired seed ss, we estimate

Ds(g)=[gs,L(E)−13∑j=13gs,Cj(E)]−[gs,L(U)−13∑j=13gs,Cj(U)].(3)D_s(g) = \left[g_{s,L}(E) - \frac{1}{3}\sum_{j=1}^{3} g_{s,C_j}(E)\right] - \left[g_{s,L}(U) - \frac{1}{3}\sum_{j=1}^{3} g_{s,C_j}(U)\right]. \tag*{(3)}

This exploratory analysis uses epoch ten and ten paired seeds. True-class probability, negative log-likelihood (NLL), probability margin and accuracy are averaged over rows; BA averages recall over the six represented classes. Probability margin is true-class probability minus the largest alternative-class probability.

Table 2 reports absolute outcomes and the two components of Eq. (3). Local-minus-distant probability changes are −0.321 percentage points for EE and +0.034 for UU; the interval for UU spans zero. Their paired difference is −0.355 (95% paired-tt interval [−0.486, −0.224]). This difference is negative in all ten seeds, although the exposed component is negative in eight. The NLL contrast is +0.00495 (95% paired-tt interval [+0.00329, +0.00660]), and the probability-margin contrast is −0.00472 ([−0.00647, −0.00297]). NLL uses natural-log units; the margin contrast uses absolute probability units. BA and accuracy intervals span zero (Table 1(e)); the four group–strategy accuracy means range from 99.56% to 99.98%.

EndpointE localE distantU localU distant
(a) Absolute endpoint means (%)
True-class probability90.65490.97589.67089.636
Accuracy99.94999.97799.60999.558
Fixed six-class BA99.66799.85294.94794.938
(b) Probability: local minus distant (percentage points)
ComponentMeanPaired-tt 95% intervalSigns (−/0/+)
Exposed (E)−0.321[−0.538, −0.104]8/0/2
Unflagged (U)+0.034[−0.141, +0.209]4/0/6
E−U−0.355[−0.486, −0.224]10/0/0

Table 2. Local and distant removal: absolute outcomes and true-class probability contrasts. E: exposed; U: matched-unflagged (588 rows each). For Distant, we average the three fixed control arms within each seed, then across the ten seeds. BA averages recall over the same six classes.

The rows of EE and UU come from 32 video prefixes. Omitting each prefix in turn leaves the mean probability contrast negative, ranging from −0.390 to −0.215 percentage points; this range describes omission-specific means, not confidence limits. The three largest-magnitude signed prefix contributions account for 85.0% of the net contrast. Each minority class occurs in only one prefix, and only 914 of 10,000 prefix-bootstrap draws retain all six classes; because averaging over present classes changes the BA target, we report no fixed six-class BA interval under prefix resampling (Appendix § 11).

ReuScope: audit design and interpretation

ReuScope links measurements of data relationships to training and evaluation comparisons, specifying the population and uncertainty for each conclusion. Table 3 connects observations to supported conclusions. The four steps begin with released lists, labels and grouping metadata; access to images, predictions and training determines which analyses are possible.

QuestionObservationSupported conclusion
Identical content?SEE-AI: 210 groups, 215 validation files; AIIMS: 6 groups, 6 files.Released bytes cross the public boundary. Label relationships are assessed separately.
Shared videos?KVASIR: 0 byte crossings; all 41 contributing prefixes shared.The split separates files while retaining source-video overlap.
Similar to which counterpart?NCC ≥0.99\ge0.99: 540 chosen-counterpart vs. 799 any-match rows.Different matching rules count different relationships.
Training sensitivity?Removal-minus-Full BA: −0.2132-0.2132 public, +0.0138+0.0138 non-KVASIR.The training contrast depends on the evaluation population.
Patient-level linkage?Shared recordings observed; cross-recording patient IDs unavailable.Unique-patient overlap cannot be quantified from public metadata.

Table 3. Selected conclusions from the CV2024 audit. The counterpart comparison counts KVASIR validation rows with exact-pHash matches in KVASIR training. Missing information differs from a measured zero; populations and uncertainty are defined with each analysis.

Table 3: Selected conclusions from the CV2024 audit. The counterpart comparison counts KVASIR validation rows with exact-pHash matches in KVASIR training. Missing information differs from a measured zero; populations and uncertainty are defined with each analysis.

1. Define the boundary and evidence unit. Specify the release, query/reference populations, the information permitted for model development and the intended generalization claim. Preserve row identities, multiplicities and labels. State what each grouping key, such as a video prefix, denotes and which identity levels, such as patient, are unavailable.

2. Measure each data relationship separately. With images, confirm positive cryptographic-hash groups byte for byte and assess label relationships separately. Use documented identifiers to measure group overlap. For perceptual similarity, specify preprocessing, the screening predicate, counterpart selection, search coverage, same-modality comparison pairs and operating limits. Report pairs, identity groups and affected rows with their own denominators.

3. Separate training and evaluation changes. Aligned prediction records support comparisons across evaluation views with predictions held fixed. With training access, compare matched pools across these views. Report the original public view alongside source-removed and stricter internal-filtered diagnostics where definable; these do not replace the official leaderboard. State sample count, class/source support, update budget, endpoint and seed pairing. Match controls to alternative explanations: Source-only versus Random fixes sample size and the nominal update budget; the within-class replacement fixes size and class counts (§ 3.3). If a required class is absent from an evaluation set or resample, mark the fixed-class statistic undefined; averaging over present classes defines a different target. Intervals across seeds on fixed lists and group resampling address different sources of uncertainty. Record outcome-informed choices and targets that cannot be estimated.

4. Report conclusions with their scope and uncertainty. For each analysis, record its boundary, predicate or contrast, counting unit, denominator, coverage, class support, estimate, uncertainty, selection scope, supported conclusion and unresolved target. Link the record to input lists and numerical evidence. Mark its status as completed, partial, not assessed or not applicable; a measured zero differs from missing evidence. When adapting grouping definitions, screens and training/evaluation interfaces, preserve these fields and the distinction between data relationships and score responses.

A worked audit record. For the any-counterpart finding, the boundary is KVASIR public training versus its 11,581 validation rows. The predicate requires an exact-pHash counterpart with NCC ≥0.99\ge0.99, and the counting unit is an affected validation row. The census finds 799 qualifying rows. Selecting one lexicographic counterpart instead gives 540 under a different rule. The result establishes candidate correspondence under the specified predicate. A score response to changing the training pool is recorded separately using Eq. (2), while patient linkage across recordings remains unknown.

Reproduction and responsible reporting. Code and data for the main analyses, including exact input lists, saved predictions and numerical records, are publicly released; Appendix § 14 gives the repository, maps results to code and data, and lists the components that are not released. Where feasible and compatible with review anonymity, share metadata-only findings with the audited party before submission, invite factual corrections and report when this occurred and what was clarified. Report organizer clarifications separately from independently verified observations. Misinterpreting release anomalies as individual misconduct could cause reputational harm to organizers or participants; report release properties without inferring clinical label correctness or individual responsibility. We contacted the organizers only after the initial submission; Appendix § 7.§ 1 documents the exchange.

Applying components to other releases. The ChestX-ray14 lists contain 86,524 train/validation and 25,596 test filenames, with 28,008 and 2,797 patient prefixes. Neither filenames nor prefixes overlap [23]. This checks the complete identifier lists, not pixel content. For image screening, we use 25,331 ISIC 2019 training images [24] and 10,662 labeled HyperKvasir images [25]. Across strata defined by the ISIC lesion-ID prefix, the sum rule yields three candidates, whereas the joint rule yields 67. Of the 67, two pass NCC ≥0.99\ge0.99 at both tested resolutions, and one is also byte-identical. Each of the two pairs links an image with an empty lesion ID to one with an MSK-prefixed ID, so these strata are metadata groups rather than verified institutions. Comparing HyperKvasir with CV2024 KVASIR yields no candidate under the sum rule and one under the joint rule; that pair fails NCC 0.99. Appendix § 13 gives the pair breakdown. These checks apply the identifier and image-screening components to other releases; they do not test the full sequence of training and evaluation comparisons.

To apply ReuScope elsewhere, adapt the source/group mappings and screen to the new release while retaining explicit boundaries, controls and result fields; the released code encodes CV2024’s sources, classes and Ulcer-replacement counts, which must be replaced. Prevalence, thresholds and effect sizes remain specific to each application.

Conclusion

CV2024 shows why benchmark audits must connect data relationships to precisely defined comparisons. Filename, byte and video checks reveal different dependencies; a source-removal contrast reverses across evaluation populations; and the exploratory local-versus-distant comparison yields a negative difference-in-differences in true-class probability. These findings underpin ReuScope, an audit protocol that specifies the available evidence, separates training from evaluation changes and reports estimates with the conditions under which they can be interpreted. Each conclusion is thus tied to a stated data boundary, comparison population and source of uncertainty.

Limitations. The audit covers one benchmark and one primary training pipeline; the frozen-CNN comparisons and other-release checks each test only part of the protocol. Shared recordings establish overlap across the public split, but missing cross-recording identifiers limit patient-level quantification. Intervals describe seed variation on fixed lists, not patient sampling. The local comparison was designed with outcome access, and the similarity screens are not calibrated duplicate detectors.

Acknowledgments and Disclosure of Funding

This work was supported by the Artificial Intelligence Industrial Convergence Cluster Development Project, funded by the Ministry of Science and ICT (MSIT, Korea) and Gwangju Metropolitan City, and by the IITP AI Semiconductor Program and the ANCHOR Program, funded by the Korean government and the Seoul Metropolitan Government (IITP-2023-RS-2023-00256081; 2026-ANCHOR-01-110).

Competing interests: The authors declare no competing interests.

Data, identity and similarity measurements

This section distinguishes two boundaries: CV2024 public training versus validation, and CV2024 images versus the upstream Kvasir-Capsule reference. For each boundary, we specify the populations, counterpart rules and counting units below. We then examine sensitivity to video-prefix grouping.

Released populations, identity and grouping

The public CV2024 workbooks define 37,607 training and 16,132 validation rows. Source labels come from the organizer workbooks’ Sheet1 column Dataset, keyed by full image_path in workbook order. All 53,739 source labels are recognized and agree with the source directories. Aggregate source counts reproduce the challenge paper’s Table I: training/validation counts are KVASIR 27,011/11,581, SEE-AI 9,996/4,289, KID 376/165 and AIIMS 224/97. The hash screen below does not redefine these source labels. Joins preserve full paths, folder labels and workbook order. We group files by SHA-256 digest and compare files within every positive cross-partition group byte for byte. Table 4 reports this census separately as identity groups and affected validation rows. Shared folder labels establish agreement between released annotations, without establishing clinical correctness.

We shared metadata-only findings, including the official-test scan, with the CV2024 organizers. The lead organizer replied that the 70–30 split was made within abnormality-wise source folders without video or case grouping, that duplicate control removed one of two visually similar sequential frames, without cryptographic checks, and that the organizers hold no patient- or examination-level labels for the Kvasir-Capsule frames; she agreed that we report these points and has sent no corrections as of October 2026. The file-identity counts reported here come from our own byte comparisons.

Table 4: Byte-identical public train–validation crossings. Rates are proportions of affected validation files; a group can contain several files. The final columns count groups whose training and validation label sets intersect or are disjoint. Renamed groups have different training and validation basenames, so a filename-only check misses them. Label-disjoint groups are release/label anomalies; label intersection does not establish clinical label truth.

Video prefixes come from attributed source filenames. All 41 contributing KVASIR prefixes occur in both public partitions; this establishes shared examination recordings. The public identifiers do not determine the number of distinct patients or additional overlap across different prefixes. Of the 43 labeled reference videos, 21 have more than 90% attributed-row coverage, defined as attributed CV2024 rows divided by the reference inventory for that prefix. Because the numerator counts rows, this need not be the fraction of unique reference images. After mapping source categories to CV2024 labels (for example, blood_fresh to Bleeding), labels agree for the 38,584 unambiguously mapped rows; eight filenames that occur under both Erosion and Pylorus in the reference are labeled Erosion in CV2024.

Source-to-reference screens and pair selection

Source-to-reference screening compares CV2024 queries with the upstream Kvasir-Capsule reference corpus, a different boundary from public training–validation overlap. Let dpd_p and ddd_d be Hamming distances between 64-bit pHash and dHash descriptors computed with ImageHash [16]. For query xx and reference corpus RR, the hash screen requires

min⁡r∈Rdp(x,r)≤6andmin⁡r∈Rdd(x,r)≤6.\min_{r \in R} d_p(x,r) \le6 \quad\text{and} \quad\min_{r \in R} d_d(x,r) \le6.

The minimizing references may differ. The joint rule instead requires a single rr satisfying both inequalities. All KVASIR queries pass both rules at radius six in this analysis, but they differ at smaller radii and for other-source candidates. Table 5 uses independent minima. PDQ provides a complementary representation [26]; its distance-50 cutoff is specific to this study and has no capsule-image sensitivity or specificity calibration.

SourcennBoth minima zeroEach min ≤2\le2Each min ≤6\le6PDQ ≤50\le50PDQ ≤90\le90
KVASIR38,59223,44438,25938,59238,35738,592
SEE-AI14,2850030451
KID541001039
AIIMS3210010320

Table 5. Query-level matching to Kvasir-Capsule.

Table 5: Query-level matching to Kvasir-Capsule. pHash and dHash are minimized independently over reference images. Both minima can be zero without a shared reference match or byte identity. PDQ is a separate perceptual hash; its cutoffs are study-specific. Distance 90 illustrates liberal candidate selection; it is not an official default.

For pair-level comparisons, we use the pHash-minimizing counterpart recorded in the retained measurements. Equal pHash distances are resolved by first occurrence in the preserved reference-cache order, rather than by a lexicographic path rule. The cache follows filesystem enumeration order; reproducing ties requires that retained ordering, not a newly sorted reference list. Both distances are computed for that same counterpart. The selected counterpart can therefore have dHash distance greater than six even when the query passes the hash screen through another reference. Separate zero minima need not share a counterpart; even a common zero-distance counterpart need not be byte-identical.

For centered grayscale intensity vectors aa, bb, normalized cross-correlation is

NCC⁡(a,b)=aTb∥a∥2∥b∥2+10−12.\operatorname{NCC}(a,b) = \frac{a^{\mathsf{T}}b}{\lVert a\rVert_2\lVert b\rVert_2 + 10^{-12}.}

We use float32 arithmetic after direct 96×9696 \times96 bicubic resizing, without registration or cropping. Pixel scores are computed for the specified counterpart; independently minimized hash distances need not describe that same pair.

Across the 38,592 selected KVASIR/reference pairs, 35,625 (92.31%) reach NCC ≥0.99\ge0.99. Table 6 groups those scored pairs by their own max⁡(dp,dd)\max(d_p,d_d). Its distance-3–6 band contains 2,372 pairs, of which 858 (36.17%) reach 0.99. A different grouping—queries whose independently minimized distances have a maximum in the range 3–6—contains 333 queries, with 313 (93.99%) reaching the NCC threshold with their selected counterpart. These strata describe different populations; neither estimates detector precision. In particular, the high rate in the query-minimum stratum does not describe all pairs whose own distances lie in that band.

max⁡(dp,dd)\max(d_p,d_d)nnMean NCCMedian NCC≥.99\ge.99≥.95\ge.95≥.90\ge.90
020,4480.99940.999699.39%99.99%100.00%
1–215,4730.99790.999593.23%99.76%99.96%
3–62,3720.98220.986236.17%96.21%99.07%
>62990.96900.97546.35%86.96%99.00%

Table 6. Normalized cross-correlation (NCC) by selected-pair hash distance.

Table 6: Normalized cross-correlation (NCC) by selected-pair hash distance. Both hash distances refer to the pair scored by NCC. The final columns give the fraction of pairs reaching each NCC threshold. Query-level minima can correspond to different reference images and do not define these bands.

Feature comparisons use the retained DINOv2-L/14 dinov2_vitl14_pretrain.pth checkpoint (LVD-142M).2 They provide a complementary representation of the same upstream-source relationship. We directly resize to 224×224224 \times224, apply ImageNet normalization, extract final-layer-normalized CLS features, normalize their length and compute cosine similarity. The 38,592 KVASIR/reference pairs have mean cosine 0.9658; 1,000 randomly drawn comparison pairs, excluding each query’s designated reference, have mean 0.5376. The five other-source screen-positive candidates and random-pair similarity summaries are reported in Table 7. These measurements characterize resemblance under a specified representation, not duplicate ground truth or independence from pretraining exposure.

PoolnnMean cosine% cosine ≥.90\ge.90
KVASIR all38,5920.965898.32
pHash 032,8540.966298.49
pHash 1–25,6730.963597.43
pHash 3–4650.950992.31
SEE-AI30.71040.00
KID10.39280.00
AIIMS10.40750.00
Random comparison1,0000.53760.30

Table 7. DINOv2 feature cosine similarity for CV2024 queries paired with Kvasir-Capsule references. All similarities use the same feature pipeline; the pHash rows partition the KVASIR pair inventory by hash distance. Threshold percentages describe feature-space similarity, not duplicate labels.

The joint rule can miss transformed copies (Table 8). In a test on 50 real frames with synthetic transforms, no pair passes the radius-six condition after any tested center crop or a 10∘10^\circ rotation, only 5 of 50 pass after a 5∘5^\circ rotation, and every JPEG-compressed or resized pair passes. The sample is selected by Python Random(0).sample from the sorted 47,238 reference JPG paths in class/filename order. Each transform acts independently on the original RGB image. For side fraction ff, the crop has width ⌊fw⌋\lfloor fw\rfloor and height ⌊fh⌋\lfloor fh\rfloor, with each offset given by half the remaining dimension rounded down; it is resized back to (w,h)(w,h). Rotations call rotate(deg, fillcolor=(0,0,0)) at +2∘+2^\circ, +5∘+5^\circ, +10∘+10^\circ, without explicit resampling or expansion arguments. JPEG quality is set to 90, 70 or 50, then the result is decoded as RGB. Resize controls pass through 128×128128 \times128 or 384×384384 \times384 before returning to the original dimensions. Other resampling and JPEG encoder options use Pillow defaults; this diagnostic’s Pillow and codec versions were not recorded. These pairs characterize the tested transformations and frames, not real-world duplicate recall.

Transformationdp,dd≤6d_p,d_d \le6dp,dd≤2d_p,d_d \le2
Center crop, 90%0/50 (0%)0/50 (0%)
Center crop, 75%0/50 (0%)0/50 (0%)
Center crop, 60%0/50 (0%)0/50 (0%)
Rotation, 2∘2^\circ40/50 (80%)5/50 (10%)
Rotation, 5∘5^\circ5/50 (10%)0/50 (0%)
Rotation, 10∘10^\circ0/50 (0%)0/50 (0%)
JPEG, quality 9050/50 (100%)50/50 (100%)
JPEG, quality 7050/50 (100%)50/50 (100%)
JPEG, quality 5050/50 (100%)48/50 (96%)
Resize, 128×128128 \times12850/50 (100%)50/50 (100%)
Resize, 384×384384 \times38450/50 (100%)50/50 (100%)

Table 8. Synthetic joint-rule hash sensitivity on 50 Kvasir-Capsule frames from 30 video prefixes. Each cell counts original–transformed pairs satisfying both 64-bit hash thresholds. Crop percentages indicate the retained fraction of each side; cropped and resized images return to the original dimensions. The same frames are used in every row. These pass rates describe the tested transformations and frames, not recall on naturally occurring duplicates.

Within-CV2024 comparisons and same-modality controls

Within the public split, Table 9 counts validation rows with a same-source training counterpart under each stated predicate. Among the 1,381 exact-pHash KVASIR rows, 540 have NCC ≥0.99\ge0.99 with the first lexicographic counterpart and 799 satisfy that threshold with any exact-pHash counterpart. These counts answer different matching questions; neither is a detector-recall estimate. On the same 1,381 selected pairs, grayscale 256 × 256 bilinear processing gives 387 pairs with NCC ≥0.99\geq0.99, compared with 540 at 96 × 96 bicubic. For centered float32 arrays with Sa=∑iai2S_a = \sum_i a_i^2 and Sb=∑ibi2S_b = \sum_i b_i^2, the alternate calculation uses SaSb\sqrt{S_a S_b} as its denominator without additive epsilon, returning zero when that denominator is at most 10−910^{-9}. The 96 × 96 calculation instead uses SaSb+10−12\sqrt{S_a}\sqrt{S_b} + 10^{-12}. Resolution, interpolation and normalization arithmetic therefore change together; this comparison does not use the 799-row any-counterpart rule. All 1,381 selected pairs share the same video prefix between their two images. Separately, minimizing the frame-index gap over same-video exact-pHash training candidates gives median one and interquartile range 1–6 over these 1,381 validation rows. The nearest same-video training frame over all 11,581 KVASIR validation rows has median one and interquartile range 1–1.

SourceValidation rowsBasename matched rowsExact-pHash rowsJoint-rule candidate rows
KVASIR11,58101,3814,896
SEE-AI4,289174218264
KID165014
AIIMS97066

Table 9. Within-source public train–validation candidates. Counts are validation rows with a training counterpart under each screen, including separate rows sharing a basename. The joint rule requires both hash distances to be at most six for the same pair. Candidate counts do not imply pixel verification; byte identities are reported separately.

Pixel checks of the other-source candidate pairs are summarized separately in Table 10. KID has pixel-screen candidates despite having no byte-identical crossing. Candidate pairs, affected validation rows and identity groups remain different units; NCC on resized grayscale images is not file-byte equality.

SourceRowsPairsMean NCC≥.99\geq.99≥.95\geq.95≥.90\geq.90
SEE-AI4,2892640.9831220228244
KID16540.9391223
AIIMS9761.0000666

Table 10. NCC for selected within-source train–validation candidate pairs. The Rows column gives the validation size; the Pairs column counts one selected training counterpart per eligible row, using the lexicographically first full path satisfying dp,dd≤6d_p, d_d \leq6 on that pair. NCC uses grayscale 96 × 96 bicubic resizing; threshold columns count selected pairs. These conditional counts do not estimate true-duplicate prevalence or detector precision.

SSIM [17] is computed with scikit-image [27], using the same grayscale geometry, data range 255, uniform 7 × 7 windows and sample covariance. Each comparison group contains 1,381 pairs. Controls exclude flagged pairs, exact basenames and exact-pHash matches. The adjacent control pairs an unflagged validation frame with the eligible same-video training frame having the smallest absolute numeric frame-index gap; equal-gap ties follow the stored cache order. Candidate pairs are deduplicated, shuffled and subsampled to 1,381 using a shared random-number stream with seed 42. Distant same-video controls instead require a frame-index gap of at least 500. Table 11 compares selected pairs with these controls.

Pair groupnnMedian5th–95th percentile95% joint-prefix gap interval
Selected exact-pHash1,3810.880[0.738, 0.963]—
Adjacent same-video1,3810.647[0.313, 0.899][+0.186, +0.285]
Distant same-video1,3810.432[0.244, 0.653][+0.401, +0.487]
Different-video, same-class1,3810.442[0.246, 0.615]—
Different-video, any-class1,3810.441[0.253, 0.612]—
Non-KVASIR, same-source/class1,3810.330[0.191, 0.495]—

Table 11. Structural similarity for selected pairs and comparison groups. Each group contains 1,381 pairs. The 5th–95th percentiles describe individual pair SSIM values. The 95% joint-prefix intervals concern selected-minus-control median SSIM for same-video controls; they resample shared video prefixes together, conditional on the selected pairs. Dashes mark the reference row or comparisons without a reported joint interval.

The selected pairs have higher median SSIM than the controls, but the distributions overlap and 68 adjacent controls also reach NCC ≥0.99\geq0.99. Unflagged pairs are therefore comparisons, not verified negative examples.

Both images share a prefix in every selected, adjacent and distant pair. These inventories contain 37, 40 and 39 prefixes, respectively. For the adjacent and distant comparisons, we jointly resample their unions of 40 and 39 prefixes, applying the same multiplicities to selected and control pairs. The 95% percentile intervals for selected-minus-control median SSIM are [0.186, 0.285] and [0.401, 0.487], respectively, over 10,000 draws, conditional on the pair inventories. These uncertainty intervals differ from the 5th–95th percentiles of individual SSIM values in Table 11. Different-video controls depend on the videos of both images, so grouping only by the validation image does not represent this dependence; their distributions are reported without confidence intervals. The non-KVASIR control spans only three source clusters and is likewise reported without a joint interval. None of these comparisons establishes different-patient sampling.

Table 11: Structural similarity for selected pairs and comparison groups. Each group contains 1,381 pairs. The 5th–95th percentiles describe individual pair SSIM values. The 95% joint-prefix intervals concern selected-minus-control median SSIM for same-video controls; they resample shared video prefixes together, conditional on the selected pairs. Dashes mark the reference row or comparisons without a reported joint interval.

Sensitivity to hypothetical video grouping

The exact-pHash rate 1,381/11,581 = 11.9247% is a census of the KVASIR validation list. Table 12 describes sensitivity to hypothetical prefix-resampling assumptions, without estimating patient membership or patient-level uncertainty. Each draw computes total flagged rows divided by total rows, preserving resampling multiplicity rather than averaging prefix rates equally.

Sampled prefixes95% range (%)Width / 41-prefix width
41[6.899, 17.996]1.000
30[6.353, 19.046]1.144
20[5.636, 20.807]1.367
10[4.389, 24.824]1.841

Table 12. Sensitivity to the number of resampled video prefixes. The exact-pHash rate, 11.9247%, is a census of KVASIR validation. These conditional percentile ranges describe hypothetical prefix sampling, not uncertainty in the enumerated count or a patient-level bound.

Each scenario draws mm prefixes with replacement from the 41-prefix empirical distribution, using 20,000 draws and resetting NumPy’s default generator to seed 42. We report raw percentile intervals for 41, 30, 20 and ten sampled prefixes, with widths relative to the 41-prefix interval. The ten-prefix scenario is the widest tested, not a patient-level bound; unknown patient groups could be fewer in number, unequal in size or otherwise different from these sampled prefixes.

Table 12: Sensitivity to the number of resampled video prefixes. The exact-pHash rate, 11.9247%, is a census of KVASIR validation. These conditional percentile ranges describe hypothetical prefix sampling, not uncertainty in the enumerated count or a patient-level bound.

A complementary scheme partitions all 41 prefixes into synthetic groups with balanced prefix counts, then resamples those groups. Across 1,000 random partitions and 20 resamples per partition, the ranges for 30, 20 and ten groups are [6.858, 18.014]%, [6.888, 17.975]% and [6.925, 17.729]%, respectively, compared with [6.899, 17.996]% for 41 prefixes. The largest shift of any interval limit is 0.27 percentage points. Balanced coarsening retains all prefixes in each partition; drawing fewer prefixes addresses a different hypothetical sampling target. Neither construction identifies the unknown patient grouping or guarantees its uncertainty coverage.

Fixed-prediction diagnostics and official-test evaluation

Public-submission diagnostics

We join submitted predictions by full path and named class columns, then re-score them on fixed evaluation views. The challenge score averages class-macro AUC and BA [3]; AUC uses the supplied scores without an added softmax. The 27 workbooks use aliases A–AA. Scoreability requires exact full-path alignment to all 16,132 validation rows, all ten named class columns, no missing values and no duplicate paths. Entries A–Y pass; Z contains bare filenames, 44 of which occur more than once (45 surplus rows), while AA contains absolute Windows paths that do not match the required paths. Both fail the path check; no path repair is applied. Of the 25 scoreable entries, 24 reproduce the organizer-reported full-list AUC, BA and combined score with absolute errors below 10−1210^{-12}, before display rounding. Team G’s workbook re-scores at BA 0.100 and AUC 0.378, whereas its reported full-list values are 0.899 and 0.944. Table 13 subtracts the reported full-list score, so the all-25 contrast includes this discrepancy as well as row filtering. The primary, score-consistent 24-entry cohort excludes Team G. The 19-entry subset additionally excludes Teams U, V, W, X and Y. In an earlier version of this work, this subset served as the headline cohort on the premise that its teams had trained their own models; that premise was not independently verified, so we retain the subset only to show sensitivity to cohort membership. An 18-entry sensitivity also omits Team A, which scores 1.0 on every public view. Such a score is consistent with predictions from a model trained on public validation, which the challenge permitted for final-test models (next paragraph); it does not by itself indicate a rule violation. The aliases keep team names out of the text but do not anonymize entries: the release pins each public workbook, whose filename names its team, by SHA-256.

Evaluation viewCohortnnMean Δ\DeltaNominal 95% BCaEntries changing rank
Non-KVASIRAll (mixed baseline)25−0.073[−0.167, −0.043]18
Non-KVASIRPrimary24−0.050[−0.067, −0.032]19
Non-KVASIRSubset19−0.062[−0.079, −0.045]14
Non-KVASIRSubset excl. A18−0.066[−0.082, −0.049]14
Internal-filteredAll (mixed baseline)25−0.063[−0.165, −0.034]18
Internal-filteredPrimary24−0.039[−0.056, −0.023]16
Internal-filteredSubset19−0.049[−0.066, −0.034]11
Internal-filteredSubset excl. A18−0.051[−0.069, −0.037]11

Table 13. Public-submission re-scoring. Score differences are relative to the organizer-reported public-validation score, the mean of macro AUC and balanced accuracy. Nominal 95% bias-corrected and accelerated (BCa) ranges resample entries individually with frames fixed. Entry independence is unestablished; we report descriptive resampling ranges and do not interpret them as calibrated population intervals. Rank changes are within each cohort. The all-25 rows use a mixed baseline: one reported public-validation score disagrees with full-list re-scoring. The primary cohort excludes that entry; the fixed subset additionally omits Teams U–Y, and its sensitivity further omits Team A.

Final-test training could use both public partitions. The supplied records do not establish, for every array, whether public-validation rows were excluded from fitting or whether the same checkpoint generated the official-test predictions. Re-scoring therefore describes fixed prediction records, without certifying held-out performance.

The nominal 95% BCa ranges [28] resample entries independently with evaluation frames fixed. SciPy uses 10,000 resamples per table row and one advancing NumPy default-generator stream initialized at seed 42, in table-row order. Within each view, entries enter the resampling in descending order of unrounded combined score, and each cohort keeps that order. Because independence among entries has not been established, we do not interpret these ranges as calibrated confidence intervals for a population of participants or models. Filtering does not improve public-to-official rank agreement in any of the four cohorts. These are evaluation-view diagnostics of stored predictions, not estimates of the benefit of retraining.

Equal-source averaging changes the weighting and represented-class support of the metric. Within a source, BA averages represented-class recall and AUC uses classes for which the one-versus-rest comparison is defined; the combined score averages those two quantities. We then average the source scores equally. For the primary 24-entry cohort, the non-KVASIR change in equal-source combined score is −0.020, compared with −0.050 under pooled aggregation (Table 14, which repeats the pooled and equal-source combined changes of Table 1(d) and adds equal-source BA). The 19-entry sensitivity gives −0.037 in equal-source BA and −0.022 in equal-source combined score. These comparisons make the metric and cohort dependence explicit. Small KID/AIIMS slices and unequal class support limit comparisons beyond these fixed lists.

MetricCohort (nn)Non-KVASIRInternal-filtered
Pooled challenge score24 (primary)−0.0503−0.0393
Equal-source BA24 (primary)−0.0308−0.0255
Equal-source combined24 (primary)−0.0199−0.0163
Pooled challenge score19 (subset)−0.0624−0.0488
Equal-source BA19 (subset)−0.0370−0.0305
Equal-source combined19 (subset)−0.0220−0.0177

Table 14. Mean paired score changes (view minus public validation) for the primary 24-entry score-consistent cohort (Team G alone excluded) and the fixed 19-entry subset sensitivity (Teams U–Y additionally omitted). The pooled challenge score averages BA and macro AUC. Equal-source scores weight sources equally: four in the public view and three in the non-KVASIR and filtered views. Within each source, BA averages recalls over present classes, and the combined score averages BA and present-class macro AUC. These descriptive means hold predictions fixed.

Table 15 reports the descriptive association with official-test ranks using each cohort’s own entries. Team A, whose public score is 1.0, strongly affects the 19-entry coefficients: excluding it raises them from 0.244, 0.068 and 0.139 to 0.463, 0.257 and 0.340 on the public, non-KVASIR and internal-filtered views. Spearman correlations use unrounded combined scores and average ranks for exact ties; no exact score ties occur in these cohorts. Lower observed correlation on the reduced views does not establish a population-level difference between correlations. These results do not calibrate either diagnostic view as a replacement leaderboard.

Official AIIMS test evaluation

A separate byte scan of the 4,385 released official-test files finds no match with the public training folder, one label-disjoint match with public validation and eight within-test duplicate groups. The challenge permitted final-test training on both public partitions [Handa et al., 2024], so the search against the training folder covers only part of the permitted final-training pool. This scan describes released file content; it establishes neither participant-specific exposure nor patient independence.

The separate official-test study evaluates DINOv2-L/14 head/LoRA adaptation on full, random-10,596 and source-only-10,596 training pools at epoch ten over ten seeds. Unlike the two-class public AIIMS subset, the official AIIMS test represents all ten classes. Its 4,385-row population and source-only training pool are distinct from the public-validation 10,591-row removal comparison. Table 16 reports means and standard errors, extending Table 1(c) with the combined score and accuracy. Normal makes up 80.4% of the test rows, and all three pools have lower accuracy than always predicting Normal; BA weights the ten classes equally. The release includes the producer code, each run’s arguments, GPU model and library versions, and the recorded confusion matrices and class-specific AUCs, permitting re-aggregation of BA, accuracy and mean AUC (Appendix H). The producer saved no checkpoints or per-image predictions, so checks beyond these aggregates require retraining.

Training poolBalanced accuracyMean AUCCombinedAccuracy
Full0.2804±0.00460.2804 \pm0.00460.7177±0.00150.7177 \pm0.00150.4991±0.00260.4991 \pm0.00260.1758±0.00980.1758 \pm0.0098
Random-10,5960.2786±0.00380.2786 \pm0.00380.7352±0.00360.7352 \pm0.00360.5069±0.00280.5069 \pm0.00280.3142±0.01190.3142 \pm0.0119
Source-only-10,5960.2688±0.00340.2688 \pm0.00340.7533±0.00130.7533 \pm0.00130.5110±0.00190.5110 \pm0.00190.0988±0.00630.0988 \pm0.0063

Table 16. Official AIIMS test evaluation. Mean ±\pm standard error (SE) over ten seeds. Source-only training uses 10,596 rows, distinct from the 10,591-row removal input. The combined score averages balanced accuracy and mean AUC. Paired by seed, Source-only exceeds Full in mean AUC in all ten seeds, whereas the interval for its BA difference includes zero (see text); these results do not validate either filtered public view as an official-test proxy.

Paired by seed label, Source-only minus Full is +0.0356 in mean AUC (95% paired-tt interval [+0.0309, +0.0403]; positive in all ten seeds) and −0.0117 in BA ([−0.0259, +0.0026]; negative in seven). The AUC gain is therefore clear, whereas the BA difference is not resolved; the combined score, which averages the two, changes by +0.0120 ([+0.0032, +0.0207]). Against Random, Source-only is +0.0180 in AUC ([+0.0112, +0.0249]) and −0.0098 in BA ([−0.0230, +0.0033]). Seed labels pair the runs, not their GPU models. replay.py recomputes these contrasts from the released confusion matrices and class AUCs. These results do not calibrate the non-KVASIR or internal-filtered public views as replacements for official-test evaluation.

Training design and evaluation populations

Training-pool summaries use ±\pm for sample standard deviation unless labeled SE or h95h_{95}. In Table 1, h95h_{95} denotes the half-width of the paired-tt interval, not a standard error. For paired differences dsd_s, the 95% interval is dˉ±t9,0.975sd/10\bar{d} \pm t_{9,0.975}s_d/\sqrt{10}, assuming independent, approximately normal seed-wise contrasts. Seed-percentile intervals use 10,000 paired resamples, NumPy’s default generator with seed 42 and linearly interpolated quantiles. These pointwise intervals are conditional on fixed lists and pool construction. They provide neither simultaneous coverage over all displayed contrasts nor patient-sampling uncertainty. Means and interval limits are rounded separately.

Optimization and training pools

DINOv2-L/14 uses a LayerNorm–linear–GELU–dropout–linear classification head with 256 hidden units and dropout probability 0.1. Rank-eight LoRA modules with scale 16/8 adapt the combined query/key/value projections in the final four blocks. For each adapted linear map, the added term is 2WupWdownx2W_{\mathrm{up}}W_{\mathrm{down}}x, with entries of WdownW_{\mathrm{down}} initialized independently as N(0,0.012)\mathcal{N}(0,0.01^2) and WupW_{\mathrm{up}} initialized to zero. Backbone evaluation mode is maintained during training. The head trains throughout ten epochs; LoRA updates begin after three head-only epochs. AdamW [29] retains its state across epochs and uses a head learning rate of 3×10−43 \times10^{-4}, a LoRA learning rate of 10−410^{-4} and weight decay of 10−410^{-4}. We use the default betas (0.9,0.999)(0.9, 0.999) and epsilon 10−810^{-8}, with no learning-rate schedule. Cross-entropy uses label smoothing 0.05 and effective-number class weights [30] wc=10rc/∑jrjw_c = 10r_c/\sum_j r_j, where rc=(1−0.999nc)/(1−0.999nc)r_c = (1 - 0.999^{n_c})/(1 - 0.999^{n_c}) for a class with ncn_c training rows; every primary pool contains all ten classes.

Training and evaluation use deterministic direct 224 ×\times 224 resizing and ImageNet normalization. The training implementation applies TorchVision Resize((224,224)) to PIL RGB inputs: bilinear interpolation with PIL antialiasing under TorchVision 0.21.0. Both stages use FP16 autocast; evaluation softmax uses float32 logits. Models are initialized through the Torch Hub entry dinov2_vitb14 in the local DINOv2 cache, without an explicit weight override. The repository revision and per-run frozen-backbone digests were not recorded. Training batches contain 128 rows, with shuffled order and the final incomplete batch dropped. Full training supplies 293 minibatches per epoch, compared with 82 for each removal, source-only and size-matched control pool. Equal epoch counts therefore yield different numbers of updates for full and reduced pools, while the random and source-only pools match both row count and nominal minibatch count. The eleven DINOv2 conditions share this pipeline, A100 hardware, seeds 0–9 and evaluation at epoch ten, giving 110 model fits. Within each condition, the training list and hyperparameters are fixed across seeds. Pool-construction experiments considered seeds 0 and 42; Appendix C.2 specifies the retained sampling rules. The available records do not fully resolve the original hyperparameter and list-selection rationale; inference remains conditional on the retained settings and lists.

The Removal pool excludes all KVASIR training rows and five other-source rows positive under the hash screen, leaving 10,591 rows. The five additional exclusions comprise two SEE-AI Erosion images, one SEE-AI Polyp image, one KID Normal image and one AIIMS Worms image. Source-only excludes only KVASIR and contains 10,596 rows. Table 17 gives their class/source support; Table 18 reports every training-pool condition on public validation.

ClassSEE-AIKIDAIIMSSEE-AIKIDAIIMStraining
Angioectasia53018022890548
Bleeding5193022320522
Erosion2,340001,003002,338
Erythema5800024900580
Foreign Body2490010700249
Lymphangiectasia3766016230382
Normal4,31231501,84913604,626
Polyp1,0903404681501,123
Ulcer0066002966
Worms001580068157
Total9,9963762244,2891659710,591

Table 17. Class and source composition after source removal. Source-only training contains 10,596 rows; non-KVASIR validation contains 4,551 rows. The last column gives class totals for the distinct 10,591-row Removal input.

Training poolRowsBalanced accuracyΔ\Delta vs. Full
Full37,6070.825 ±\pm 0.008—
Random10,5960.779 ±\pm 0.005−0.046
Source-only10,5960.611 ±\pm 0.008−0.214
Removal10,5910.612 ±\pm 0.007−0.213
Class-match-R10,5910.738 ±\pm 0.005−0.087
Comp10,5960.734 ±\pm 0.007−0.092
A10,5960.665 ±\pm 0.003−0.161
B10,5960.743 ±\pm 0.006−0.082
C10,5960.661 ±\pm 0.005−0.165
DdupD_{\mathrm{dup}}10,5960.737 ±\pm 0.005−0.088
DuniqD_{\mathrm{uniq}}10,5960.739 ±\pm 0.005−0.086

Table 18. Training-pool results on the public validation list (16,132 rows). Balanced accuracy is mean ±\pm sample SD over ten seeds at epoch ten. Removal and Class-match-R contain 10,591 rows; other reduced pools contain 10,596. Replacement controls A–D are defined in Appendix C.2.

Matched composition controls

The random control draws 10,596 rows from full training. Class-match-R is a separate 10,591-row draw matching the class counts of Removal; Comp contains 10,596 rows and matches the class counts of Source-only. Ordered lists are reproducible from the training workbook without presorting or sampling with replacement. Random uses pandas [31] sample(n=10596, random_state=0) and retains the returned order. For both class-matched pools, process classes in Table 17 order: draw as many KVASIR rows as possible up to each target count, then fill any remainder from non-KVASIR. Comp uses the integer random_state=0 separately for every draw. Class-match-R instead passes one advancing NumPy RandomState(0) through all draws. Concatenate class blocks with KVASIR rows first, then shuffle once using a fresh integer random_state=0. These recipes reproduce every ordered row with pandas 2.1.4 and NumPy 1.26.4. Both class-matched pools preferentially sample KVASIR; they are not uniform class-only draws. Comp contains 6,687 KVASIR, 3,716 SEE-AI, 158 AIIMS and 35 KID rows. Its Normal, Angioectasia, Foreign Body, Lymphangiectasia and Ulcer rows are all KVASIR; its Erosion, Erythema and Polyp rows are 81–94% SEE-AI; its Worms rows are all AIIMS; and its Bleeding rows are 312 KVASIR, 209 SEE-AI and one KID. Class-match-R differs from these counts by at most two rows in any class and source. Pool sampling seeds are distinct from training seeds.

The following controls are variants of Comp and use the same training pipeline. Comp contains 66 KVASIR Ulcer images. Whenever Normal rows are replaced below, the same 66 KVASIR Normal rows are used, selected from Comp in its existing order with pandas sample(n=66, random_state=42).

  • A. Replace Comp’s 66 KVASIR Ulcer images with 66 AIIMS Ulcer images, preserving every non-Ulcer identity.

  • B. Starting from Comp, add those 66 AIIMS Ulcer images and remove the 66 Normal rows; retain the original KVASIR Ulcer images.

  • C. Starting from A, replace the 66 Normal rows with second copies of its 66 AIIMS Ulcer images.

DdupD_{\mathrm{dup}}. Starting from Comp, replace the 66 Normal rows with second copies of its 66 KVASIR Ulcer images.

DuniqD_{\mathrm{uniq}}. Starting from Comp, replace the 66 Normal rows with 66 additional unique KVASIR Ulcer images.

For DuniqD_{\mathrm{uniq}}, exclude Comp’s Ulcer paths from Full’s KVASIR Ulcer rows, retain workbook order, then apply sample(n=66, random_state=42). In A–D, retained rows keep their relative order and additions are appended without a final shuffle. Added AIIMS rows follow workbook order; duplicated KVASIR Ulcer rows follow Comp order. Thus A preserves non-Ulcer identities and relative order, not their absolute positions. B, C and both D pools share the same remaining background. Table 18 reports all training pools before the paired comparisons in Table 19.

ContrastPublic Δ\Delta [95% CI]Non-KVASIR Δ\Delta [95% CI]
Source-only − Random−0.1682 [−0.1754, −0.1610]+0.0624 [+0.0516, +0.0732]
A − Comp−0.0690 [−0.0752, −0.0628]+0.0978 [+0.0898, +0.1057]
B − Comp+0.0092 [+0.0034, +0.0151]+0.0927 [+0.0855, +0.0999]
C − A−0.0042 [−0.0085, +0.0002]−0.0050 [−0.0111, +0.0011]
DdupD_{\mathrm{dup}} − C+0.0765 [+0.0706, +0.0824]−0.0882 [−0.0961, −0.0802]
DdupD_{\mathrm{dup}} − Comp+0.0033 [−0.0022, +0.0088]+0.0046 [−0.0026, +0.0118]
DuniqD_{\mathrm{uniq}} − DdupD_{\mathrm{dup}}+0.0020 [+0.0010, +0.0030]+0.0002 [−0.0010, +0.0014]
Removal − Source-only+0.0011 [−0.0044, +0.0066]+0.0017 [−0.0078, +0.0112]

Table 19. Training-pool contrasts by evaluation population. Each cell gives the mean BA difference and a pointwise 95% paired-tt interval over ten seeds. Each contrast includes any changes to class weights and sampling induced by the stated training-pool changes.

Table 18: Training-pool results on the public validation list (16,132 rows). Balanced accuracy is mean ±\pm sample SD over ten seeds at epoch ten. Removal and Class-match-R contain 10,591 rows; other reduced pools contain 10,596. Replacement controls A–D are defined in Appendix § 9.§ 2.

Table 19 compares these specified list changes on their common backgrounds. A-minus-Comp measures replacement of the specified Ulcer images; C-minus-A additionally changes class counts and repetition. These comparisons do not exhaust class/source interactions, and their sums are arithmetic decompositions rather than identified causal shares of the full removal effect.

For C-minus-A on public validation, the 95% seed-percentile interval [−0.0077, −0.0006] excludes zero, whereas the paired-tt interval in Table 19 includes it. Evidence for a nonzero contrast therefore depends on the interval construction; neither result establishes equivalence.

The separate equivalence check for B-minus-Comp uses a 90% paired-tt interval around the +0.009238 BA difference: [+0.0045, +0.0140]. The ±0.010 margin was fixed in an earlier version of this work, after the first four seeds of a previous run of this comparison; the fits reported here were trained later, and the margin has no external calibration. Because the 90% interval extends beyond +0.010, equivalence within the margin is not established. Likewise, removal-minus-source-only is +0.0011 with 95% interval [−0.0044, +0.0066]. This contrast measures the response to deleting five rows under the training pipeline, including the resulting changes to weights and sampling; an interval spanning zero does not demonstrate equivalence or isolate an image-content mechanism.

Validation views, class support and uncertainty

Table 20 crosses training pools with three fixed evaluation views: public, non-KVASIR and internal-filtered validation. It repeats the values of Table 1(a) so that the per-source and per-class results below can be read beside them. The predictions within each training condition are held fixed across views, separating evaluation-population sensitivity from retraining. All ten classes occur in every view.

Training poolTrain rowsPublicNon-KVASIRInternal-filtered
Full37,6070.8250.7450.773
Random10,5960.7790.6950.707
Comp10,5960.7340.3880.401
Class-match-R10,5910.7380.3910.406
Removal10,5910.6120.7590.778
Source-only10,5960.6110.7570.776

Table 20. DINOv2-L/14 training pools crossed with fixed validation views. Entries are epoch-ten mean BA over ten matched seeds; predictions are held fixed across views. Public, non-KVASIR and internal-filtered views contain 16,132, 4,551 and 4,326 rows, respectively, each supporting all ten classes.

The internal filter starts with the 4,551 non-KVASIR validation rows. All 37,607 training and 16,132 validation rows have pHash values, so the implementation’s hash-availability condition excludes none. Within each source, validation rows are flagged if their basename occurs in training or their pHash equals a training pHash. The union of flagged basenames across sources removes every matching validation row, preserving order: 225 rows with 221 unique basenames are removed, leaving 4,326. No same-class, dHash or NCC condition is imposed.

Table 21: Validation performance by source. BA averages recall over the classes represented in each source. The contrast is Removal minus Full; intervals use ten paired seeds. Source-wise macro averages are not additive contributions to pooled BA.

Removal-minus-Full reverses sign, becoming positive on the 4,551-row non-KVASIR view. Source-only-minus-Random is also positive there despite being negative on public validation. Table 21 reports source-specific BA over the classes represented in each source; these macro averages are not additive contributions to pooled BA. KVASIR lacks Worms; SEE-AI lacks Ulcer and Worms. KID represents Angioectasia, Bleeding, Lymphangiectasia, Normal and Polyp, while public AIIMS represents Ulcer and Worms.

SourcennClassesFullRemovalΔ\Delta95% paired-tt
KVASIR11,58190.9430.470−0.4730[−0.4817, −0.4644]
SEE-AI4,28980.6970.710+0.0129[+0.0013, +0.0245]
KID16550.8120.768−0.0443[−0.0863, −0.0022]
AIIMS9720.9290.951+0.0215[+0.0083, +0.0346]

Table 21.

The 4,326-row internal-filtered view further excludes within-source candidate matches. Its Removal-minus-Full contrast is +0.0051 ([−0.0043, +0.0145]), and its Source-only-minus-Full contrast is +0.0037 ([−0.0049, +0.0122]). Both intervals span zero. Filtering defines diagnostic populations and does not establish patient independence.

Table 22: Per-class mean recall on public validation over ten seeds. All classes are reported. Removal-minus-Full recall differences average to the pooled BA contrast.

The class-count-matched pools drawn from Full have lower non-KVASIR BA than Full: Comp and Class-match-R reach 0.388 and 0.391 BA, respectively, compared with 0.745 for Full. Thus a control can match class counts without preserving performance on another source mixture. Because the dominant source differs across classes in these pools (Appendix § 9.§ 2), source appearance can predict class; the non-KVASIR loss is consistent with this shortcut. The class-wise recall differences in Table 22 average to the pooled BA difference. The large Ulcer change accompanies strong source/class association and sparse non-KVASIR support, so it cannot be attributed to class-specific content alone.

ClassRowsFullRemovalΔ\Delta recall
Angioectasia4970.7890.667−0.122
Bleeding3590.8550.842−0.014
Erosion1,1550.7040.657−0.047
Erythema2970.5580.403−0.155
Foreign Body3400.8960.650−0.246
Lymphangiectasia3430.8250.592−0.233
Normal12,2870.9850.587−0.397
Polyp5000.6640.636−0.029
Ulcer2860.9760.096−0.879
Worms681.0000.991−0.009

Table 22.

On public validation, Full and Removal have mean AUCs 0.9879 and 0.8812, and combined scores 0.9066 and 0.7466 (change −0.1600). Selecting each run’s highest public-validation BA gives 0.8306 and 0.6178, a contrast of −0.2129 compared with −0.2132 at epoch ten. This selected-epoch comparison uses validation outcomes for both selection and scoring and is therefore descriptive.

Frozen-backbone CNN comparisons

ResNet-50 and ConvNeXt-Tiny use frozen ImageNet-pretrained backbones held in evaluation mode and the same 256-unit nonlinear head structure described above. TorchVision weight identifiers are IMAGENET1K_V2 and IMAGENET1K_V1, respectively. Their input transform resizes the short side to 256 with bilinear interpolation and antialiasing enabled, center-crops to 224 and applies ImageNet normalization. Training uses FP16 autocast for ResNet-50 and BF16 for ConvNeXt-Tiny; evaluation runs without autocast. AdamW uses learning rate 3×10−43 \times10^{-4}, weight decay 10−410^{-4}, the same class-weighted loss and batch size 128. Cosine decay spans the ten-epoch update budget. Each backbone is trained with ten seeds on full training, source-only training and the matched random pool and evaluated at epoch ten, giving 60 model fits. Within each seed, conditions use the same physical A100. BatchNorm buffers remain fixed, stochastic depth is inactive, and the schedule advances after successful mixed-precision updates.

Table 23: Source sensitivity with frozen CNN backbones and nonlinear heads. Full uses 37,607 training rows; Random and Source-only each use 10,596. Values are ten-seed mean BA; Δ\Delta is Source-only minus Random.

BackboneValidationFullRandomSource-onlyΔ\Delta
ResNet-50Public0.6970.6580.458−0.2002
ResNet-50Non-KVASIR0.5780.5220.610+0.0878
ConvNeXt-TinyPublic0.6920.6450.436−0.2096
ConvNeXt-TinyNon-KVASIR0.5620.5070.602+0.0955

Table 23. Source sensitivity with frozen CNN backbones and nonlinear heads.

Both CNNs have negative Source-only-minus-Random BA contrasts on public validation and positive contrasts on non-KVASIR validation (Table 23). For ResNet-50, the corresponding 95% paired-tt intervals are [−0.2024, −0.1979] and [+0.0836, +0.0921]; for ConvNeXt-Tiny, they are [−0.2137, −0.2055] and [+0.0919, +0.0990]. Thus, the direction of this contrast depends on the evaluation population in both frozen-backbone configurations as well as the adapted transformer. Their absolute scores do not provide a controlled architecture comparison because adaptation and preprocessing differ.

Sensitivity to removal rules and supervision

These controls test how removing source-attributed CV2024 training rows or changing their labels affects performance. They use the DINOv2-L/14 head/LoRA architecture, adapter initialization, image transform, loss and optimizer settings specified in Appendix C.1. Class weights follow Appendix C.1 for training lists with all ten classes; AIIMS removal lacks Worms training support, as discussed below.

The threshold, label and source-omission families use ten seed identities and final-epoch metrics on fixed evaluation lists; the matched add-back comparison uses four. GPU models differ across the source-omission and threshold experiments, so common seed labels do not imply matched hardware. At each epoch, these runs also evaluate Kvasir-Capsule’s split_1, mapped to CV2024 labels (Appendix F.1), without using it to select the reported epoch.

These results were aggregated from per-seed final-epoch records; complete per-image predictions are unavailable for these auxiliary runs. They are descriptive controls for the stated pipelines, not isolated causal contributions to the removal effect.

Hash and pixel removal thresholds

The pHash/dHash controls apply the hash screen, with its independent minima, at radii zero and two. The NCC controls remove selected source matches above the stated pixel cutoff. Both families remove only KVASIR training rows and retain all 10,596 other-source rows. Table 24 reports their training populations. Lowering a pixel cutoff removes more rows, changing size, class counts and source content together. NCC-80 retains two KVASIR rows as well as the five other-source rows absent from Removal, so the difference from Removal is seven training rows.

Removal ruleTrain rowsKVASIR keptPublic BA
Full37,60727,0110.825±0.0080.825 \pm0.008
dp,dd≤0d_p, d_d \le021,24110,6450.813±0.0070.813 \pm0.007
dp,dd≤2d_p, d_d \le210,8252290.712±0.0080.712 \pm0.008
NCC ≥0.99\ge0.9912,6972,1010.744±0.0080.744 \pm0.008
NCC ≥0.95\ge0.9510,7151190.699±0.0040.699 \pm0.004
NCC ≥0.90\ge0.9010,613170.654±0.0080.654 \pm0.008
NCC ≥0.85\ge0.8510,60590.618±0.0080.618 \pm0.008
NCC ≥0.80\ge0.8010,59820.615±0.0100.615 \pm0.010
Source-only10,59600.611±0.0080.611 \pm0.008
Removal10,59100.612±0.0070.612 \pm0.007

Table 24. Training-data removal thresholds for DINOv2-L/14, evaluated on all 16,132 public-validation rows. BA is the epoch-ten mean ±\pm sample SD over seeds 0–9. dpd_p and ddd_d are independently minimized pHash and dHash distances to the source reference; NCC uses the selected source counterpart. Full, Source-only and Removal are this family’s own reference runs. Appendix § 10.§ 1 describes these auxiliary experiments and their hardware differences.

The hash arms use L40S GPUs, the NCC arms use A100 or RTX A6000 GPUs, and Source-only uses RTX A6000; Full and Removal use A100. Absolute scores are therefore not a common-hardware comparison with the primary results. The family’s Full, Removal and Source-only references are its own run records, distinct from the primary fits in Appendix § 9; their means nevertheless agree with the primary means to three decimals.

Within the threshold-control training family, the NCC-90 BA loss relative to the family’s Full reference is approximately 80% of the Removal loss. This percentage is an arithmetic comparison of fixed-list outcomes, not a decomposition of leakage. Nominally ordered cutoffs do not guarantee monotone source-wise performance.

Label assignment and source removal

The reassignment condition uses ten fixed seed-specific lists. Each preserves all 37,607 image paths and all 10,596 non-KVASIR labels, but changes all 27,011 KVASIR labels to different CV2024 classes. All ten classes occur as replacement labels, and the KVASIR histogram changes. The sampling procedure for these lists is unavailable, so results are conditional on those assignments. The permutation control instead permutes the KVASIR one-hot label rows, preserving the image list and class histogram. This permutation still lowers public BA to approximately 0.504 (Table 25). Initialization and label corruption share a seed, so seed variation includes both choices. This contrast establishes sensitivity to supervision while leaving instance memorization unidentified.

Label controlHistogram keptPublic BANon-KVASIR BA
ReassignmentNo0.414±0.0060.414 \pm0.0060.716±0.0100.716 \pm0.010
PermutationYes0.504±0.0100.504 \pm0.0100.736±0.0110.736 \pm0.011

Table 25. Sensitivity to KVASIR label assignment. Each model retains all 37,607 training paths; only the labels of the 27,011 KVASIR rows change. BA is mean ±\pm sample SD at epoch ten over ten seeds.

Removing SEE-AI or KID also lowers performance on the corresponding held-out source (Table 26). These experiments differ from scoring the KVASIR-removal models separately by source: each changes the training source itself. Class support differs across these experiments, and there is no matched-size control for each removed source. Source-omission runs use A100, RTX A6000 or L40S GPUs. AIIMS removal eliminates the sole Worms training support and is therefore not treated as a directly comparable source-generalization test. The results support a broader source-dependence interpretation without separating domain familiarity from image-level exposure.

Omitted sourceTrain rowsEval. rowsClassesFull BAOmitted BA
KVASIR10,59611,58190.9420.466 ± 0.015
SEE-AI27,6114,28980.6970.239 ± 0.008
KID37,23116550.8120.495 ± 0.057

Table 26. Held-out-source BA after removing that source from training. Each row is a distinct training intervention; evaluation uses only the omitted source’s public-validation rows. Full is the corresponding full-training reference from each experiment’s own runs, so its KVASIR value (0.942) differs in the third decimal from the primary runs in Table 21 (0.943). Omitted BA reports epoch-ten mean ± sample SD over ten seeds.

This control asks how much public BA returns when a small set of Kvasir-Capsule frames is restored to Source-only. Adding 284 Kvasir-Capsule rows to the 10,596-row Source-only pool raises public BA from 0.6101 to 0.7048 across four matched seeds (change +0.0947). The baseline and add-back arms share the DINOv2-L/14 head and rank-eight LoRA configuration, training script, optimizer settings, seeds 0–3, batch size 128, ten epochs and RTX A6000 hardware. The add-back arm also ran seeds 4–9, four on RTX A6000 and two on A100, without that paired baseline. Over all ten seeds it reaches mean public BA 0.703. Paired by seed label with the A100 Removal reference runs of Appendix D.1, it is higher by +0.091 (95% paired-tt interval [+0.084, +0.099]; positive in all ten seeds). That pool difference is 289 rows, the 284 added frames plus the five hash-screen rows that Removal excludes, and eight of the ten pairs mix GPU models. All added filenames already occur in CV2024 (200 in training and 84 in validation), and all are included in the related-source split_1 evaluation list. The construction excludes designated nearest-reference filenames, not every possible occurrence or match.

The result therefore describes re-exposure under a changed training pool. Because sample size, class weights and content all change, it does not measure unseen-source transfer or an isolated source-domain effect.

Local versus distant removal: probability and accuracy

Matched lists, training and estimand

Let E0E_0 be the initial 799 exposed validation rows, each with at least one exact-pHash training counterpart reaching NCC ≥0.99\ge0.99. Before testing stratum eligibility, define the common removal set as

R0={t:∃v∈E0, dp(v,t)=0∧NCC⁡(v,t)≥0.99}.R_0 = \{t : \exists v \in E_0,\ d_p(v,t) = 0 \land\operatorname{NCC}(v,t) \ge0.99\}.

The 1,715 qualifying pairs have 1,100 distinct training rows. Each removed row must belong to a pair that individually passes both conditions; expanding to all exact-pHash counterparts of E0E_0 would instead remove 1,563 rows. Every arm removes R0R_0, unchanged by subsequent eligibility filtering.

Eligibility requires enough distant training rows to match a stratum’s proposed local removal. One stratum requires 256 distant controls but has only 141 eligible rows, so its 211 exposed evaluation rows are excluded. Its 298 qualifying training counterparts remain in the common removal set, while its proposed local neighbors remain in every arm’s training list. Additional local and distant removals come only from eligible strata.

Within each video/class training list sorted by frame index and filename, local removal takes the immediately preceding and following positions around each removed counterpart, merges overlapping neighborhoods, and excludes the counterpart positions. Distant controls match the lengths of the resulting local runs with nonoverlapping windows drawn uniformly from currently valid windows, using seeded restarts. Eligible distant rows lie outside the local set and at least 50 frame indices from every exposed row in the stratum. They belong to no cross-split exact-pHash group and have no exposed counterpart for which both hash distances are at most six. Local rows have a median frame-index gap of two to the nearest exposed row and a 95th percentile of eight; six have a gap of at least 50. List adjacency therefore need not imply temporal proximity. The eligible strata supply 876 additional removals per arm, giving 1,976 removals in total.

Before eligibility filtering, unflagged validation rows were sampled without replacement to match the 799 exposed rows by video/class. After excluding the ineligible stratum from both groups, evaluation retains 588 exposed and 588 unflagged rows. Unflagged rows have no pHash in a cross-split exact-pHash group; this status does not establish absence of exposure. Each evaluation group contains 547 Normal, 19 Angioectasia, 15 Bleeding, four Erythema, two Foreign Body and one Lymphangiectasia row. Every minority class occurs in only one prefix. Conclusions concern these eligible groups with uneven class support.

The four arms are local removal and three fixed distant-control draws, numbered 101, 202 and 303. They use the DINOv2-L/14 head/LoRA architecture, preprocessing, class-weighted objective, optimization settings and batch size in Appendix C, with ten training epochs. The ten seeds are 4–9 and 14–17. All four arms within a seed use the same GPU model; seed blocks run on NVIDIA A100, RTX A6000 or L40S devices. The 40 model fits yield 47,040 epoch-ten prediction rows, including repeated evaluation of the same frames. The release includes these probabilities, the fixed evaluation and removal lists and the training code (Appendix H). Equation (3) first contrasts local removal with the mean of the three distant arms within each seed and evaluation group, then subtracts the unflagged contrast from the exposed contrast. It compares removal strategies and does not identify an exposure-versus-no-exposure or memorization effect.

Conditional inference and design scope

The design, seed set, control choices and endpoint criterion were selected with access to evaluation outcomes. Seeds 14–17 were run first, after earlier designs with different arms (one on these same seeds); seeds 4–9 were added later. Apart from one-epoch smoke runs, the retained records contain no other seed for the reported design. All intervals are exploratory, conditional on the selected lists and unadjusted for selection. Their 95% levels are nominal; coverage after outcome-informed design and endpoint selection is not established. The three distant-control draws remain fixed across seeds.

The endpoints are true-class probability, negative log-likelihood with probability floor 10−1210^{-12}, true-class probability minus the largest other-class probability, micro accuracy and macro recall over the six represented classes. Probability, margin and accuracy changes are expressed in percentage points where indicated. The probability contrast is −0.355 percentage points, with paired-tt interval [−0.486, −0.224]. The NLL difference-in-differences is positive and the margin difference-in-differences is negative across all ten seeds. These directions describe correlated changes in the same probability vectors, not independent confirmations. Accuracy and BA each have four negative and six zero seed contrasts, with paired-tt intervals spanning zero; the experiment does not establish an accuracy decrease.

Table 2 in the main text gives absolute endpoint levels and separates the two components of Eq. (3). The exposed local-minus-distant probability change is −0.321 percentage points, compared with +0.034 for matched-unflagged rows. The exposed component is negative in eight seeds and positive in two; its difference from the unflagged component is negative in all ten. Thus ten negative differences-in-differences do not imply a negative exposed-group contrast in every seed. All four absolute levels compare removal strategies; none is an unremoved-model baseline. Video/class matching does not equalize the observed endpoint levels.

Table 27 gives paired-tt intervals, restating the h95h_{95} half-widths of Table 1(e) as intervals and adding NLL and the probability margin; Table 28 gives bootstrap ranges. Seed resampling preserves all four arms within a seed. Prefix resampling jointly weights the 32 contributing prefixes across arms and evaluation groups while averaging the ten fixed seeds. Both resampling procedures use 10,000 draws. They condition on different components and are not independent confirmations; neither jointly integrates seed randomness, control selection and unknown patient membership. For accuracy and BA, seed-percentile intervals exclude zero, whereas the reported prefix ranges reach zero and the paired-tt intervals span it.

EndpointMean contrast95% paired-tt intervalSeed signs
True-class probability−0.00355-0.00355[−0.00486,−0.00224][-0.00486, -0.00224]10/0/0
NLL+0.00495+0.00495[+0.00329,+0.00660][+0.00329, +0.00660]0/0/10
Probability margin−0.00472-0.00472[−0.00647,−0.00297][-0.00647, -0.00297]10/0/0
Balanced accuracy−0.00194-0.00194[−0.00485,+0.00097][-0.00485, +0.00097]4/6/0
Accuracy−0.00079-0.00079[−0.00163,+0.00004][-0.00163, +0.00004]4/6/0

Table 27. Local-versus-distant removal: exposed-minus-unflagged contrasts in Eq. (3). Probability, probability-margin, balanced-accuracy and accuracy contrasts are in absolute score units, not percentage points; NLL uses natural logarithms. Seed signs count negative/zero/positive contrasts over ten paired seeds. Intervals are pointwise and do not adjust for design selection.

Endpoint95% seed percentile95% video-prefix percentile range
True-class probability[−0.00462,−0.00244][-0.00462, -0.00244][−0.00693,−0.00067][-0.00693, -0.00067]
NLL[+0.00357,+0.00632][+0.00357, +0.00632][+0.00092,+0.00931][+0.00092, +0.00931]
Probability margin[−0.00615,−0.00325][-0.00615, -0.00325][−0.00920,−0.00093][-0.00920, -0.00093]
Balanced accuracy[−0.00460,−0.00004][-0.00460, -0.00004][−0.00399,+0.00000][-0.00399, +0.00000]
Accuracy[−0.00153,−0.00017][-0.00153, -0.00017][−0.00161,+0.00000][-0.00161, +0.00000]

Table 28. Bootstrap ranges for local-versus-distant removal: exposed-minus-unflagged, local-minus-distant contrasts in Eq. (3). Probability, margin, balanced-accuracy and accuracy contrasts are in absolute score units, not percentage points; NLL uses natural logarithms. Ranges are pointwise and exploratory, without adjustment for design selection. Seed resampling retains the pairing of all four arms. Video-prefix resampling is joint across arms and evaluation strata, with training seeds averaged within each draw. For balanced accuracy, the seed range uses the fixed six classes, whereas the prefix range averages recall over the classes present in each draw.

The fixed-six-class BA statistic is undefined in 9,086 of 10,000 prefix-bootstrap draws (90.86%); only 914 retain all six classes. We therefore do not report a percentile interval for that fixed-class target under this resampling rule. The reported present-class BA range instead averages recall over the classes present in each draw, a different, draw-dependent target. Omitting absent classes does not recover fixed-six-class inference. Continuous endpoints and micro accuracy do not require this absent-class averaging rule. The 32 leave-one-prefix-out mean probability contrasts range from −0.390 to −0.215 percentage points; these are omission-specific means, not confidence limits. The three largest-magnitude signed prefix contributions sum to 85.0% of the signed net contrast. Normal contributes approximately 68%; these overlapping decompositions cannot be added.

Kvasir-Capsule lists and training-population sensitivities

The upstream lists define a separate evaluation setting from CV2024. We first state their overlap properties, then describe the grouped-split, frame-count and video-selection experiments that use them.

Official Kvasir-Capsule list scope

The official Kvasir-Capsule lists [15] split_0 and split_1 contain 23,061 and 24,100 rows, respectively, at the inspected repository revision.3 Each has 25 video prefixes, with 43 in their union and seven shared prefixes. These lists differ from the 47,238-entry class-folder inventory used for source matching; list-row totals are not counts of unique reference images. The second list has 24,092 unique filenames; the lists have no filename intersection or shared video/class pair, and the minimum cross-fold frame-index gap among frames from the same video is 201. Filename disjointness thus coexists with prefix overlap, without establishing adjacent same-class reuse or patient disjointness. The seven shared prefixes in these distributed lists differ from the video-disjoint description in Smedsrud et al. [2021] (Technical Validation); this observation does not establish which lists produced that article’s experimental results. These boundaries belong to Kvasir-Capsule, not the CV2024 public split or the AIIMS test; their overlap properties cannot be transferred between protocols.

Kvasir-Capsule split and training-population sensitivities

These experiments concern the upstream Kvasir-Capsule lists, rather than the ten-class CV2024 task. They illustrate why evaluation boundaries, represented-class support and epoch selection must accompany a reported score, using video grouping, training-set size and video selection as distinct design choices. The tables were generated from 28 run histories, with recorded counts and class supports checked against the split metadata.

Training procedure and recorded epochs. The models use DINOv2-L/14 with rank-eight LoRA on the last four attention blocks and a 256-unit nonlinear head. Inputs are resized directly to 224 × 224 with ImageNet normalization. Training uses batches of 128, class-weighted cross-entropy with label smoothing 0.05 and head/LoRA learning rates 3×10−4/10−43 \times10^{-4}/10^{-4}. Class weights follow the effective-number rule of Appendix § 9.§ 1 over the eleven classes, with ncn_c counted in the training list; a class absent from that list enters the normalizing sum with rc=0.001r_c = 0.001 and then receives weight zero. Unlike the primary pipeline, these runs use two rather than three head-only epochs, recreate AdamW each epoch and place the backbone in training mode during optimization. Epoch caps are 20 for grouped splitting and scaling and 12 for video selection, with evaluation-BA patience of five and four epochs, respectively. Terminal results use the last recorded epoch; selected results use the epoch with the highest evaluation BA in each history. Because evaluation controls stopping and selection, neither value is an untouched-test estimate. Means and sample standard deviations describe the two recorded runs per setting.

Video-grouped folds. The five folds partition the video prefixes in the combined official lists. BA averages recall over the 6–9 classes present in each test fold. The Blood class is absent from training in one fold. The two recorded runs per fold reset the training random generators to zero, so their labels do not denote independent initialization seeds. Their descriptive spread therefore cannot quantify seed uncertainty; overlapping training folds also preclude treating fold scores as independent dataset replications. Table 29 reports outcomes for each fold’s represented classes.

FoldTest rowsTest classesMissing train classTerminal BASelected best BA
09,4188None0.3959 ± 0.00020.4127 ± 0.0000
19,4219None0.2668 ± 0.00030.2869 ± 0.0003
29,4256None0.4105 ± 0.00130.4805 ± 0.0001
39,4759None0.3922 ± 0.00250.4392 ± 0.0001
49,4228Blood0.2726 ± 0.00000.2801 ± 0.0005

Table 29. Video-grouped evaluation on Kvasir-Capsule (zero-based folds). BA is mean ± descriptive across-record SD over two recorded runs. Terminal denotes the last recorded epoch; best is selected using evaluation BA. Evaluation BA also determines early stopping. Last recorded epochs: 10–17. BA averages recall over the classes present in each test fold; training sets overlap across folds.

Training-set size. Class-stratified samples of the official split_0 list are evaluated on split_1, which contains all eleven classes. Sampling seeds change the selected rows, while training resets its random generators to zero. The full-size condition consequently repeats the same list and initialization. Table 30 separates the two recorded values. Mean terminal BA is 0.236 at half size and 0.234 at full size in these recorded runs. Row count, update count and stopping epoch vary together; these runs do not identify a general scaling law.

FractionTrain rowsTerminal BASelected best BALast recorded epoch range
0.12,3020.1339 ± 0.01750.1666 ± 0.00577–11
0.255,7610.1846 ± 0.02060.2123 ± 0.00578–11
0.511,5280.2364 ± 0.00840.2501 ± 0.01257–9
1.023,0610.2339 ± 0.00010.2574 ± 0.000011

Table 30. Class-stratified training-frame sampling on Kvasir-Capsule. BA is mean ± descriptive across-record SD over two recorded runs. Terminal denotes the last recorded epoch; best is selected using evaluation BA. Evaluation BA also determines early stopping.

Video selection at a fixed row budget. The final experiment draws different numbers of split_0 videos and constructs 23,000 training rows by cyclic repetition or random truncation of their frames. Evaluation remains on split_1. Here the seed controls both video selection and training initialization. The selected frame pool and class coverage change with the video count; some training pools lack classes present in evaluation. The official lists also share video prefixes (Appendix § 12.§ 1). Mean terminal BA increases from 0.184 for the smallest video set to 0.240 for the largest (Table 31). This is an association under the stated sampling and stopping procedure, not an isolated video-count effect or evidence of patient-disjoint generalization.

VideosSelected pool rows (mean)Terminal BASelected best BALast recorded epoch range
54,255.50.1835 ± 0.04170.1987 ± 0.03985–7
108,744.00.1939 ± 0.02510.2313 ± 0.00099
1514,009.00.2094 ± 0.01620.2244 ± 0.01209–12
2017,289.00.2117 ± 0.00760.2401 ± 0.00175–8
2523,061.00.2400 ± 0.00790.2640 ± 0.000910

Table 31. Kvasir-Capsule video selection at a fixed budget of 23,000 effective training rows. BA is mean ± descriptive across-record SD over two recorded runs. Terminal denotes the last recorded epoch; best is selected using evaluation BA. Evaluation BA also determines early stopping. Pool size precedes repetition or truncation to the effective training list.

External screens and their scope

For ChestX-ray14, we inspect train_val_list.txt and test_list.txt, acquired from the alkzar90/NIH-Chest-X-ray-dataset mirror of the NIH release. Rechecking a fixed mirror revision reproduces the counts below; the original acquisition date and revision were not retained.4 The lists contain 86,524 and 25,596 filenames, with 28,008 and 2,797 patient prefixes. The filename and patient-prefix intersections are both empty. This complete-list check concerns identifiers in the release associated with the ChestX-ray dataset family [23]; pixel content and the correctness of patient identifiers are not independently checked.

The image-level screens use the 25,331-image ISIC 2019 training archive [24, 32] (Codella et al. 2018; Hernández-Pérez et al. 2024) and 10,662 labeled HyperKvasir images [25]. ISIC comparisons cross metadata strata within that inventory. BCN, HAM and MSK denote prefixes of the metadata field lesion_id; empty IDs form the unknown-lesion-ID stratum. These are metadata groups, not verified independent institutions. For HyperKvasir, we compare its labeled inventory with CV2024 KVASIR images. Neither comparison estimates train–validation prevalence, and the latter does not audit HyperKvasir’s cross-validation folds.

The sum rule (dp+dq≤6d_p + d_q \le6) yields three ISIC candidates and no HyperKvasir/CV2024 KVASIR candidate. The broader joint rule (dp≤6d_p \le6 and dq≤6d_q \le6) yields 67 and one, respectively (Table 32). Both rules score a single pair, unlike the hash screen’s independent minima. We evaluate every joint-rule candidate. Two ISIC pairs pass NCC ≥0.99\ge0.99 and the HyperKvasir pair fails under both 96×9696 \times96 bicubic and 256×256256 \times256 bilinear grayscale processing. Byte comparison establishes identity for one passing ISIC pair; similarity alone does not establish it for the other. Both link an empty lesion ID to an MSK-prefixed ID. These pair counts estimate neither image-level prevalence nor detector specificity.

Comparisondp+dd≤6d_p + d_d \le6dp,dd≤6d_p, d_d \le6NCC ≥.99\ge.99, 96NCC ≥.99\ge.99, 256
BCN / HAM0100
HAM / unknown lesion ID16400
Unknown lesion ID / MSK2222
Other ISIC combinations0000
ISIC total36722
HyperKvasir / CV2024 KVASIR0100

Table 32. External hash-candidate pairs and pixel similarity. The sum-rule and joint-rule columns score single pairs under different rules. Every joint-rule candidate was evaluated from original images. ISIC strata follow lesion-ID metadata, including an unknown-ID group; they are not verified institutions. NCC columns count joint-rule candidates passing the threshold using 96×9696 \times96 bicubic and 256×256256 \times256 bilinear grayscale images.

These checks cover ReuScope’s identifier and image-screening components. Pair counts here, affected validation-row counts and the cross-release removals reported by Cassidy et al. [2022] use different populations and units. Detector-recall comparisons also require aligned identifiers, annotations and search rules; a shared NCC threshold is insufficient. An empty candidate set does not exclude other forms of overlap.

Reproducibility, access and asset terms

Code and data access

The ReuScope repository at https://github.com/eunseobchoi/ReuScope provides the audit code, the training and inference code of the primary, local-removal, CNN and official-test runs, the released data and scripts that recompute reported results from them. Its REPRODUCE.md maps each released result to its producing code, inputs, outputs and checking command, and lists the analyses whose code is not released. Its data/results folder (5 MB) contains ordered primary training-pool and evaluation memberships; fixed local evaluation and removal lists; retained byte-identity, video-prefix and pair measurements; primary/CNN seed-level metric records; 47,040 local prediction rows with ten-class probabilities; official-test confusion matrices and class-specific AUCs; and derived public-entry scores. The data/predictions folder (127 MB) contains the saved epoch-ten probabilities of all 110 primary and 60 CNN fits on their evaluation lists. A Croissant 1.1 description (croissant.json) lists every released data file with its SHA-256.

The list representation preserves row order and repeated occurrences, with image paths relative to the public dataset root; an exporter writes the training and evaluation CSVs read by the reported runs, with identical rows and order. Participant submission workbooks are read from the challenge repository at a fixed revision rather than redistributed. Original images and pretrained weights remain with their providers. Model checkpoints, per-epoch histories (which give the best-epoch values in Section 3.3 and Appendix C), execution logs, code for the auxiliary training analyses (Appendix D) and Kvasir-Capsule experiments (Appendix F), and the code of the feature, transform, PDQ, video-prefix resampling and external-release screens (Appendix G) are not distributed; Appendix H.3 lists every omitted component.

Computational environment and compute

Primary DINOv2 and CNN runs record one visible A100 (80 GB) per fit. Local-removal runs configure one A100 (80 GB), RTX A6000 (48 GB) or L40S (48 GB) per seed. Additional CV2024 controls ran on these GPU models, as identified in the run records, without recorded device counts. Official-test runs used L40S (28 fits), A100 and RTX A6000 GPUs; the launcher requested one GPU per run and two A100s for Full seed 0, for which the producer uses data parallelism when several devices are visible, and the visible device count was not recorded. Kvasir-Capsule runs recorded neither GPU model nor count. Jobs ran on a shared university GPU cluster. Primary training uses PyTorch 2.6.0 [Ansel et al., 2024], CUDA 12.4 and cuDNN 9.1. Contemporaneous environment probes and launch records identify TorchVision [TorchVision maintainers and contributors, 2016] 0.21.0+cu124 and Pillow [Clark and Pillow contributors, 2026] 12.1.1 for the primary campaign; these are campaign-level records, not per-fit version fields. CNN runs record Python 3.10.18, NumPy 2.2.6, pandas 2.3.3 and scikit-learn [Pedregosa et al., 2011] 1.7.2. The released code computes AUCs with scikit-learn’s roc_auc_score. Numerical re-aggregation used Python 3.12.3, NumPy 1.26.4 [Harris et al., 2020], SciPy 1.17.1 [Virtanen et al., 2020], pandas 2.1.4 and scikit-learn 1.9.0, without loading model weights. Image-processing and CNN settings are in Appendices A and C. Identical seeds do not guarantee bitwise agreement across environments.

Table 33 reports elapsed execution time from the run records. Times include data loading and evaluation inside each run and exclude scheduler waiting. CNN runs allocated at most 3.3 GiB of GPU memory; official-test launchers requested 6–16 CPU cores and 80–256 GB of host memory. The public CV2024 training and validation images occupy about 0.4 GB.

Experiment familyRunsGPU (runs)Minutes per runHours
Primary DINOv2110A100 (110)10.7–32.222.23
Local removal40A6000 (24), A100 (8), L40S (8)28.8–64.724.34
Frozen-backbone CNN60A100 (60)3.0–8.65.32
Official test30L40S (28), A100 (1), A6000 (1)7.8–20.96.06
Auxiliary CV2024 controls154A100, A6000, L40S21.8 (mean)55.96
DINOv2 feature extraction1RTX 4070 Ti SUPER (1)9.50.16
Kvasir-Capsule28not recorded—≈4.8
Total423≈118.8

Table 33. Compute of the reported experiments. Minutes per run give the range of elapsed time for one run (the mean for the auxiliary controls); Hours give each family’s total elapsed execution time. GPU counts are as described in the text. Auxiliary CV2024 controls comprise the threshold, label, source-omission and add-back runs and their Full, Removal and Source-only references. The Kvasir-Capsule total and the overall total are estimates (see text).

Kvasir-Capsule runs timed only their training loops, 2.92 hours in total; the estimate in Table 33 adds evaluation and setup time, scaling each run’s own training throughput by the evaluation-to-training cost ratio of the 30 official-test runs (median 0.89) and adding the primary runs’ median per-run setup (9 s), which gives 4.77 hours. The maximum ratio (1.21) and setup (30 s) would give 5.57 hours and a project total of 119.6 hours. Their hardware was not recorded; they train DINOv2-L/14 with the batch sizes that ran on single 48 GB GPUs in the local experiments. Image measurements are CPU-bound: timing the released implementation on 800 images and 200 pairs on one core of an AMD Ryzen 7 7800X3D and extrapolating gives about 1.2 core-hours for all 105,362 images and 53,464 pairs. Replay, prediction recomputation and participant re-scoring each finish within a minute on one CPU core. The full project used more compute than Table 33: 60 additional CNN diagnostic fits (5.26 hours) and preliminary and failed runs that were not fully recorded.

Numerical replay and implementation scope

Install the repository’s pinned CPU dependencies and run, from the repository root,

python replay.py --output replay.json

with a new output filename. The command re-aggregates the main training and cohort comparisons from recorded metrics, recomputes local endpoints and seed/prefix intervals from saved probabilities, and aggregates official-test confusion matrices and class AUCs. It also recounts the included relationship measurements and SSIM control summaries. Then

python recompute.py --output recomputed.json

re-scores all 450 primary and CNN seed/view records from the saved probabilities and checks them, with the contrasts in Table 1(b), against the recorded values; Table 18, Table 19, Table 20, Table 22 and Table 23 follow from the same predictions, and Table 21 can be computed from their source labels. re-score.py re-scores the 25 readable participant workbooks fetched from the upstream repository for Table 1(d) and Table 13–Table 15. kvasir_splits.py, written for this release rather than taken from the original analysis, rechecks the official Kvasir-Capsule list properties of Appendix § 12.§ 1 from the upstream lists at the fixed revision. Official-test AUC replay averages stored class AUCs; it does not reconstruct their rankings from per-image scores. These commands verify saved outputs; they do not rerun training, inference or image measurement, and they do not regenerate the typeset tables, figures or paper.

reuse.py implements byte-confirmed SHA-256 groups, KVASIR prefix counts, independent pHash/dHash minima and 96×9696 \times96 exact-pHash counterpart NCC, with optional selected-pair SSIM. The minimum-distance function does not select reference identities. controls.py constructs Full, Source-only, Removal, Random, Comp, Class-match-R and Ulcer replacement A, plus the three evaluation views. Both modules encode CV2024’s four sources and ten classes; controls.py also requires the 66-row Ulcer sets of replacement A and a screen that flags every KVASIR training row. Another release requires replacing these constants. analysis.py provides pooled/equal-source scoring, paired-tt and seed-bootstrap contrasts, the local difference-in-differences and official-test aggregation. example.py uses synthetic inputs. The training directory contains the DINOv2 primary and local-removal recipes, the CNN runner and trainer, and the official-test producer, each with its recorded environment. The official-test producer is the archived file, the CNN trainer’s executable code is unchanged, and the DINOv2 recipes matched their archived producers on reduced CPU inputs in trainable tensors, random-number state, losses and epoch metrics. Seeds and data order are fixed, but mixed-precision GPU training is not guaranteed to be bitwise reproducible, even on identical hardware; the DINOv2 Torch Hub revision and frozen-weight digest of the original runs were not recorded, and the official-test producer saved no checkpoints or per-image predictions.

The release omits local/distant sampling code, construction code for controls B–D, PDQ and feature/transform audits, reference-counterpart selection (Table 6), the basename and joint-rule candidate searches (Tables 9–10), matched-SSIM control construction, the 256×256256 \times256 NCC implementation, video-prefix resampling (Table 12), the official-test byte scan, the external-release screens and code for the auxiliary CV2024 and Kvasir-Capsule training analyses. Retained outputs of the counterpart selection and candidate searches are included where available. The alternate-resolution result is included only as retained per-prefix counts. Kvasir-Capsule histories lack per-run hardware records and verified links to the code of each run.

New hash and pixel measurements require original images; hash-based candidate searches can use recorded measurements. Model training and inference additionally require the pretrained weights and a CUDA GPU; the official-test producer also reads the Kvasir-Capsule images and its split_1 list. Patient-level validation requires identifiers absent from the public metadata.

Example: a training-pool contrast

This example uses ReuScope’s record structure for a training comparison and complements the relationship record in § 4. Its ordered inputs and seed-level metric records are included in data/results.

Question and boundary. How does Full-to-Removal training change public-validation BA?

Inputs and metric. Full/Removal contain 37,607/10,591 training rows. Their models share the 16,132-row validation list; BA averages recall over the same ten classes, and seeds 0–9 are paired.

Estimate and uncertainty. Mean BA difference: −0.2132-0.2132 (95% paired-tt: [−0.2200-0.2200, −0.2065-0.2065]). This pointwise interval reflects seed variation conditional on fixed lists and pipeline, without adjustment for analysis selection.

Conclusion. Removal has lower public-validation BA under this pipeline; the comparison does not isolate the contribution of leakage.

Data, software and model terms

The public implementation’s LICENSE applies MIT to author-written code. Author-created measurements, predictions and derived results in the repository’s data folder use CC BY 4.0, as stated in its license notice. Adapted CV2024 row metadata retain provider attribution and terms. Neither license grants rights to upstream images or weights. Organizer code is excluded because the examined repository lacks a repository-level LICENSE; scoring follows this paper’s metric definitions, and participant workbooks are read from that repository rather than redistributed.

The examined CV2024 training/validation and test release records state CC BY 4.0 [10 b]; the test record also specifies research-only use. The challenge paper states research-only use for the datasets collectively [3]. Among its component sources, the SEE-AI dataset states CC BY 4.0 and asks users to cite its article [12], KID provides its datasets for academic research upon registration [13], and the AIIMS frames come from a private collection [14]; Kvasir-Capsule terms are given below. Kvasir-Capsule’s article and repository state CC BY 4.0, while the repository also includes competition/commercial permission wording. The ISIC 2019 training release is provided under CC BY-NC 4.0, with source credit to BCN_20000 (Department of Dermatology, Hospital Clínic de Barcelona), HAM10000 (ViDIR Group, Department of Dermatology, Medical University of Vienna) and MSK (Anonymous). HyperKvasir is provided under CC BY 4.0; its repository also requires prior written permission for competition or commercial use.5 Users obtain the original images under the applicable provider terms.

NIH’s dataset FAQ describes use as unrestricted and requests the original download link, NIH Clinical Center acknowledgment and the CVPR 2017 citation [23]. We acknowledge the NIH Clinical Center as the provider of the ChestX-ray14 identifier lists.6

DINOv2 code and pretrained weights use Apache 2.0. PyTorch, TorchVision, NumPy, SciPy, pandas, scikit-learn and scikit-image use BSD-3-Clause, subject to file-specific notices; ImageHash uses BSD-2-Clause, and Pillow uses MIT-CMU. The pdqhash Python bindings use MIT; underlying PDQ uses BSD-3-Clause.7

TorchVision’s ResNet-50 IMAGENET1K_V2 and ConvNeXt-Tiny IMAGENET1K_V1 checkpoints use ImageNet-1K pretraining [Deng et al., 2009, Russakovsky et al., 2015]. TorchVision notes that pretrained weights may carry training-data terms; ImageNet’s database conditions specify noncommercial research and education.8 Model identifiers are specified in Appendix § 9. Weights are obtained separately under provider terms, not redistributed or relicensed.

LLM assistance. LLMs assisted only with code and manuscript preparation; they are not components of ReuScope, the image models or the numerical and statistical calculations.

References

  1. [1]Sayash Kapoor and Arvind Narayanan. Leakage and the reproducibility crisis in machine-learning-based science. Patterns, 4(9):100804, 2023. doi: 10.1016/j.patter.2023.100804.DOI
  2. [2]Dewinda J. Rumala. How you split matters: Data leakage and subject characteristics studies in longitudinal brain MRI analysis. In Clinical Image-Based Procedures, Fairness of AI in Medical Imaging, and Ethical and Philosophical Issues in Medical Imaging, volume 14242 of Lecture Notes in Computer Science, pages 235–245, Cham, 2023. Springer Nature Switzerland. doi: 10.1007/978-3-031-45249-9_23.DOI
  3. [3]Palak Handa, Amirreza Mahbod, Florian Schwarzhans, Ramona Woitek, Nidhi Goel, Manas Dhir, Deepti Chhabra, Shreshtha Jha, Pallavi Sharma, Vijay Thakur, Simarpreet Singh Chawla, Deepak Gunjan, Jagadeesh Kakarla, and Balasubramanian Raman. Capsule Vision 2024 Challenge: Multi-class abnormality classification for video capsule endoscopy. arXiv:2408.04940, 2024. URL https://arxiv.org/abs/2408.04940v3.
  4. [4]Marcel Roth, Micha V. Nowak, Adrian Krenzer, and Frank Puppe. Domain-adaptive pre-training of self-supervised foundation models for medical image classification in gastrointestinal endoscopy. arXiv:2410.21302, 2024. URL https://arxiv.org/abs/2410.21302v4. Dataset name: EndoExtend24.
  5. [5]Sayash Kapoor, Emily M. Cantrell, Kenny Peng, Thanh Hien Pham, Christopher A. Bail, Odd Erik Gundersen, Jake M. Hofman, Jessica Hullman, Michael A. Lones, Momin M. Malik, Priyanka Nanayakkara, Russell A. Poldrack, Inoliwua Deborah Raji, Michael Roberts, Matthew J. Salganik, Marta Serra-Garcia, Brandon M. Stewart, Gilles Van de Wiele, and Arvind Narayanan. REFORMS: Consensus-based recommendations for machine-learning-based science. Science Advances, 10 (18):eadk3452, 2024. doi: 10.1126/sciadv.adk3452.DOI
  6. [6]Patrick Ramos, Ryan Ramos, and Noa Garcia. Data leakage in visual datasets. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pages 6368–6378, October 2025. doi: 10.1109/ICCVW69036.2025.000661. URL https://openaccess.thecvf.com/content/ICCV2025W/Findings/html/Ramos_Data_Leakage_in_Visual_Datasets_ICCVW_2025_paper.html.
  7. [7]Björn Barz and Joachim Denzler. Do we train on test data? Purging CIFAR of near-duplicates. Journal of Imaging, 6(6):41, 2020. doi: 10.3390/jimaging6060041.
  8. [8]Tuan Truong, Farnaz Khun Jush, and Matthias Lenga. Benchmarking pretrained vision embeddings for near- and duplicate detection in medical images. In 2024 IEEE International Symposium on Biomedical Imaging (ISBI), pages 1–5. IEEE, 2024. doi: 10.1109/ISBI56570.2024.10635550.DOI
  9. [9]Amelia Jiménez-Sánchez, Natalia-Rozalia Avlona, Dovile Juodelyte, Théo Sourget, Caroline Vang-Larsen, Anna Rogers, Hubert Dariusz Zając, and Veronika Cheplygina. Copycats: The many lives of a publicly available medical imaging dataset. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 113383–113404. Curran Associates, Inc., 2024. doi: 10.52202/079017-3602. URL https://proceedings.neurips.cc/paper_files/paper/2024/file/cdbeaeb8a0313940a5752c4ec8838ca6-Paper-Datasets_and_Benchmarks_Track.pdf.
  10. [10]Palak Handa, Amirreza Mahbod, Florian Schwarzhans, Ramona Woitek, Nidhi Goel, Deepti Chhabra, Shreshtha Jha, Pallavi Sharma, Vijay Thakur, Simarpreet Singh Chawla, Manas Dhir, Deepak Gunj an, Jagadeesh Kak arla, and Balasubramanian Raman. Training and Validation Dataset of Capsule Vision 2024 Challenge. Figshare dataset, January 2025a. URL https://figshare.com/articles/dataset/Training_and_Validation_Dataset_of_Capsule_Vision_2024_Challenge/26403469/3. Published 11 January 2025.
  11. [12]Akihito Yokote, Junji Umeno, Keisuke Kawasaki, Shin Fujioka, Yuta Fuyuno, Yuichi Matsuno, Yuichiro Yoshida, Noriyuki Imazu, Satoshi Miyazono, Tomohiko Moriyama, Takanari Kitazono, and Takehiro Torisu. Small bowel capsule endoscopy examination and open access database with artificial intelligence: The SEE-artificial intelligence project. DEN Open, 4(1):e258, 2024. doi: 10.1002/deo2.258.DOI
  12. [13]Anastasios Koulaouzidis, Dimitris K. Iakovidis, Diana E. Yung, Emanuele Rondonotti, Uri Koplov, John N. Plevris, Ervin Toth, Abraham Eliakim, Gabrielle Wurm Johansson, Wojciech Marlicz, Georgios Mavrogenis, Artur Nemeth, Henrik Thorlacius, and Gian Eugenio Tontini. KID project: An internet-based digital video atlas of capsule endoscopy for research purposes. Endoscopy International Open, 5(6):E477–E483, 2017. doi: 10.1055/s-0043-105488.DOI
  13. [14]Nidhi Goel, Samarjeet Kaur, Deepak Gunj an, and S. J. Mahapatra. Dilated CNN for abnormality detection in wireless capsule endoscopy images. Soft Computing, 26(3):1231–1247, 2022. doi: 10.1007/s00500-021-06546-y.DOI
  14. [15]Pia H. Smedsrud, Vajira Thambawita, Steven A. Hicks, Henrik Gjestang, Oda Olsen Nedrejord, Espen Næss, Hanna Borgli, Debesh Jha, Tor Jan Derek Berstad, Sigrun L. Eskeland, Mathias Lux, Håvard Espeland, Andreas Petlund, Duc Tien Dang Nguyen, Enrique Garcia-Ceja, Dag Johansen, Peter T. Schmidt, Ervin Toth, Hugo L. Hammer, Thomas de Lange, Michael A. Riegler, and Pål Halvorsen. Kvasir-Capsule, a video capsule endoscopy dataset. Scientific Data, 8(1):142, 2021. doi: 10.1038/s41597-021-00920-z. Dataset: https://doi.org/10.17605/OSF.IO/DV2AG.
  15. [16]Johannes Buchner. ImageHash: A Python perceptual image hashing module. Python package, version 4.3.2, 2025. URL https://github.com/JohannesBuchner/imagehash. Accessed 27 September 2026.
  16. [17]Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, April 2004. doi: 10.1109/TIP.2003.819861.
  17. [18]Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaeddin El-Nouby, Mahmoud As-sran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. DINOv2: Learning robust visual features without supervision. Transactions on Machine Learning Research, 2024. URL https://openreview.net/forum?id=a68SUt6zFt.
  18. [19]Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=nZeVKeeFYf9.arxiv.org/abs/2106.09685
  19. [20]Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, June 2016. doi: 10.1109/CVPR.2016.90. URL https://openaccess.thecvf.com/content_cvpr_2016/html/He_Deep_Residual_Learning_CVPR_2016_paper.html.DOI
  20. [21]Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A ConvNet for the 2020s. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11966–11976, June 2022. doi: 10.1109/CVPR52688.2022.01167. URL https://openaccess.thecvf.com/content/CVPR2022/html/Liu_A_ConvNet_for_the_2020s_CVPR_2022_paper.html.DOI
  21. [22]Lena Maier-Hein, Matthias Eismann, Annika Reinke, Sinan Onogur, Marko Stankovic, Patrick Scholz, Tal Arbel, Hrvoje Bogunovic, Andrew P. Bradley, Aaron Carass, Carolin Feldmann, Alejandro Frangi, Peter M. Full, Bram van Ginneken, Allan Hanbury, Katrin Haube, Michal Kozubek, Bennett A. Landman, Keno März, Oskar Maier, Klaus Maier-Hein, Bjoern H. Menze, Henning Müller, Peter F. Neher, Wiro Niessen, Nasir Rajpoot, Gregory C. Sharp, Korsuk Sirinukunwattana, Stefanie Speidel, Christian Stock, Danail Stoyanov, Abdel Aziz Taha, Fons van der Sommen, Ching-Wei Wang, Marc-André Weber, Guoyan Zheng, Pierre Jannin, and Annette Kopp-Schneider. Why rankings of biomedical image analysis competitions should be interpreted with care. Nature Communications, 9(1):5217, 2018. doi: 10.1038/s41467-018-07619-7.
  22. [23]Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, Mohammadhadi Bagheri, and Ronald M. Summers. ChestX-Ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3462–3471. IEEE, 2017. doi: 10.1109/CVPR.2017.369.DOI
  23. [24]International Skin Imaging Collaboration. ISIC 2019 Challenge: Training data. Challenge dataset, 25,331 training images, 2019. URL https://challenge.isic-archive.com/data/#2019. Accessed 27 September 2026.
  24. [25]Hanna Borgli, Vajira Thambawita, Pia H. Smedsrud, Steven Hicks, Debesh Jha, Sigrun L. Eskeland, Kristin Ranheim Randel, Konstantin Pogorelov, Mathias Lux, Duc Tien Dang Nguyen, Dag Johansen, Carsten Griwodz, Håkon K. Stensland, Enrique Garcia-Ceja, Peter T. Schmidt, Hugo L. Hammer, Michael A. Riegler, Pål Halvorsen, and Thomas de Lange. HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy. Scientific Data, 7(1):283, 2020. doi: 10.1038/s41597-020-00622-y.DOI
  25. [26]Antigone Davis and Guy Rosen. Open-sourcing photo- and video-matching technology to make the internet safer. Facebook Newsroom, August 2019. URL https://about.fb.com/news/2019/08/open-source-photo-video-matching/. PDQ technical documentation: https://github.com/facebook/ThreatExchange/blob/ec3671bafc70136c4178ac300b75281ee4c6a0c6/hashing/hashing.pdf.
  26. [27]Stéfan van der Walt, Johannes L. Schönberger, Juan Nunez-Iglesias, François Boulogne, Joshua D. Warner, Neil Yager, Emmanuelle Gouillart, Tony Yu, and the scikit-image contributors. scikit-image: image processing in Python. PeerJ, 2:e453, June 2014. ISSN 2167-8359. doi: 10.7717/peerj.453.DOI
  27. [28]Bradley Efron. Better bootstrap confidence intervals. Journal of the American Statistical Association, 82(397):171–185, 1987. doi: 10.1080/01621459.1987.10478410.DOI
  28. [29]Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=Bkg6RiCqY7.
  29. [30]Yin Cui, Menglin Jia, Tsung-Yi Lin, Yang Song, and Serge Belongie. Class-balanced loss based on effective number of samples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9268–9277, June 2019. URL https://openaccess.thecvf.com/content/CVPR_2019/html/Cui_Class-Balanced_Loss_Based_on_Effective_Number_of_Samples_CVPR_2019_paper.html.arxiv.org/abs/1901.05555
  30. [31]Wes McKinney. Data structures for statistical computing in Python. In Proceedings of the 9th Python in Science Conference, pages 56–61, 2010. doi: 10.25080/Majora-92bf1922-00a.DOI
  31. [32]Philipp Tschandl, Cliff Rosendahl, and Harald Kittler. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data, 5(1):180161, 2018. doi: 10.1038/sdata.2018.161.DOI

Paper details

vlnc:W_q43atm3t
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents