betaFeedback

WorksComputer Science

Vision-language classification of Torsades de Pointes risk from pseudo-coloured ECG images: do T-wave morphology and colour context help or hinder?

Abstract

Contextual prompting is widely presented as a cheap route to better performance from vision models, but the evidence comes largely from benchmarks in which an attribute named in a prompt is also present in the image, confounding the two channels. We investigate the significance of context for a semantically complex biomedical signal classification task using a domain-adapted vision-language model, BiomedCLIP. Building on our previous work that renders electrocardiogram (ECG) signals as colour-enhanced images to flag Torsades de Pointes risk using the QT-nomogram and pseudo-colouring technique, we cross four context strategies with four T-wave morphology label sets, evaluated zero-shot and with a class-balanced linear probe, under five-fold stratified cross-validation on 9,789 images. Zero-shot inference is unusable. Once trained, the colour-enhanced images outperform the colourless ones on every metric: balanced accuracy 0.903 versus 0.825, sensitivity 0.914 versus 0.827, specificity 0.892 versus 0.823, ROC-AUC 0.955 versus 0.888 and Matthews correlation coefficient (MCC) 0.437 versus 0.311. Crossing colour-in-image with colour-in-prompt separates the confounded channels: the image contributes +0.126 MCC (+0.078 balanced accuracy, +0.067 ROC-AUC) whether or not the prompt names colour, whereas naming it changes nothing under any metric. Context is two-sided: the best prompt set gains 0.008 MCC where colour is present and 0.016 where it is not, while the worst loses 0.017 and 0.013. For a signal rendered as an image, the leverage lies in how the signal is drawn rather than in how it is described - and where the endpoint is an interval measurement, prompts naming T-wave morphology describe the wrong quantity.

Introduction

Contextual prompting — supplying a vision–language model with text describing attributes of the scene — is a central strand of computer-vision research [1, 2, 3], and a contested one. One body of work treats contextual attributes as a meaningful contributor to recognition [1, 2, 4, 3]; another argues that context is largely redundant when appearance information is strong and valuable only when the visual signal is weak [5]. An intermediate position holds that the form context takes matters as much as its presence [6, 7]. Almost all of this evidence comes from benchmarks of common objects [8, 9], where the target is conspicuous.

Two things make that evidence hard to interpret. First, on such benchmarks the attribute named in the prompt is also visible in the image, so when performance improves it is not possible to say which channel did the work. Second, the labels are object identities, which prompts can name directly; many applied tasks are not like this.

Torsades de Pointes (TdP) risk assessment is not like this. TdP [10] is a potentially fatal polymorphic ventricular tachycardia associated with drug-induced prolongation of the QT interval. Whether a QT value is dangerous depends on heart rate, so the decision is made with a QT nomogram, which plots QT against heart rate and separates an at-risk region above a threshold line [11]. The label is thus a statement about interval duration relative to a rate-dependent threshold — not about waveform shape. Alahmadi et al. [12, 13] built an explainable algorithm around this rule and made its decision visible by pseudo-colouring: the area under the repolarisation curve is filled with a colour scale anchored so that the nomogram’s threshold for the beat’s own heart rate is rendered dark orange, with anything beyond it red [14]. The same beats are also available without colour.

This dataset supports a controlled test that the usual benchmarks do not. Colour can be present or absent in the image (the two renderings) and named or not named in the prompt (a colour label set), and crossing the two separates the confounded channels: if the discriminative work is done by the text, performance should track whether colour is named; if by the image, it should track whether colour is present. Separately, because the reference algorithm was deliberately built to be robust to T-wave morphology — QT-prolonging drugs distort T-wave shape, which is what invalidates automated QT measurement [12, 15] — morphological description is a proxy for the label rather than a statement of it.

We therefore ask: (RQ1) does morphological context help or hinder a classifier whose labels are morphology-independent, and does that depend on whether the label-defining cue is visible? (RQ2) when a vision–language model appears to use a visual attribute named in a prompt (e.g., colour), is that attribute read from the image or carried by the text?

Contributions: We (1) formulate nomogram-defined TdP risk detection from pseudo-coloured ECG images as a context-driven vision–language problem using BiomedCLIP [16]; (2) evaluate a factorial design of four context strategies ×\times four morphology label sets, zero-shot and trained, under five-fold stratified cross-validation on both renderings; (3) show that the colour signal is read from the image and that naming it in text adds nothing, in either rendering and under every metric; and (4) show that morphological context helps only where the label-defining overlay is absent, that single-word morphological categories are the weakest prompts tested, and that the gains are operating-point gains rather than improvements in separability.

Related work

Wang and Zhu [3] distinguish appearance context (colours and shapes) from semantic context (meaning carried by a description), and separate levels of use from prior knowledge through global to local; Torralba et al. [17] make a parallel case for spatial context. Their taxonomy maps onto our design: the colour overlay and T-wave shape are appearance context, the prompts are semantic context. Wang and Zhu also warn that bad context can hurt as much as good context helps. Against this, Wolf and Bileschi [5] argue that high-level context is redundant when appearance evidence is strong and useful only when it is weak, and Zhang et al. [7] show that different in-context examples produce markedly different results.

PerceptionCLIP [1] is the closest methodological antecedent: it enriches CLIP [18] prompts with contextual attributes and uses similarity scores to approximate the conditional probabilities needed for perception-based classification, reporting gains across datasets from ImageNet [9] to CelebA [19] and CUB-200 [20], while noting CLIP’s sensitivity to the exact text used — the motivation for our four label sets. Alternatives externalise reasoning to a language model [21, 22]. Ge et al. [4] and Lu et al. [6] find that prompt semantics and prompt structure govern how strongly text binds to visual entities, and Zhang et al. [23] use region-referring prompts with multimodal language models [24]. Godec et al. [2] introduce virtual classes that collapse to one decision, which we adopt for our multi-prompt label sets.

In medical imaging, Zhao et al. [25] find CLIP-driven applications underperform because CLIP is pre-trained on natural images, the usual remedy being domain pre-training. We therefore use BiomedCLIP [16], pre-trained on fifteen million biomedical image–text pairs; Cardiac-CLIP [26] is trained on 3D cardiac CT and is too distant from a rendered ECG. On the clinical side, the QT nomogram [11] defines our labels, pseudo-colouring [14, 12, 13] produces the coloured rendering, and Lester and Paglialunga [27] supply the morphological vocabulary used in our prompts.

What is missing from this literature is a setting in which the strength of the appearance cue can be varied deliberately while labels and signals are held fixed. That is what follows.

Materials and methods

Data and labels

The collection comprises 9,792 PNG images, each showing a single beat spanning one R-to-R interval, in two subsets: pseudo-coloured and colourless. The underlying signals derive from a randomised clinical trial of four QT-active drugs and placebo in healthy subjects [28, 29], the corpus used to develop the source algorithm of pseudoc-colouring the ECGS [12].

The classification of TdP risk is based on the QT-nomogram, using the clinical trials reported values of the uncorrected QT interval (QT) that is read against the observed heart rate on the nomogram of Chan et al. [11], and the beat is labelled high risk if the pair falls on or above the threshold line and low risk otherwise. High risk is the positive class. Analyses use the images the pipeline processed successfully: 5,035 pseudo-coloured (181 high-risk, 4,854 low-risk) and 4,754 colourless (181 high-risk, 4,573 low-risk), 9,789 in total. Both subsets are strongly imbalanced, at 3.6–3.8% positives. Images were opened with Pillow [30], converted to RGB and passed through the BiomedCLIP validation transform (224 ×\times 224); no other preprocessing was applied, so any difference between conditions is attributable to the prompts.

The pseudo-colour overlay

Sequential time is added to each beat as a third dimension from approximately the onset of the Q wave, and the area under the curve is filled with a spectrum-approximation colour sequence in which each successive 40 ms — one small square on the standard ECG grid — is mapped to one of nine colour indices [14, 12]. The scale is anchored to the nomogram, not to the beat: the QT value the nomogram gives as the risk threshold for that beat’s heart rate is index seven, rendered dark orange; indices eight and nine (40 and 80 ms beyond) are red and dark red; indices six to one run back through orange, yellow, lime, green, blue and purple (Figure 1).

The nine-index pseudo-colour scale

Figure 1. The nine-index pseudo-colour scale, with swatches sampled from the dataset images. Each index spans 40 ms. Index seven is the QT value the nomogram gives as the risk threshold for the beat’s own heart rate; indices eight and nine lie beyond it. Because the anchor moves with heart rate the scale encodes a rate-normalised judgement, and because the window is fixed the whole scale appears on every beat [14, 12].

Three properties matter. (i) The coloured window is fixed at 9 ×\times 40 = 360 ms regardless of the beat’s QT, so every image of either class contains the whole scale; the classes are distinguished by where the T wave sits on it (Figure 2). (ii) Because the anchor moves with heart rate, the overlay presents an already rate-normalised judgement, whereas recovering the same decision from the colourless rendering requires recovering both interval and rate. (iii) The overlay is a rendering of the labelling rule, so the coloured condition asks whether the model can read a clinical explainability overlay while the colourless condition asks whether it can infer the decision from waveform appearance. The performance gap between subsets should be read in that light rather than as a difference in task difficulty.

High-risk and low-risk beats with the pseudo-colour overlay

(a) High-risk (left) and low-risk (right) beats with the pseudo-colour overlay.

The same ECGs without pseudo-colour

(b) The same ECGs without pseudo-colour.

Figure 2. Example of single heartbeat ECG. In the high-risk example the repolarisation is still elevated at the dark-orange and red indices; in the low-risk example the T wave has returned to baseline while still in the cool part of the scale.

Prompts and context strategies

Four morphology label sets of increasing structural detail were defined as text prompts. No Label (2 prompts): “Normal”; “Abnormal”. L1 (2): “Normal T wave”; “Abnormal T wave”. L2 (6): “Normal” plus five single-word abnormality categories — “Flat”, “Large”, “Inverted”, “Biphasic” and “Notched” — supplied exactly as listed, with no accompanying description (Figure 3). L3 (10): descriptive sentences naming both the abnormality and its reason, e.g. “Abnormal large T-wave, broader and larger than initial signal”; “Abnormal T-wave with a deep inversion”. Following Godec et al. [2], every non-Normal prompt resolves to the positive class, keeping the decision binary while directing attention to different properties of the trace. Morphology is a proxy channel: QT-prolonging drugs alter T-wave shape in drug-specific ways [15, 27], so morphological description carries information about drug exposure and hence indirectly about prolongation, but never defines the label.

Examples of pseudo-coloured ECGs with T-wave morphologies

Figure 3. Examples of pseudo-coloured ECGs with the T-wave morphologies named in label set L2 and described in L3. “Large” denotes a T-wave taller or broader than usual.

A separate two-prompt colour set names the overlay: “a T wave with a high amount of blue/green” (low risk) and “a T wave with a high amount of orange/red” (high risk). Because the colouring fills the area under the curve, a prolonged T wave is literally filled with warm colour, so localising the description to the T wave makes the prompt approximately correct; what it does not express is the threshold structure of the scale.

Crossing the two sets gives four context strategies: Cˉ+M\bar{C} + M (morphology only), C+MˉC + \bar{M} (colour only), C+MC + M (both) and Cˉ+Mˉ\bar{C} + \bar{M} (neither, the no-context baseline). Colour Cˉ\bar{C} strategies were retained on the colourless subset as a negative control. Crossing strategies with renderings gives the 2×22 \times2 design — colour present or absent in the image, named or not named in the prompt — that answers RQ2.

Model, fusion and training

BiomedCLIP [16] was loaded from the Hugging Face Hub [31] through open_clip [32] and used frozen throughout; no weights are updated. Image and text embeddings are L2L_2-normalised, and scaled cosine similarities are softmaxed into two distributions per image: PmP_m over morphology prompts and PcP_c over the two colour prompts. The C + M strategy fuses them with a weight W=αP+(1−α)W = \alpha P + (1 - \alpha) computed from the colour distribution (α=0.5\alpha= 0.5, fixed a priori): prompts containing “Normal” are multiplied by Wblue/greenW_{\mathrm{blue/green}} and “Abnormal” prompts by Worange/redW_{\mathrm{orange/red}}, and the result is renormalised. With α=0.5\alpha= 0.5 the factor lies in [0.5,1.0][0.5, 1.0], so colour can modulate but never override morphology.

Experiment 1 uses these distributions directly for zero-shot prediction. Experiment 2 concatenates the normalised image embedding with the strategy’s probability vector [33], Xconcat=[Xemb,Pstrategy]X_{\mathrm{concat}} = [X_{\mathrm{emb}}, P_{\mathrm{strategy}}], and fits a logistic-regression probe [34] with class_weight="balanced", following the linear-probe protocol of Radford et al. [18]. Under Cˉ+Mˉ\bar{\mathrm{C}} + \bar{\mathrm{M}} the appended block carries no contextual information, making the strategy comparison a clean test of what prompts contribute. Five-fold stratified cross-validation preserves class proportions (≈36\approx36 positives per fold).

Evaluation

Metrics are derived from the confusion matrix [35]: sensitivity and specificity [36], balanced accuracy, positive predictive value (PPV), F1 [37], matthews correlation coefficient (MCC) [38, 39] and ROC-AUC [40]. Sensitivity, specificity, balanced accuracy and ROC-AUC are prevalence-independent and at under 4% positives cannot register the false-alarm burden, which PPV and MCC can. Reported pp-values are from paired tests over the five folds and Per-fold results and full confusion matrices are given in the Supplementary Material (Table S1–Table S14).

SetStrategyTPFNTNFPSens.Spec.BAPPVF1MCCAUC
No LabelC+M\mathrm{C}+\mathrm{M}1592285040040.880.180.530.040.070.030.52
No LabelC+Mˉ\mathrm{C}+\bar{M}175697238820.970.200.580.040.080.080.58
No LabelCˉ+M\bar{C}+\mathrm{M}14932101738370.820.210.520.040.070.010.51
No LabelCˉ+Mˉ\bar{C}+\bar{M}0181485400.001.000.500.000.000.000.50
L1C+M\mathrm{C}+\mathrm{M}9784302818260.540.620.580.050.090.060.57
L1C+Mˉ\mathrm{C}+\bar{M}175697238820.970.200.580.040.080.080.58
L1Cˉ+M\bar{C}+\mathrm{M}5612538899650.310.800.560.050.090.050.55
L1Cˉ+Mˉ\bar{C}+\bar{M}0181485400.001.000.500.000.000.000.50
L2C+M\mathrm{C}+\mathrm{M}1810148531.000.000.500.040.070.000.50
L2C+Mˉ\mathrm{C}+\bar{M}175697238820.970.200.580.040.080.080.58
L2Cˉ+M\bar{C}+\mathrm{M}1810248521.000.000.500.040.070.000.50
L2Cˉ+Mˉ\bar{C}+\bar{M}0181485400.001.000.500.000.000.000.50
L3C+M\mathrm{C}+\mathrm{M}4713441147400.260.850.550.060.100.050.55
L3C+Mˉ\mathrm{C}+\bar{M}175697238820.970.200.580.040.080.080.58
L3Cˉ+M\bar{C}+\mathrm{M}2415744084460.130.910.520.050.070.030.52
L3Cˉ+Mˉ\bar{C}+\bar{M}0181485400.001.000.500.000.000.000.50

Table S1. Zero-shot results on the pseudo-coloured subset (181 high-risk, 4,854 low-risk).

SetStrategyTPFNTNFPSens.Spec.BAPPVF1MCCAUC
No LabelCˉ+Mˉ\bar{C} + \bar{M}3067481660.830.820.830.150.260.310.87
No LabelC+MˉC + \bar{M}3067421720.830.810.820.150.250.300.87
No LabelC+MC + M3067481660.830.820.830.150.260.310.87
No LabelCˉ+M\bar{C} + M3067511630.830.820.830.160.260.310.87
L1Cˉ+Mˉ\bar{C} + \bar{M}3067511630.830.820.830.160.260.310.87
L1C+MˉC + \bar{M}3067421720.830.810.820.150.250.300.87
L1C+MC + M3067521620.830.820.830.160.260.310.87
L1Cˉ+M\bar{C} + M3067511630.830.820.830.160.260.310.87
L2Cˉ+Mˉ\bar{C} + \bar{M}2797271870.750.800.770.130.220.250.85
L2C+MˉC + \bar{M}3067421720.830.810.820.150.250.300.87
L2C+MC + M2887261880.780.790.790.130.220.260.85
L2Cˉ+M\bar{C} + M3067511630.830.820.830.160.260.310.87
L3Cˉ+Mˉ\bar{C} + \bar{M}3157601540.860.830.850.170.280.330.88
L3C+MˉC + \bar{M}3067421720.830.810.820.150.250.300.87
L3C+MC + M3157641500.860.840.850.170.290.340.88
L3Cˉ+M\bar{C} + M3067511630.830.820.830.160.260.310.87

Table S14. Fold 5 — trained colourless subset.

Results

Zero-shot inference is unusable

Zero-shot performance (Table 1) is unusable on both renderings and highly sensitive to prompt formulation. The Cˉ+Mˉ\bar{\mathrm{C}} + \bar{\mathrm{M}} baseline labels every image low-risk; under L2 the morphology strategies label almost every image high-risk (5,033 of 5,035 on the coloured subset), giving a nominally perfect sensitivity against zero specificity. Balanced accuracy sits at or near 0.50 almost throughout and MCC near zero, i.e. at chance [38]. No prompt design rescues this, which motivates the trained regime.

SetStrategyPseudo-coloured Sens.Spec.BAPPVMCCAUCColourless Sens.Spec.BAPPVMCCAUC
No LabelCˉ+M\bar{C} + M0.820.210.520.040.010.510.960.100.530.040.040.52
No LabelC+MˉC + \bar{M}0.970.200.580.040.080.581.000.000.500.040.000.50
No LabelC+MC + M0.880.180.530.040.030.520.970.050.510.040.020.51
No LabelCˉ+Mˉ\bar{C} + \bar{M}0.001.000.500.000.000.500.001.000.500.000.000.50
L1Cˉ+M\bar{C} + M0.310.800.560.050.050.550.920.130.530.040.030.52
L1C+MˉC + \bar{M}0.970.200.580.040.080.581.000.000.500.040.000.50
L1C+MC + M0.540.620.580.050.060.570.970.030.500.04−0.000.49
L1Cˉ+Mˉ\bar{C} + \bar{M}0.001.000.500.000.000.500.001.000.500.000.000.50
L2Cˉ+M\bar{C} + M1.000.000.500.040.000.500.990.010.500.040.010.50
L2C+MˉC + \bar{M}0.970.200.580.040.080.581.000.000.500.040.000.50
L2C+MC + M1.000.000.500.040.000.500.990.010.500.040.000.50
L2Cˉ+Mˉ\bar{C} + \bar{M}0.001.000.500.000.000.500.001.000.500.000.000.50
L3Cˉ+M\bar{C} + M0.130.910.520.050.030.520.460.560.510.040.010.51
L3C+MˉC + \bar{M}0.970.200.580.040.080.581.000.000.500.040.000.50
L3C+MC + M0.260.850.550.060.050.550.800.240.520.040.020.52
L3Cˉ+Mˉ\bar{C} + \bar{M}0.001.000.500.000.000.500.001.000.500.000.000.50

Table 1. Zero-shot results (Experiment 1) on both renderings. Sens. is sensitivity, Spec. specificity, BA balanced accuracy, PPV positive predictive value, AUC the area under the ROC curve. Almost every configuration collapses: Cˉ+M\bar{C} + M predicts every image low-risk, while most others predict nearly every image high-risk. Balanced accuracy sits at or near 0.50 and PPV never exceeds 0.06, barely above the 3.6–3.8% base rate. Full confusion matrices are in Supplementary Tables S1–S2.

One row is informative rather than degenerate, and we return to it in § 4.3: the colour-only strategy C + M uses an identical prompt pair on both renderings, yet reaches balanced accuracy 0.58 and MCC 0.08 on coloured images while collapsing to 0.50 and 0.00 on colourless ones.

Training separates the classes; aggregated context does not help

Training changes the picture categorically (Table 2). Averaged over folds the pseudo-coloured model reaches sensitivity 0.914, specificity 0.892, balanced accuracy 0.903, PPV 0.240, MCC 0.437 and ROC-AUC 0.955; the colourless model reaches 0.827, 0.823, 0.825, 0.157, 0.311 and 0.888. No configuration collapses. Every metric falls on the colourless subset: sensitivity by 8.8 points, balanced accuracy by 7.8, MCC by 12.6 and ROC-AUC by 6.7.

SetStrategyPseudo-coloured Sens.Spec.BAPPVMCCAUCColourless Sens.Spec.BAPPVMCCAUC
No LabelCˉ+M\bar{C} + M0.9170.8930.9050.2420.4390.9560.8230.8210.8220.1550.3070.888
No LabelC+MˉC + \bar{M}0.9170.8910.9040.2390.4370.9540.8230.8220.8230.1550.3080.888
No LabelC+MC + M0.9170.8930.9050.2420.4390.9560.8290.8220.8260.1560.3110.888
No LabelCˉ+Mˉ\bar{C} + \bar{M}0.9170.8910.9040.2400.4370.9560.8290.8240.8260.1570.3120.890
L1Cˉ+M\bar{C} + M0.9110.8960.9040.2460.4420.9580.8290.8240.8260.1570.3120.888
L1C+MˉC + \bar{M}0.9170.8910.9040.2390.4370.9540.8230.8220.8230.1550.3080.888
L1C+MC + M0.9110.8950.9030.2460.4420.9580.8240.8240.8240.1560.3090.886
L1Cˉ+Mˉ\bar{C} + \bar{M}0.9170.8910.9040.2400.4370.9560.8290.8240.8260.1570.3120.890
L2Cˉ+M\bar{C} + M0.9060.8870.8960.2310.4240.9520.8180.8150.8170.1500.2990.882
L2C+MˉC + \bar{M}0.9170.8910.9040.2390.4370.9540.8230.8220.8230.1550.3080.888
L2C+MC + M0.9060.8850.8960.2280.4210.9520.8230.8150.8190.1500.3010.882
L2Cˉ+Mˉ\bar{C} + \bar{M}0.9170.8920.9040.2400.4380.9560.8290.8240.8260.1570.3120.890
L3Cˉ+M\bar{C} + M0.9170.8960.9060.2480.4460.9580.8350.8310.8330.1640.3220.894
L3C+MˉC + \bar{M}0.9170.8910.9040.2390.4370.9540.8230.8220.8230.1550.3080.888
L3C+MC + M0.9110.8950.9030.2440.4400.9560.8400.8340.8370.1670.3280.892
L3Cˉ+Mˉ\bar{C} + \bar{M}0.9170.8920.9040.2400.4380.9560.8290.8240.8260.1570.3120.890

Table 2. Trained results (Experiment 2), averaged over five stratified folds. Sens. is sensitivity, Spec. specificity, BA balanced accuracy, PPV positive predictive value, AUC the area under the ROC curve. Best configuration per rendering in bold. Strategies: Cˉ+M\bar{C} + M morphology only, C+MˉC + \bar{M} colour only, C+MC + M both, Cˉ+Mˉ\bar{C} + \bar{M} no context. Every configuration on the colour-enhanced rendering exceeds every configuration on the colourless one, on every metric.

Aggregated over all prompt designs, context does nothing. On the pseudo-coloured subset context-bearing configurations average MCC 0.437 against 0.437 for no-context (−0.001); on the colourless subset, 0.310 against 0.312 (−0.002). Both differences are far below the fold-to-fold spread of the baseline itself.

Selected well, context helps, but unevenly. The best pseudo-coloured configuration, L3 + Cˉ+M\bar{C} + M, reaches MCC 0.446±0.016 against a matched baseline of 0.438±0.014, but wins in only three folds of five (+0.008, p=0.41p = 0.41). The best colourless configuration, L3 + C + M, reaches 0.328 ± 0.013 against 0.312 ± 0.007, winning in all five folds (+0.016, p=0.035p = 0.035). The asymmetry is the RQ1 result: context is reliable where the appearance signal is weak and fold-dependent where it is strong.

Label-set means order identically on both renderings and are not monotone in detail: pseudo-coloured L3 0.440, L1 0.439, No Label 0.438, L2 0.430; colourless L3 0.318, L1 0.310, No Label 0.309, L2 0.305. L2 is the weakest set on both renderings and under balanced accuracy as well as MCC, despite sitting between L1 and L3 in detail. Finally, C + M̄ is identical across label sets (MCC 0.308 colourless) because it ignores morphology entirely, and it falls below the no-context baseline (0.312).

The colour signal comes from the image, not the prompt

Table 3 crosses colour in the image with colour in the prompt. The pattern is unambiguous and identical under all three metrics. Moving from colourless to coloured images gains 0.126 MCC, and gains almost exactly that whether or not the prompt mentions colour (+0.126 named, +0.127 not named); in balanced accuracy both gains are +0.078, in ROC-AUC both +0.067. Naming colour changes nothing: −0.001 MCC when the colour is there to be named (p=0.58p = 0.58) and −0.001 when it is not (p=0.75p = 0.75), with an interaction of −0.001. The entire colour effect is carried by the image.

Colour in the image
Colour in the promptPresentAbsentImage effect
MCC
Named0.436 (0.010)0.310 (0.013)+0.126
Not named0.438 (0.009)0.311 (0.009)+0.127
Prompt effect−0.001 (p=0.58p = 0.58)−0.001 (p=0.75p = 0.75)interaction −0.001
Balanced accuracy
Named0.903 (0.019)0.825 (0.006)+0.078
Not named0.903 (0.018)0.825 (0.007)+0.078
Prompt effect−0.001 (p=0.83p = 0.83)−0.001 (p=0.70p = 0.70)interaction +0.000
ROC-AUC
Named0.9550.888+0.067
Not named0.9560.889+0.067

Table 3. Colour in the image crossed with colour in the prompt, trained regime, averaged over all four label sets (SD over folds where shown). “Named” pools C + M and C + M; “not named” pools C̄ + M and C̄ + M̄. The pattern is identical under all three metrics.

The zero-shot regime makes the point more sharply, because there the prompt is the only thing standing between the encoder and the decision, and the colour-only prompt pair is identical across renderings. On coloured images it recovers 175 of 181 positives and, importantly, 972 true negatives (balanced accuracy 0.58, MCC 0.08, ROC-AUC 0.58). On colourless images the same prompt labels every one of the 4,754 images positive: not a single true negative (0.50, 0.00, 0.50). The prompt supplies no discriminative content of its own; it is a query that returns something only when the image can answer it.

This also constrains the colourless result above. Since naming colour has no main effect there under any metric, the advantage of L3 + C + M cannot be a general benefit of colour context and must be a label-set-specific interaction.

Discussion

Colour-enhanced images outperform colourless ones on every metric

The pseudo-colour overlay, not the prompt, is what separates the two classes. Once trained, the colour-enhanced rendering beats the colourless one on every measure we report: balanced accuracy 0.903 against 0.825, sensitivity 0.914 against 0.827, specificity 0.892 against 0.823, PPV 0.240 against 0.157, ROC-AUC 0.955 against 0.888 and MCC 0.437 against 0.311. The advantage is therefore not an artefact of one imbalance-sensitive statistic: it appears in threshold-free ranking (ROC-AUC), at the operating point (balanced accuracy, MCC), and in both error directions taken separately (sensitivity and specificity move together). This is what one should expect given § 3.2: the overlay is a rendering of the labelling rule itself, already normalised for heart rate, so the encoder is reading the answer rather than inferring it. The corollary is that the gap should not be read as the diagnostic value of colour in an ECG; it is the value of showing a model the output of the algorithm that generated its labels.

Text as query, image as evidence

RQ2 admits an unusually clean answer. The same colour prompt is discriminative when the image contains colour and degenerate when it does not, while the image contributes its full ≈ 0.126 MCC whether or not the prompt mentions colour. In a contrastive vision–language model, a prompt naming a visual attribute appears to function as a query over the image rather than as evidence about it: the text selects a direction in the joint embedding space, and whether anything discriminative lies along that direction is a fact about the image.

The methodological consequence generalises. A common pattern in context-driven work is to show that adding an attribute prompt improves performance and conclude the model is using the attribute. That does not follow, because the attribute is normally present in image and prompt at once. Where a paired rendering exists — images that do and do not contain the attribute, prompt held fixed — the confound is removable, and we would encourage this as a routine control.

It also reframes the negative result on the pseudo-coloured subset. That no prompt beats the baseline there is not evidence that BiomedCLIP fails to understand the overlay; the image-side evidence in Table 3 shows the encoder extracts it well. It is evidence that once extracted, there is nothing left for text to add — redundancy in Wolf and Bileschi’s [5] sense, demonstrated by direct manipulation rather than inferred from a ceiling.

A morphology-independent label bounds what morphology prompts can do

The usefulness of morphological context tracks whether the label-defining cue is visible. Where the overlay presents the rate-normalised nomogram decision, no strategy improves reliably on the baseline. Where it is removed, morphology is the only proxy available and the best configuration improves in all five folds. This is more specific than “context helps when appearance is weak”: removing the overlay removes the answer, so prompting can contribute only where the answer must be inferred.

The size of that contribution is bounded by the proxy relation. Because the source algorithm is deliberately robust to T-wave morphology [12], and because QT-prolonging drugs distort morphology in drug-specific ways [15], morphological description correlates with the label largely through drug exposure rather than through the quantity the label measures. A gain of 0.016 MCC is about what such a proxy should deliver.

This also explains L2, which underperforms every other set on both renderings despite sitting between L1 and L3 in detail. Prompt structure is part of the story [6, 1], but the task supplies a sharper one: L2 names morphological categories — flat, large, inverted, biphasic, notched — every one of which can occur on either side of the nomogram line, so as a set it carries little information about the quantity being predicted while directing attention toward shape. L3 smuggles extent back in (“broader and larger than initial signal”), and it is the language of extent, not the taxonomy, that tracks a prolonged repolarisation. L2 is not badly phrased so much as badly aimed. The practical implication is sharper than “prefer sentences to labels”: a prompt should name the attribute that defines the label, and a clinically standard taxonomy is no substitute when the taxonomy classifies something else. Notably, no prompt set tested here names the QT interval or the risk threshold at all.

It is worth stating plainly what context does when the colour cue is present, since this is the condition in which prompting is most often assumed to help. The answer is that it can only lose. Against the matched no-context baseline on colour-enhanced images, the best prompt set (L3 + C̄ + M) gains 0.008 MCC, 0.002 balanced accuracy and 0.002 ROC-AUC, while the worst (L2 + C + M) loses 0.017, 0.008 and 0.004 — the downside is roughly twice the upside on MCC and four times on balanced accuracy. Naming colour specifically, when colour is there to be named, is exactly neutral (C + M̄: MCC 0.437, balanced accuracy 0.904, ROC-AUC 0.954, against a baseline of 0.437, 0.904 and 0.956). Where the overlay already encodes the decision, then, the expected return on prompt engineering is zero and the variance is not: the safest prompt is none. Without colour the two sides are comparable (+0.016 and −0.013 MCC; +0.011 and −0.009 balanced accuracy), and the best configuration improves in all five folds, so it is only in the impoverished rendering that context is worth designing at all.

Operating-point gains, not better separability

One divergence deserves comment rather than suppression. The advantage of the best colourless configuration is +0.016 in MCC (5/5 folds, p=0.035p = 0.035), +0.011 in balanced accuracy (4/5, p=0.11p = 0.11) and +0.002 in ROC-AUC (2/5, p=0.62p = 0.62). An effect present at the operating point but not in the threshold-free ranking is not a contradiction; it is a claim about what the context does. ROC-AUC measures ranking, which morphology context does not measurably improve; MCC and balanced accuracy measure performance at the threshold the balanced probe selects, and there it helps. Context is shifting the decision boundary rather than improving separability. Accordingly we describe these as operating-point gains throughout, and we suggest that studies comparing prompt designs on imbalanced data report both metric families: a threshold-free metric alone would have shown nothing here, and a threshold-dependent metric alone would have overstated what was found.

Conclusion

Once trained, the colour-enhanced rendering outperforms the colourless one on every metric — balanced accuracy 0.903 against 0.825, sensitivity 0.914 against 0.827, specificity 0.892 against 0.823, ROC-AUC 0.955 against 0.888 and MCC 0.437 against 0.311 — confirming that the pseudo-colour overlay carries the discriminative signal. Aggregated over all prompt designs, context has no appreciable effect on either rendering (MCC 0.437 vs. 0.437 colour-enhanced and 0.310 vs. 0.312 colourless; balanced accuracy 0.903 vs. 0.904 and 0.825 vs. 0.826). It is nonetheless two-sided. On colour-enhanced images the best prompt set gains 0.008 MCC, 0.002 balanced accuracy and 0.002 ROC-AUC over the matched baseline while the worst loses 0.017, 0.008 and 0.004, so where the overlay already encodes the decision the downside of context exceeds its upside. Without colour the two sides are comparable (+0.016 and −0.013 MCC), the best configuration improves in all five folds, and context becomes worth designing.

CRediT authorship contribution statement

Dario Labrador Alonso: Methodology, Software, Formal analysis, Investigation, Data curation, Writing – original draft, Writing – review & editing. Alaa Alahmadi: Conceptualization, Resources, Data curation, Supervision, Writing – review & editing.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability

The colour-enhanced ECG images are produced and provided by the principal investigator (Alaa Alahmadi), and all data, code and analysis results will be made available on request.

Supplementary Material

Tables S1–S2 give the full zero-shot results, Tables S3–S4 the trained results averaged over folds, and Tables S5–S14 the individual folds.

Label sets (the Set column) are the four sets of T-wave morphology text prompts, of increasing structural detail:

SetPromptsContent
No Label2“Normal”; “Abnormal”. Bare class names, no morphological content.
L12“Normal T wave”; “Abnormal T wave”. Minimal context.
L26“Normal” plus the single-word categories “Flat”, “Large”, “Inverted”, “Biphasic”, “Notched”.
L310Descriptive sentences naming the abnormality and its reason, e.g. “Abnormal large T wave, broader and larger than initial signal”.

Strategies (the Strategy column) are the four ways context is admitted at inference: Cˉ+M\bar{C}+\mathrm{M} morphology prompts only, C+Mˉ\mathrm{C}+\bar{M} colour prompts only, C+M\mathrm{C}+\mathrm{M} both, Cˉ+Mˉ\bar{C}+\bar{M} neither (the no-context baseline). C\mathrm{C} denotes the colour label set and M\mathrm{M} the morphology label set; a bar denotes absence.

BA is balanced accuracy, the mean of sensitivity and specificity. All metrics except ROC-AUC are computed exactly from the integer confusion-matrix counts. Every prompt other than the Normal prompt maps to the positive (high-risk) class, so the decision remains binary in all label sets.

SetStrategyTPFNTNFPSens.Spec.BAPPVF1MCCAUC
No LabelC + M176523043430.970.050.510.040.070.020.51
No LabelC + Mˉ\bar{M}1810045731.000.000.500.040.070.000.50
No LabelCˉ\bar{C} + M174744641270.960.100.530.040.080.040.52
No LabelCˉ\bar{C} + Mˉ\bar{M}0181457300.001.000.500.000.000.000.50
L1C + M175614044330.970.030.500.040.07−0.000.49
L1C + Mˉ\bar{M}1810045731.000.000.500.040.070.000.50
L1Cˉ\bar{C} + M1671460139720.920.130.530.040.080.030.52
L1Cˉ\bar{C} + Mˉ\bar{M}0181457300.001.000.500.000.000.000.50
L2C + M18012645470.990.010.500.040.070.000.50
L2C + Mˉ\bar{M}1810045731.000.000.500.040.070.000.50
L2Cˉ\bar{C} + M18016545080.990.010.500.040.070.010.50
L2Cˉ\bar{C} + Mˉ\bar{M}0181457300.001.000.500.000.000.000.50
L3C + M14437111934540.800.240.520.040.080.020.52
L3C + Mˉ\bar{M}1810045731.000.000.500.040.070.000.50
L3Cˉ\bar{C} + M8497256820050.460.560.510.040.070.010.51
L3Cˉ\bar{C} + Mˉ\bar{M}018145730.001.000.500.000.000.000.50

Table S2. Zero-shot results on the colourless subset (181 high-risk, 4,573 low-risk).

SetStrategyTPFNTNFPSens.Spec.BAPPVF1MCCAUC
No LabelC + M33.23.0866.6104.20.920.890.900.240.380.4390.956
No LabelC + Mˉ\bar{M}33.23.0865.2105.60.920.890.900.240.380.4370.954
No LabelCˉ\bar{C} + M33.23.0866.6104.20.920.890.900.240.380.4390.956
No LabelCˉ\bar{C} + Mˉ\bar{M}33.23.0865.4105.40.920.890.900.240.380.4370.956
L1C + M33.03.2869.6101.20.910.900.900.250.390.4420.958
L1C + Mˉ\bar{M}33.23.0865.2105.60.920.890.900.240.380.4370.954
L1Cˉ\bar{C} + M33.03.2869.2101.60.910.900.900.250.390.4420.958
L1Cˉ\bar{C} + Mˉ\bar{M}33.23.0865.4105.40.920.890.900.240.380.4370.956
L2C + M32.83.4861.0109.80.910.890.900.230.370.4240.952
L2C + Mˉ\bar{M}33.23.0865.2105.60.920.890.900.240.380.4370.954
L2Cˉ\bar{C} + M32.83.4859.6111.20.910.890.900.230.360.4210.952
L2Cˉ\bar{C} + Mˉ\bar{M}33.23.0865.8105.00.920.890.900.240.380.4380.956
L3C + M33.23.0869.8101.00.920.900.910.250.390.4460.958
L3C + Mˉ\bar{M}33.23.0865.2105.60.920.890.900.240.380.4370.954
L3Cˉ\bar{C} + M33.03.2868.4102.40.910.890.900.240.380.4400.956
L3Cˉ\bar{C} + Mˉ\bar{M}33.23.0865.8105.00.920.890.900.240.380.4380.956

Table S3. Trained results on the pseudo-coloured subset, averaged over five stratified folds.

SetStrategyTPFNTNFPSens.Spec.BAPPVF1MCCAUC
No LabelCˉ\bar{C} + M29.86.4751.2163.40.820.820.820.150.260.3070.888
No LabelC + Mˉ\bar{M}29.86.4752.0162.60.820.820.820.160.260.3080.888
No LabelCˉ\bar{C} + M30.06.2752.2162.40.830.820.830.160.260.3110.888
No LabelCˉ\bar{C} + Mˉ\bar{M}30.06.2753.6161.00.830.820.830.160.260.3120.890
L1Cˉ\bar{C} + M30.06.2753.2161.40.830.820.830.160.260.3120.888
L1C + Mˉ\bar{M}29.86.4752.0162.60.820.820.820.160.260.3080.888
L1Cˉ\bar{C} + M29.86.4753.2161.40.820.820.820.160.260.3090.886
L1Cˉ\bar{C} + Mˉ\bar{M}30.06.2753.6161.00.830.820.830.160.260.3120.890
L2Cˉ\bar{C} + M29.66.6745.8168.80.820.820.820.150.250.2990.882
L2C + Mˉ\bar{M}29.86.4752.0162.60.820.820.820.160.260.3080.888
L2Cˉ\bar{C} + M29.66.4745.2169.40.820.810.820.150.250.3010.882
L2Cˉ\bar{C} + Mˉ\bar{M}30.06.2753.4161.20.830.820.830.160.260.3120.890
L3Cˉ\bar{C} + M30.26.0759.8154.80.830.830.830.160.270.3220.894
L3C + Mˉ\bar{M}29.86.4752.0162.60.820.820.820.160.260.3080.888
L3Cˉ\bar{C} + M30.45.8762.6152.00.840.830.840.170.280.3280.892
L3Cˉ\bar{C} + Mˉ\bar{M}30.06.2753.6161.00.830.820.830.160.260.3120.890

Table S4. Trained results on the colourless subset, averaged over five stratified folds.

SetStrategyTPFNTNFPSens.Spec.BAPPVF1MCCAUC
No LabelCˉ+M\bar{C} + M3708491211.000.880.940.230.380.450.97
No LabelC+MˉC + \bar{M}3708491211.000.880.940.230.380.450.97
No LabelC+MC + M3708481221.000.870.940.230.380.450.97
No LabelCˉ+Mˉ\bar{C} + \bar{M}3708511191.000.880.940.240.380.460.97
L1Cˉ+M\bar{C} + M3618541160.970.880.930.240.380.450.97
L1C+MˉC + \bar{M}3708491211.000.880.940.230.380.450.97
L1C+MC + M3618541160.970.880.930.240.380.450.97
L1Cˉ+Mˉ\bar{C} + \bar{M}3708511191.000.880.940.240.380.460.97
L2Cˉ+M\bar{C} + M3528411290.950.870.910.210.350.410.96
L2C+MˉC + \bar{M}3708491211.000.880.940.230.380.450.97
L2C+MC + M3528411290.950.870.910.210.350.410.96
L2Cˉ+Mˉ\bar{C} + \bar{M}3708521181.000.880.940.240.390.460.97
L3Cˉ+M\bar{C} + M3618491210.970.880.920.230.370.440.97
L3C+MˉC + \bar{M}3708491211.000.880.940.230.380.450.97
L3C+MC + M3618511190.970.880.930.230.380.440.97
L3Cˉ+Mˉ\bar{C} + \bar{M}3708521181.000.880.940.240.390.460.97

Table S5. Fold 1 — trained pseudo-coloured subset.

SetStrategyTPFNTNFPSens.Spec.BAPPVF1MCCAUC
No LabelCˉ+M\bar{C} + M3248651060.890.890.890.230.370.420.95
No LabelC+MˉC + \bar{M}3338621090.920.890.900.230.370.430.96
No LabelC+MC + M3248631080.890.890.890.230.360.420.95
No LabelCˉ+Mˉ\bar{C} + \bar{M}3248641070.890.890.890.230.370.420.95
L1Cˉ+M\bar{C} + M324872990.890.900.890.240.380.430.95
L1C+MˉC + \bar{M}3338621090.920.890.900.230.370.430.96
L1C+MC + M3248681030.890.890.890.240.370.430.95
L1Cˉ+Mˉ\bar{C} + \bar{M}3248641070.890.890.890.230.370.420.95
L2Cˉ+M\bar{C} + M3248691020.890.890.890.240.380.430.95
L2C+MˉC + \bar{M}3338621090.920.890.900.230.370.430.96
L2C+MC + M3248681030.890.890.890.240.370.430.95
L2Cˉ+Mˉ\bar{C} + \bar{M}3248641070.890.890.890.230.370.420.95
L3Cˉ+M\bar{C} + M324875960.890.900.900.250.390.440.95
L3C+MˉC + \bar{M}3338621090.920.890.900.230.370.430.96
L3C+MC + M333872990.920.900.910.250.390.450.95
L3Cˉ+Mˉ\bar{C} + \bar{M}3248641070.890.890.890.230.370.420.95

Table S6. Fold 2 — trained pseudo-coloured subset.

SetStrategyTPFNTNFPSens.Spec.BAPPVF1MCCAUC
No LabelCˉ+M\bar{C} + M3338671040.920.890.900.240.380.440.95
No LabelC+MˉC + \bar{M}3338691020.920.890.910.240.390.440.96
No LabelC+MC + M3338701010.920.900.910.250.390.440.95
No LabelCˉ+Mˉ\bar{C} + \bar{M}3338671040.920.890.900.240.380.440.95
L1Cˉ+M\bar{C} + M3338681030.920.890.910.240.380.440.96
L1C+MˉC + \bar{M}3338691020.920.890.910.240.390.440.96
L1C+MC + M342872990.940.900.920.260.400.460.96
L1Cˉ+Mˉ\bar{C} + \bar{M}3338671040.920.890.900.240.380.440.95
L2Cˉ+M\bar{C} + M3248711000.890.900.890.240.380.430.95
L2C+MˉC + \bar{M}3338691020.920.890.910.240.390.440.96
L2C+MC + M3248681030.890.890.890.240.370.430.95
L2Cˉ+Mˉ\bar{C} + \bar{M}3338681030.920.890.910.240.380.440.95
L3Cˉ+M\bar{C} + M342873980.940.900.920.260.400.460.96
L3C+MˉC + \bar{M}3338691020.920.890.910.240.390.440.96
L3C+MC + M342872990.940.900.920.260.400.460.96
L3Cˉ+Mˉ\bar{C} + \bar{M}3338681030.920.890.910.240.380.440.95

Table S7. Fold 3 — trained pseudo-coloured subset.

SetStrategyTPFNTNFPSens.Spec.BAPPVF1MCCAUC
No LabelCˉ+M\bar{C} + M333873980.920.900.910.250.400.450.96
No LabelC+MˉC + \bar{M}324872990.890.900.890.240.380.430.95
No LabelC+MC + M333872990.920.900.910.250.390.450.96
No LabelCˉ+Mˉ\bar{C} + \bar{M}3338671040.920.890.900.240.380.440.96
L1Cˉ+M\bar{C} + M333874970.920.900.910.250.400.450.96
L1C+MˉC + \bar{M}324872990.890.900.890.240.380.430.95
L1C+MC + M324872990.890.900.890.240.380.430.96
L1Cˉ+Mˉ\bar{C} + \bar{M}3338671040.920.890.900.240.380.440.96
L2Cˉ+M\bar{C} + M3338581130.920.880.900.230.360.420.95
L2C+MˉC + \bar{M}324872990.890.900.890.240.380.430.95
L2C+MC + M3338561150.920.880.900.220.360.420.95
L2Cˉ+Mˉ\bar{C} + \bar{M}3338671040.920.890.900.240.380.440.96
L3Cˉ+M\bar{C} + M333878930.920.900.910.260.410.460.96
L3C+MˉC + \bar{M}324872990.890.900.890.240.380.430.95
L3C+MC + M324877940.890.900.900.250.400.440.96
L3Cˉ+Mˉ\bar{C} + \bar{M}3338671040.920.890.900.240.380.440.96

Table S8. Fold 4 — trained pseudo-coloured subset.

SetStrategyTPFNTNFPSens.Spec.BAPPVF1MCCAUC
No LabelCˉ+M\bar{C} + M315879920.860.910.880.250.390.430.95
No LabelC+MˉC + \bar{M}315874970.860.900.880.240.380.420.93
No LabelC+MC + M315880910.860.910.880.250.390.440.95
No LabelCˉ+Mˉ\bar{C} + \bar{M}315878930.860.900.880.250.390.430.95
L1Cˉ+M\bar{C} + M315880910.860.910.880.250.390.440.95
L1C+MˉC + \bar{M}315874970.860.900.880.240.380.420.93
L1C+MC + M315880910.860.910.880.250.390.440.95
L1Cˉ+Mˉ\bar{C} + \bar{M}315878930.860.900.880.250.390.430.95
L2Cˉ+M\bar{C} + M3248661050.890.890.890.230.370.420.95
L2C+MˉC + \bar{M}315874970.860.900.880.240.380.420.93
L2C+MC + M3248651060.890.890.890.230.370.420.95
L2Cˉ+Mˉ\bar{C} + \bar{M}315878930.860.900.880.250.390.430.95
L3Cˉ+M\bar{C} + M315874970.860.900.880.240.380.420.95
L3C+MˉC + \bar{M}315874970.860.900.880.240.380.420.93
L3C+MC + M3068701010.830.900.860.230.360.400.94
L3Cˉ+Mˉ\bar{C} + \bar{M}315878930.860.900.880.250.390.430.95

Table S9. Fold 5 — trained pseudo-coloured subset.

SetStrategyTPFNTNFPSens.Spec.BAPPVF1MCCAUC
No LabelCˉ+M\bar{C} + M3077671470.810.840.820.170.280.320.92
No LabelC+MˉC + \bar{M}3167691450.840.840.840.180.290.340.92
No LabelC+MC + M3077681460.810.840.830.170.280.320.92
No LabelCˉ+Mˉ\bar{C} + \bar{M}3077641500.810.840.820.170.280.320.92
L1Cˉ+M\bar{C} + M2987681460.780.840.810.170.270.310.92
L1C+MˉC + \bar{M}3167691450.840.840.840.180.290.340.92
L1C+MC + M2897691450.760.840.800.160.270.300.91
L1Cˉ+Mˉ\bar{C} + \bar{M}3077641500.810.840.820.170.280.320.92
L2Cˉ+M\bar{C} + M3077621520.810.830.820.160.270.320.92
L2C+MˉC + \bar{M}3167691450.840.840.840.180.290.340.92
L2C+MC + M3077631510.810.830.820.170.280.320.92
L2Cˉ+Mˉ\bar{C} + \bar{M}3077641500.810.840.820.170.280.320.92
L3Cˉ+M\bar{C} + M2987731410.780.850.810.170.280.320.92
L3C+MˉC + \bar{M}3167691450.840.840.840.180.290.340.92
L3C+MC + M2987771370.780.850.820.170.290.320.91
L3Cˉ+Mˉ\bar{C} + \bar{M}3077641500.810.840.820.170.280.320.92

Table S10. Fold 1 — trained colourless subset.

SetStrategyTPFNTNFPSens.Spec.BAPPVF1MCCAUC
No LabelCˉ+M\bar{C} + M3067351800.830.800.820.140.240.290.87
No LabelC+MC + M2977411740.810.810.810.140.240.290.87
No LabelC+MC + M3067361790.830.800.820.140.240.290.87
No LabelC+MˉC + \bar{M}3067431720.830.810.820.150.250.300.88
L1Cˉ+M\bar{C} + M3067381770.830.810.820.140.250.300.87
L1C+MC + M2977411740.810.810.810.140.240.290.87
L1C+MC + M3067361790.830.800.820.140.240.290.87
L1C+MˉC + \bar{M}3067431720.830.810.820.150.250.300.88
L2Cˉ+M\bar{C} + M3067351800.830.800.820.140.240.290.88
L2C+MC + M2977411740.810.810.810.140.240.290.87
L2C+MC + M3067381770.830.810.820.140.250.300.88
L2C+MˉC + \bar{M}3067421730.830.810.820.150.250.300.88
L3Cˉ+M\bar{C} + M3067401750.830.810.820.150.250.300.88
L3C+MC + M2977411740.810.810.810.140.240.290.87
L3C+MC + M3157381770.860.810.830.150.250.310.88
L3C+MˉC + \bar{M}3067431720.830.810.820.150.250.300.88

Table S11. Table S11: Fold 2 — trained colourless subset.

SetStrategyTPFNTNFPSens.Spec.BAPPVF1MCCAUC
No LabelCˉ+M\bar{C} + M2977591560.810.830.820.160.260.310.87
No LabelC+MC + M2977611540.810.830.820.160.260.310.87
No LabelC+MC + M2977611540.810.830.820.160.260.310.87
No LabelC+MˉC + \bar{M}2977641510.810.830.820.160.270.310.87
L1Cˉ+M\bar{C} + M3067621530.830.830.830.160.270.320.87
L1C+MC + M2977611540.810.830.820.160.260.310.87
L1C+MC + M3067621530.830.830.830.160.270.320.87
L1C+MˉC + \bar{M}2977641510.810.830.820.160.270.310.87
L2Cˉ+M\bar{C} + M3067541610.830.820.830.160.260.310.88
L2C+MC + M2977611540.810.830.820.160.260.310.87
L2C+MC + M3067551600.830.830.830.160.270.310.88
L2C+MˉC + \bar{M}2977641510.810.830.820.160.270.310.87
L3Cˉ+M\bar{C} + M3067731420.830.840.840.170.290.340.87
L3C+MC + M2977611540.810.830.820.160.260.310.87
L3C+MC + M3067761390.830.850.840.180.290.340.87
L3C+MˉC + \bar{M}2977641510.810.830.820.160.270.310.87

Table S12. Table S12: Fold 3 — trained colourless subset.

SetStrategyTPFNTNFPSens.Spec.BAPPVF1MCCAUC
No LabelCˉ+M\bar{C} + M3067471680.830.820.820.150.260.310.91
No LabelC+MC + M3067471680.830.820.820.150.260.310.91
No LabelC+MC + M3157481670.860.820.840.160.260.320.91
No LabelC+MˉC + \bar{M}3157461690.860.820.840.150.260.320.91
L1Cˉ+M\bar{C} + M3157471680.860.820.840.160.260.320.91
L1C+MC + M3067471680.830.820.820.150.260.310.91
L1C+MC + M3157471680.860.820.840.160.260.320.91
L1C+MˉC + \bar{M}3157461690.860.820.840.150.260.320.91
L2Cˉ+M\bar{C} + M3157511640.860.820.840.160.270.320.88
L2C+MC + M3067471680.830.820.820.150.260.310.91
L2C+MC + M3157441710.860.810.840.150.260.310.88
L2C+MˉC + \bar{M}3157461690.860.820.840.150.260.320.91
L3Cˉ+M\bar{C} + M3157531620.860.820.840.160.270.320.92
L3C+MC + M3067471680.830.820.820.150.260.310.91
L3C+MC + M3157581570.860.830.840.160.280.330.92
L3C+MˉC + \bar{M}3157461690.860.820.840.150.260.320.91

Table S13. Table S13: Fold 4 — trained colourless subset.

References

  1. [1]B. An, S. Zhu, M.-A. Panaitescu-Liess, C. K. Mummadi, F. Huang, PerceptionCLIP: Visual classification by inferring and conditioning on contexts (2024). arXiv:2308.01313. URL https://arxiv.org/abs/2308.01313
  2. [2]M. Godec, S. Sternig, P. M. Roth, H. Bischof, Context-driven clustering by multi-class classification in an active learning framework, in: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2010, pp. 19–24. doi:10.1109/CVPRW.2010.5543886.DOI
  3. [3]X. Wang, Z. Zhu, Context understanding in computer vision: A survey, Computer Vision and Image Understanding 229 (2023) 103646. doi:10.1016/j.cviu.2023.103646.DOI
  4. [4]X. Ge, F. Chen, S. Xu, F. Tao, J. M. Jose, Cross-modal semantic enhanced interaction for image-sentence retrieval, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2023, pp. 1022–1031.arxiv.org/abs/2210.08908
  5. [5]L. Wolf, S. Bileschi, A critical view of context, International Journal of Computer Vision 69 (2006) 251–261. doi:10.1007/s11263-006-7538-0.DOI
  6. [6]J. Lu, J. Rao, K. Chen, X. Guo, Y. Zhang, B. Sun, C. Yang, J. Yang, Evaluation and enhancement of semantic grounding in large vision-language models (2024). arXiv:2309.04041. URL https://arxiv.org/abs/2309.04041
  7. [7]Y. Zhang, K. Zhou, Z. Liu, What makes good examples for visual in-context learning, in: Advances in Neural Information Processing Systems (NeurIPS), 2023.arxiv.org/abs/2301.13670
  8. [8]T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, C. L. Zitnick, Microsoft COCO: Common objects in context, in: European Conference on Computer Vision (ECCV), Springer, 2014, pp. 740–755.arxiv.org/abs/1405.0312
  9. [9]J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei, ImageNet: A large-scale hierarchical image database, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009, pp. 248–255. doi:10.1109/CVPR.2009.5206848.DOI
  10. [10]F. J. Dowd, Torsades de pointes, in: Reference Module in Biomedical Sciences, Elsevier, 2014. doi:10.1016/B978-0-12-801238-3.05734-5.
  11. [11]A. Chan, G. K. Isbister, C. M. J. Kirkpatrick, S. B. Duffull, Drug-induced QT prolongation and torsades de pointes: evaluation of a QT nomogram, QJM: An International Journal of Medicine 100 (10) (2007) 609–615. doi:10.1093/qjmed/hcm072.
  12. [12]A. Alahmadi, A. Davies, J. Royle, L. Goodwin, K. Cresswell, Z. Arain, M. Vigo, C. Jay, An explainable algorithm for detecting drug-induced QT-prolongation at risk of Torsades de Pointes (TdP) regardless of heart rate and T-wave morphology, Computers in Biology and Medicine 131 (2021) 104281. doi:10.1016/j.compbiomed.2021.104281.DOI
  13. [13]A. Alahmadi, A. Davies, M. Vigo, C. Jay, Personalized, intuitive and visual QT-prolongation monitoring using patient-specific QTc threshold with pseudo-coloring and explainable AI, Journal of Electrocardiology 81 (2023) 218–223. doi:10.1016/j.jelectrocard.2023.09.012.DOI
  14. [14]A. Alahmadi, A. Davies, M. Vigo, C. Jay, Pseudo-colouring an ECG enables lay people to detect QT-interval prolongation regardless of heart rate, PLOS ONE 15 (8) (2020) e0237854. doi:10.1371/journal.pone.0237854.DOI
  15. [15]J. Vicente, L. Johannesen, J. W. Mason, W. J. Crumb, E. Pueyo, N. Stockbridge, D. G. Strauss, Comprehensive T wave morphology assessment in a randomized clinical study of dofetilide, quinidine, ranolazine, and verapamil, Journal of the American Heart Association 4 (4) (2015) e001615. doi:10.1161/JAHA.114.001615.DOI
  16. [16]S. Zhang, Y. Xu, N. Usuyama, H. Xu, J. Bagga, R. Tinn, S. Preston, R. Rao, M. Wei, N. Valluri, C. Wong, A. Tupini, Y. Wang, M. Mazzola, S. Shukla, L. Liden, G. Gao, A. Crabtree, B. Piening, C. Bifulco, M. P. Lungren, T. Naumann, S. Wang, H. Poon, BiomedCLIP: A multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs (2025). arXiv:2303.00915. URL https://arxiv.org/abs/2303.00915
  17. [17]A. Torralba, K. P. Murphy, W. T. Freeman, M. A. Rubin, Context-based vision system for place and object recognition, in: Proceedings of the Ninth IEEE International Conference on Computer Vision, 2003, pp. 273–280. doi:10.1109/ICCV.2003.1238354.DOI
  18. [18]A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, I. Sutskever, Learning transferable visual models from natural language supervision (2021). arXiv:2103.00020. URL https://arxiv.org/abs/2103.00020
  19. [19]Z. Liu, P. Luo, X. Wang, X. Tang, Deep learning face attributes in the wild, in: Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2015, pp. 3730–3738. doi:10.1109/ICCV.2015.425.DOI
  20. [20]C. Wah, S. Branson, P. Welinder, P. Perona, S. Belongie, The Caltech-UCSD Birds-200-2011 dataset, Tech. Rep. CNS-TR-2011-001, California Institute of Technology (2011).
  21. [21]W. Berrios, G. Mittal, T. Thrush, D. Kiela, A. Singh, Towards language models that can see: Computer vision through the lens of natural language (2023). arXiv:2306.16410. URL https://arxiv.org/abs/2306.16410
  22. [22]H. Sakai, S. S. Lam, Large language models for health care text classification: Systematic review, JMIR AI 5 (2026) e79202. doi:10.2196/79202.DOI
  23. [23]Y. Zhang, Z. Ma, X. Gao, S. Shakiah, Q. Gao, J. Chai, GroundHOG: Grounding large language models to holistic segmentation (2024). arXiv:2402.16846. URL https://arxiv.org/abs/2402.16846
  24. [24]S. Yin, C. F. Fu, S. Zhao, K. Li, X. Sun, T. Xu, E. Chen, A survey on multimodal large language models, National Science Review 11 (2024). doi:10.1093/nsr/nwae403.DOI
  25. [25]Z. Zhao, Y. Liu, H. Wu, M. Wang, Y. Li, S. Wang, L. Teng, D. Liu, Z. Cui, Q. Wang, D. Shen, CLIP in medical imaging: A survey, Medical Image Analysis 102 (2025) 103551. doi:10.1016/j.media.2025.103551.DOI
  26. [26]Y. Hu, Y. Zheng, S. Miao, X. Zhang, J. Xia, Y. Qi, Y. Zhang, Y. He, Q. Chen, J. Ye, H. Qiao, X. Hu, L. Xu, J. Zhang, H. Liu, M. Zheng, Y. Wang, D. Zhang, J. Zhang, W. Shao, Y. Liu, L. Zhang, G. Yang, Cardiac-CLIP: A vision-language foundation model for 3D cardiac CT images (2025). arXiv:2507.22024. URL https://arxiv.org/abs/2507.22024
  27. [27]R. M. Lester, S. Pagliarunga, Surfing the T wave: A primer on ECG T wave morphologies encountered in clinical trials and impact on the QT interval and patient safety, Clinical and Translational Science 18 (2025) e70145. doi:10.1111/cts.70145.
  28. [28]L. Johannesen, J. Vicente, J. W. Mason, C. Sanabria, K. Waite-Labott, M. Hong, P. Guo, J. Lin, J. S. Sørensen, L. Galeotti, J. Florian, M. Ugander, N. Stockbridge, D. G. Strausss, Differentiating drug-induced multichannel block on the electrocardiogram: randomized study of dofetilide, quinidine, ranolazine, and verapamil, Clinical Pharmacology and Therapeutics 96 (5) (2014) 549–558. doi:10.1038/clpt.2014.155.DOI
  29. [29]T. Pollard, B. E. Moody, L.-W. H. Lehman, B. J. Gow, C. Fernandes, C. Xie, A. Johnson, R. G. Mark, T. Heldt, PhysioNet as a global platform for biomedical research, Nature Health 1 (8) (2026) 792–795. doi:10.1038/s44360-026-00096-z. URL https://doi.org/10.1038/s44360-026-00096-z
  30. [30]Pillow Contributors, Pillow (PIL fork) documentation, https://pillow.readthedocs.io/en/stable/reference/Image.html, accessed: 1 February 2026.
  31. [31]Hugging Face, Hugging Face – the AI community building the future, https://huggingface.co/, accessed: 10 February 2026 (2026).
  32. [32]G. Ilharco, M. Wortsman, R. Wightman, C. Gordon, N. Carlini, R. Taori, A. Dave, V. Shankar, H. Namkoong, J. Miller, H. Hajishirzi, A. Farhadi, L. Schmidt, OpenCLIP: An open source implementation of CLIP, https://github.com/mlfoundations/open_clip, accessed: 1 February 2026.
  33. [33]G. Cheng, J. Han, A survey on object detection in optical remote sensing images, ISPRS Journal of Photogrammetry and Remote Sensing 117, see Section 6.2.1 (2016). doi:10.1016/j.isprsjprs.2016.03.014.DOI
  34. [34]F. Pedregosa, V. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, É. Duchesnay, Scikit-learn: Machine learning in Python, Journal of Machine Learning Research 12 (2011) 2825–2830.arxiv.org/abs/1201.0490
  35. [35]IBM, What is a confusion matrix?, https://www.ibm.com/think/topics/confusion-matrix, accessed: 5 February 2026 (2021).
  36. [36]K. M. McKay, L. L. Lim, R. N. Van Gelder, Rational laboratory testing in uveitis: A Bayesian analysis, Survey of Ophthalmology 66 (2021) 802–825, see Section 2.1. doi:10.1016/j.survophthal.2021.02.002.DOI
  37. [37]scikit-learn developers, f1 score – scikit-learn documentation, https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html, accessed: 21 February 2026.
  38. [38]D. Chicco, G. Jurman, The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation, BMC Genomics 21 (2020) 6. doi:10.1186/s12864-019-6413-7.DOI
  39. [39]scikit-learn developers, matthews_corrcoef – scikit-learn documentation, https://scikit-learn.org/stable/modules/generated/sklearn.metrics.matthews_corrcoef.html, accessed: 11 March 2026 (2026).
  40. [40]Google for Developers, Classification: ROC and AUC, https://developers.google.com/machine-learning/crash-course/classification/roc-and-auc, accessed: 22 February 2026.

Paper details

vlnc:W_xcahmuq4
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 4

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents