betaFeedback

WorksComputer Science

CONTEXT-PCa: Evaluating the role of clinical context in MRI-based AI for detecting clinically significant prostate cancer (study protocol)

Abstract

The objective of this study is to investigate the role of clinical variables - their breadth, source, and quality - in the development, utility, and generalisability of an AI system for bi-parametric MRI (bpMRI)-based diagnosis of clinically significant prostate cancer (csPCa). Clinical context is added to the published PI-CAI-2B image-only model, whose weights are fixed. The study cohort comprises 6,022 biparametric MRI cases from 4,540 patients at University College London Hospitals NHS Foundation Trust (2019-2024), of which 4,834 cases (3,632 patients) are held out for internal testing. Forty-three clinical variables are drawn from structured health records, clinical letters, MRI reports, and automated image measurements. The primary endpoint is the difference in the area under the receiver operating characteristic curve (AUROC) between the image-context and image-only models on the internal test set. Secondary analyses compare combining strategies, the contribution of individual variables, and performance across subgroups. All models are further tested on five external cohorts from primary diagnosis and population-based screening.

STUDY PROTOCOL - OBSERVATIONAL RETROSPECTIVE STUDY

*Corresponding author: Yipeng Hu ([email protected])

Background and Rationale

Prostate cancer is the most commonly diagnosed male malignancy worldwide, with 1,414,249 new cases and 375,000 deaths annually (2020 GLOBOCAN estimates) [1]. Diagnosis increasingly relies on magnetic resonance imaging (MRI) followed by targeted biopsy, a pathway supported by clinical trials (PROMIS [2], PRECISION [3]) and prospective studies (MRI-FIRST (Rouvière et al., 2019) and 4M (van der Leest et al., 2019)). Bi-parametric MRI (bpMRI), which omits gadolinium-based contrast enhancement in favour of T2-weighted and diffusion-weighted sequences alone, has gained traction as a faster, cheaper, and contrast-free alternative with comparable diagnostic accuracy. In the UK alone, approximately 200,000 men undergo prostate MRI each year.

The Prostate Imaging-Reporting and Data System version 2.1 (PI-RADS v2.1) (Turkbey et al., 2019) provides a structured framework for reporting prostate MRI, yet interpretation remains subjective, radiologist-dependent, and constrained by a global shortage of trained readers. Clinically significant prostate cancer (csPCa), defined following the PI-CAI challenge labelling procedure (Saha et al., 2024) as ISUP grade group ≥2\ge2 (Gleason score ≥3+4\ge3+4) in any core, is one of the primary targets in clinical detection pathways because it is associated with with an increased risk of adverse oncological outcomes and generally requires closer surveillance or consideration of active treatment (Epstein et al., 2016; Cornford et al., 2024).

Clinical information available within the prostate cancer diagnostic pathway is highly heterogeneous. Established predictors such as PSA, PSA density, prostate volume, age, DRE findings, radiologist score, ethnicity, and family history are routinely used for risk assessment (Roobol et al., 2013; Ankerst et al., 2014; Turkbey et al., 2019; Cornford et al., 2024). Other factors, including longitudinal PSA change, BMI, lifestyle behaviours, laboratory abnormalities, comorbidities, and periprostatic adipose tissue (PPAT), are suspected to be associated with the diagnosis and prediction of prostate cancer and may provide additional clinical context, although their predictive contribution may be more population- and setting-dependent.

Despite their established clinical relevance, the added value of clinical variables within MRI-based AI systems remains uncertain. Existing studies have reported mixed results, with some showing no statistically significant benefit from adding clinical information and others demonstrating clear improvements; moreover, performance can depend substantially on how clinical and imaging information are integrated (Roest et al., 2023, 2024; Woźnicki et al., 2020; Hiremath et al., 2021). This leaves unresolved which clinical variables provide genuine complementary value beyond imaging, which may introduce redundant or dataset-specific associations that impair generalisability, and how heterogeneous clinical and imaging information should best be integrated within AI models.

Objectives

This study investigates the role of clinical variables - their breadth, source, and quality - in the development, utility, and generalisability of an AI system for bi-parametric MRI (bpMRI)-based diagnosis of clinically significant prostate cancer (csPCa, ISUP ≥2\ge2). The objectives below give effect to that question.

Primary Objective

To determine whether combining clinical variables with bpMRI improves the detection of csPCa relative to bpMRI alone, evaluated on the held-out UCLH test set. Clinical-only models are evaluated alongside as a reference for the performance attainable without imaging.

Secondary Objectives

  1. Quantify the incremental predictive value of individual clinical variables through add-one-in and leave-one-out ablation analyses.

  2. Determine whether the value of clinical context depends on the stage at which it is combined with imaging features - early, mid, or late combining.

  3. Determine whether a clinical variable’s contribution depends on its source and quality: structured extraction, automated image analysis, or extraction from free text; its missingness rate; and, for free-text variables, its validated extraction reliability.

  4. Characterise performance heterogeneity across subgroups defined by age, ethnicity, PSA level, and radiologist score.

  5. Identify which clinical variables drive individual predictions, and whether their contributions are consistent across combining strategies.

  6. Evaluate the image-only and image-context models on external cohorts from primary diagnostic and population screening settings.

Study Design

This is a retrospective cohort study using routinely collected electronic health record (EHR) data and clinically acquired MRI studies from University College London Hospitals (UCLH) NHS Foundation Trust. The study follows a diagnostic prediction model development and internal validation design, reported in accordance with the STROBE statement (von Elm et al., 2007) for observational studies and the TRIPOD+AI guideline [6] for prediction model studies involving AI/machine learning.

Setting and Study Period

Data are drawn from the UCLH Epic EHR system, covering the period from April 2019 (system go-live) to March 2024. UCLH is a tertiary academic centre in central London with a high-volume prostate MRI service.

Clinical prostate MRI is performed across several UCLH sites: the UCLH Main Building, where three scanners are potentially used for prostate MRI; Westmoreland Street Hospital; the Macmillan Cancer Centre; the Grafton Way Building; and the Royal National ENT and Eastman Dental Hospitals. Approximately 12 consultant radiologists reported prostate MRI over the study period. The cohort therefore spans multiple sites, scanners, and reporting radiologists.

Prostate MRI at UCLH is reported using the Likert scale, which records the reporting radiologist’s overall level of suspicion for clinically significant cancer on a subjective five-point scale. Unlike a PI-RADS category, which is assigned from the images alone, a Likert score permits the radiologist to weigh clinical context such as PSA, prostate volume, and prior biopsy history alongside the imaging findings.

Both scales run from 1 to 5, and reporting practice may vary between sites and over the study period. Throughout this protocol, radiologist score denotes the five-point level of suspicion recorded in the report, whether assigned as a PI-RADS category or as a Likert score, and the scale actually used will be recorded alongside the value for every case. Where the distinction bears on interpretation it is stated explicitly.

Data Splitting and Evaluation Paradigm

The image-only model is the published PI-CAI-2B system (Saha et al., 2026), with fixed weights; it was developed on cohorts external to this study (Section 7.2). All development of the clinical components uses a UCLH adaptation data set. The UCLH study cohort is divided into an adaptation set (10%), for adapting the model to the local setting, a validation set (10%), and an internal test set (80%) held out for all primary analyses (Figure 1). External testing uses five further cohorts (Section 7.6).

Study Population

Unit of Analysis

The unit of analysis is the MRI case: one prostate MRI study together with the clinical data available at the time it was acquired and the csPCa status ascertained for it. A patient may contribute more than one case, since men on active surveillance undergo serial imaging. Eligibility criteria are stated at the level at which they act: criteria concerning the person (age, opt-out status) remove the patient and all of their cases, whereas criteria concerning an individual study (prior treatment, non-diagnostic image quality) remove only that case.

Inclusion Criteria

  1. Male patients aged ≥18\ge18 years.

  1. Underwent prostate MRI at UCLH during the study period for initial diagnostic assessment or for active surveillance. Studies performed for staging of already-diagnosed prostate cancer, and studies performed for indications other than prostate cancer detection, are not eligible.

  1. csPCa status ascertainable by the hierarchical labelling procedure of § 5, which is applied in strict priority order: (a) histopathology; then (b) absence of any recorded lesion; then (c) a maximum radiologist score (PI-RADS or Likert) <3<3; then (d) an explicitly negative report. A case is labelled by the first applicable step, and a case for which no step applies is excluded.

Only route (a) can establish csPCa-positive status; routes (b)–(d) can establish only a presumed negative. All csPCa-positive cases therefore derive from histopathology, and the negative class is a mixture of biopsy-confirmed and presumed-benign cases. The consequences are addressed in § 9.

Exclusion Criteria

  1. MRI studies performed after the patient received local prostate cancer treatment (radical prostatectomy, radiotherapy, brachytherapy, HIFU or focal therapy). Only studies postdating the treatment are excluded; earlier studies from the same patient remain eligible. Prior biopsy is not treatment, so studies performed in men on active surveillance who have undergone biopsy alone remain eligible. MRI after androgen deprivation or other systemic therapy is retained and flagged; because such therapy can reduce gland volume, PSA and tumour conspicuity, a sensitivity analysis excluding these cases is reported.

  1. Patients who have opted out via the National Data Opt-Out scheme (NHS England, 2026).

  1. MRI studies with severe artefact rendering images non-diagnostic, as judged by the reporting radiologist.

Flowchart of patients and MRI cases from adult men seen at UCLH for suspected or known prostate cancer through exclusions, study cohort classification, and patient-level split into adaptation, validation, and internal test sets.

Figure 1. Flow of patients and MRI cases into the UCLH study cohort, and its division into adaptation, validation, and internal test sets. Exclusion boxes give the patients who had any excluded case, so patient counts in successive boxes need not differ by that number. Graphic text: Adult men seen at UCLH for suspected or known prostate cancer (April 2019 – March 2024) Npatients=8,157N_{\mathrm{patients}} = 8{,}157; National Data Opt-Out: Npatients=23N_{\mathrm{patients}} = 23; No MRI procedure recorded: Npatients=12N_{\mathrm{patients}} = 12; MRI not linked to an imaging study: NMRI cases=332N_{\mathrm{MRI\ cases}} = 332 (Npatients=58N_{\mathrm{patients}} = 58); Participants in the PRIME or ReIMAGINE study: NMRI cases=290N_{\mathrm{MRI\ cases}} = 290 (Npatients=79N_{\mathrm{patients}} = 79); Npatients=7,985N_{\mathrm{patients}} = 7{,}985; NMRI cases=12,902N_{\mathrm{MRI\ cases}} = 12{,}902; Directly indicated as post-treatment MRI or no matched MRI report in the source records; NMRI cases=5,558N_{\mathrm{MRI\ cases}} = 5{,}558 (Npatients=2,636N_{\mathrm{patients}} = 2{,}636); Npatients=5,349N_{\mathrm{patients}} = 5{,}349; NMRI cases=7,344N_{\mathrm{MRI\ cases}} = 7{,}344; csPCa status not ascertainable; NMRI cases=196N_{\mathrm{MRI\ cases}} = 196 (Npatients=164N_{\mathrm{patients}} = 164); Npatients=5,185N_{\mathrm{patients}} = 5{,}185; NMRI cases=7,148N_{\mathrm{MRI\ cases}} = 7{,}148; Excluded after review of clinical notes; NMRI cases=492N_{\mathrm{MRI\ cases}} = 492 (Npatients=287N_{\mathrm{patients}} = 287); MRI on or after prostate cancer treatment (prostatectomy, radiotherapy, brachytherapy, HIFU or focal therapy): 459; DL prostate segmentation failure (<5<5 mL): 31; Not a diagnostic prostate MRI: 2; Npatients=4,966N_{\mathrm{patients}} = 4{,}966; NMRI cases=6,656N_{\mathrm{MRI\ cases}} = 6{,}656; No histopathology and maximum Likert score 3: csPCa status not ascertainable; NMRI cases=634N_{\mathrm{MRI\ cases}} = 634 (Npatients=573N_{\mathrm{patients}} = 573); Study cohort: MRI before any local prostate cancer treatment; Npatients=4,540N_{\mathrm{patients}} = 4{,}540; NMRI cases=6,022N_{\mathrm{MRI\ cases}} = 6{,}022; Retained and flagged: prior BPH surgery 188; androgen deprivation or other systemic therapy before MRI 32; csPCa positive; NMRI cases=2,902N_{\mathrm{MRI\ cases}} = 2{,}902 (48.2%); Histopathology GG ≥2\geq2: 2,431; Biopsy GG ≥2\geq2 in follow-up notes: 224; No histopathology, maximum Likert 4–5 (presumed positive): 247; csPCa negative; NMRI cases=3,120N_{\mathrm{MRI\ cases}} = 3{,}120 (51.8%); Histopathology: 1,476 (benign or negative biopsy 684; GG1 791; GG ≥2\geq2 on record, labelled negative 1); Negative or GG1 biopsy in follow-up notes: 173; No histopathology: 1,471 (no lesion reported 513; maximum Likert 1–2 958); Patient-level split, stratified by csPCa status; Adaptation (10%): Npatients=454N_{\mathrm{patients}} = 454, NMRI cases=595N_{\mathrm{MRI\ cases}} = 595; Validation (10%): Npatients=454N_{\mathrm{patients}} = 454, NMRI cases=593N_{\mathrm{MRI\ cases}} = 593; Internal test (80%): Npatients=3,632N_{\mathrm{patients}} = 3{,}632, NMRI cases=4,834N_{\mathrm{MRI\ cases}} = 4{,}834.

Data Source

Structured clinical data are extracted from the UCLH Epic Data Warehouse via the OMOP Extraction System (OMOP ES), a governance-by-default pipeline developed in collaboration between UCL ARC, DHCT, and the UCLH CRDM team. The OMOP ES applies the following safeguards automatically:

  • Exclusion of patients registered on the National Data Opt-Out.

  • Exclusion of sexual health conditions.

  • Replacement of postcode with Lower Layer Super Output Area (LSOA).

Prostate MRI studies are retrieved from UCLH PACS (Picture Archiving and Communication System) and the Vendor Neutral Archive (VNA). DICOM images are anonymised (removal/pseudonymisation of identifying DICOM tags) and converted to NIfTI format for analysis.

Figure 2 shows the form each source takes and the route by which it reaches the analysis dataset. Date of birth is not retained: the extract carries year of birth only, and age is derived from it and the MRI date. Clinical event dates are retained at full precision because the linkage of pathology to imaging depends on the interval between them (§ 5).

Structured extract (OMOP), histopathology report, radiology report, clinic letter, and source-route-variables table showing data sources and extraction routes

Figure 2. Data sources and the route by which each reaches the analysis dataset. The excerpts are illustrative synthetic examples, not real patient records. Shading marks the report section to which the keyword rules of Table S1 apply, and the placeholders substituted during pseudonymisation.

Variables and Outcomes

Primary Outcome

Presence of clinically significant prostate cancer (csPCa), defined following the PI-CAI challenge labelling procedure (Saha et al. (2024)) as a case-level binary label (0 = no csPCa; 1 = csPCa) derived using the following hierarchical procedure:

  1. Histopathology (highest priority): If a Gleason score from prostate biopsy or radical prostatectomy is available, the case is labelled csPCa-positive when Gleason score ≥7\ge7 (i.e., ISUP grade group ≥2\ge2; Gleason ≥3+4\ge3+4) and csPCa-negative when Gleason score <7<7.

  2. Lesion count: If no histopathology is available and the radiology report records no lesion, the case is labelled csPCa-negative.

  3. Radiologist score: If neither histopathology nor lesion count is available and the maximum radiologist score (PI-RADS or Likert) is <3<3, the case is labelled csPCa-negative. This follows the PI-CAI assumption that cases scored 1–2 without biopsy are presumed benign, applied here to whichever scale the report used.

  4. Radiology report keywords: If the above conditions are unmet, cases with explicitly negative radiology reports (e.g., “no suspicious lesion”, “screen negative”) are labelled csPCa-negative. The complete rule set, including the exclusion patterns that prevent a label being assigned where the report expresses diagnostic uncertainty, is given in Table S1.

IDPattern (case-insensitive)Rationale
Exclusion patterns (evaluated first; a match prevents any label being assigned)
E1suspicio(us|n), concerning (unnegated)Any surviving expression of suspicion is incompatible with an explicitly negative report
E2equivocal, indeterminate, borderlineDiagnostic uncertainty; the case should be resolved by biopsy rather than by rule
E3cannot (be )?exclude(d)?, not excludedExplicit non-exclusion of malignancy
E4recommend(ed)? biopsy, for biopsy, proceed to biopsyOnward action implying residual suspicion
E5extracapsular extension, seminal vesicle invasion, ECE, SVIStaging features of established disease
E6metasta(sis|ses|tic), lymphadenopathy, bone lesionDisease beyond the prostate
E7progression, interval increase, enlargingChange implying active disease, relevant to active surveillance studies
E8(probabl(e|y)|likely|favour(s|ed)?) within the same clause as a malignancy termHedged positive assertion
Inclusion patterns (a match in the absence of any exclusion match labels the case csPCa-negative)
N1no (suspicious|significant|concerning) (lesion|abnormality|focus|area)Explicit absence of a suspicious finding
N2no evidence of (clinically significant )?(prostate )?(cancer|malignancy|tumour)Explicit absence of malignancy
N3screen(ing)? negativeLocal reporting convention for a negative study
N4normal (prostate )?(MRI|study|appearances)Global statement of normality
N5unremarkable (prostate|appearances|study)Global statement of normality
N6no (focal )?(lesion|abnormality) (identified|seen|detected|demonstrated|visualised)Explicit absence of a focal finding
N7benign (appearances|features|changes) not accompanied by any E-patternPositive assertion of benignity
N8no target(able)? lesionExplicit absence of a biopsy target

Table S1. Radiology report keyword rules applied at step 4 of outcome labelling. Patterns are shown in a simplified regular-expression notation; alternation is denoted by a vertical bar and optional elements by a question mark.

Linkage of pathology to imaging. A histopathology result is linked to an MRI case only where the specimen was obtained within 3 months after the MRI and before any intervening prostate cancer treatment. Where a patient has several MRI studies and several specimens, each specimen is linked to the most recent preceding eligible MRI, and each MRI case takes the earliest specimen linked to it. An MRI case with no specimen inside the window is labelled by the remaining steps of the hierarchy rather than by a later specimen, so that disease detected long after the index study is not attributed to it. The sensitivity of the results to this interval is examined in § 8.9.

Alignment with PI-CAI. This procedure aligns with the labelling rules of the PI-CAI challenge (Saha et al., 2024): the priority given to histopathology, the ISUP grade group ≥2\ge2 threshold, and the presumption that unbiopsied cases scored 1–2 are benign are all adopted unchanged.1

Imaging Predictors

Prostate MRI studies at UCLH are acquired on 1.5T and 3T scanners as part of routine clinical care. The bi-parametric MRI protocol comprises three input channels.

SequencePurposeKey Parameters
T2-weighted imaging (T2W)Anatomical and zonal assessment and lesion morphologyHigh-resolution T2W imaging
High bb-value diffusion-weighted imaging (HBV)Detection of restricted diffusion and lesion conspicuityHigh bb-value ≥1400\ge1400 s/mm2^2
Apparent diffusion coefficient (ADC) mapQuantitative assessment of diffusion restrictionCalculated from diffusion-weighted imaging (DWI) acquired at multiple bb-values

Table 1. Bi-parametric MRI sequences used as imaging predictors.

Image preprocessing.

  1. Format conversion: DICOM series are converted to NIfTI using dcm2niix. Manufacturer rescaling is applied at conversion so that ADC maps are expressed in ×10−6\times10^{-6} mm2^2/s irrespective of vendor; series whose scaling cannot be resolved are excluded. The high bb-value series is identified from the bb-value recorded in the standard or vendor-private tags, and whether it was acquired or computed is recorded for each case.

  1. Anonymisation: Identifying tags are removed or pseudonymised following the Basic Application Level Confidentiality Profile of DICOM PS3.15 Annex E. Series liable to carry burned-in annotation, such as secondary captures, are excluded.

  1. Spatial standardisation: T2W, high bb-value DWI, and ADC are resampled into a common reference frame at a fixed voxel spacing of 0.5×0.5×3.00.5 \times0.5 \times3.0 mm, to a uniform matrix of 256×256×20256 \times256 \times20 about the image centre. This reference frame is used for the image-derived measurements (§ 5.6); the input to the image model is prepared separately in step 4.

  1. Model-specific preprocessing: The input to PI-CAI-2B is prepared with the preprocessing released with the model (Saha et al., 2026): the prostate is segmented on T2W; ADC and high bb-value DWI are registered to T2W; the volumes are cropped to the prostate bounding box with a 10 mm margin, and resampling and intensity normalisation follow the nnU-Net plans of the model (Isensee et al., 2021). The preprocessing is identical to that used to train PI-CAI-2B, necessary for using its fixed weights (§ 8.1).

  1. Quality control: Converted volumes are checked for correct orientation, plausible geometry, and non-empty content. In addition, the median ADC within the segmented prostate is computed per scanner and compared against a physiologically plausible range of approximately 1,0001{,}000–1,800×10−6 mm2/s1{,}800 \times10^{-6}\ \mathrm{mm}^2/\mathrm{s}; a systematic departure indicates unresolved vendor scaling rather than a biological difference. Failures are listed and excluded.

Clinical Predictor Variables

VariableSourceTypeTransformNormalisation
I. Patient context
AgeStructured EHRContinuousNoneZ-score
EthnicityStructured EHRCategorical (6)One-hot encodedNone
Height (cm)Structured EHRContinuousNoneZ-score
Weight (kg)Structured EHRContinuousNoneZ-score
BMI (kg/m2\mathrm{m}^2)Structured EHRContinuouslog⁡(x+1)\log(x + 1)Z-score
BMI changeStructured EHRContinuousNoneZ-score
Smoking statusLLMCategorical (3)One-hot encodedNone
Alcohol statusLLMCategorical (7)One-hot encodedNone
Family hx any cancerLLMBinaryBinary encoded (0/1)None
Family hx prostate ca.LLMBinaryBinary encoded (0/1)None
Family hx breast ca.LLMBinaryBinary encoded (0/1)None
Family hx ovarian ca.LLMBinaryBinary encoded (0/1)None
Hx other cancersLLMBinaryBinary encoded (0/1)None
DiabetesLLMBinaryBinary encoded (0/1)None
HypertensionLLMBinaryBinary encoded (0/1)None
Cardiovascular diseaseLLMBinaryBinary encoded (0/1)None

Table 2. Clinical predictor variables, data sources, types, and preprocessing.

VariableSourceTypeTransformNormalisation
COPDLLMBinaryBinary encoded (0/1)None
AsthmaLLMBinaryBinary encoded (0/1)None
Chronic kidney diseaseLLMBinaryBinary encoded (0/1)None
DRE findingStructured EHRBinaryBinary encoded (0/1)None
PSA, laboratory (ng/mL)Structured EHRContinuouslog⁡(x+1)\log(x + 1)Z-score
PSA change, laboratoryStructured EHRContinuousNoneZ-score
PSA, MRI report (ng/mL)LLMContinuouslog⁡(x+1)\log(x + 1)Z-score
PSA change, MRI reportDerivedContinuousNoneZ-score
BloodStructured EHRBinaryBinary encoded (0/1)None
UrineStructured EHRBinaryBinary encoded (0/1)None
5-ARI use at MRILLMBinaryBinary encoded (0/1)None
Aspirin / NSAID use at MRILLMBinaryBinary encoded (0/1)None
Prior BPH interventionLLMBinaryBinary encoded (0/1)None
II. Radiologist-derived context
Radiologist score (1–5)LLMOrdinalNoneNone (raw)
MRI lesion countLLMCountlog⁡(x+1)\log(x + 1)Z-score
Lesion count with radiologist score ≥3\ge3LLMCountlog⁡(x+1)\log(x + 1)Z-score
Overall lesion lateralityLLMCategoricalOne-hot encodedNone
Index lesion sideLLMCategoricalOne-hot encodedNone
Index lesion zoneLLMMulti-label categoricalMulti-hot encodedNone

Table 9.

VariableSourceTypeTransformNormalisation
Index lesion levelLLMMulti-label categoricalMulti-hot encodedNone
III. Quantitative imaging context
Prostate volume (mL)DL segmentationContinuouslog⁡(x+1)\log(x+1)Z-score
PSA density (ng/mL/mL)DerivedContinuouslog⁡(x+1)\log(x+1)Z-score
PPAT volume (mL)DL segmentationContinuouslog⁡(x+1)\log(x+1)Z-score
PPAT / prostate volume ratioDerivedContinuouslog⁡(x+1)\log(x+1)Z-score
PPAT thickness (mm)DerivedContinuousNoneZ-score
PPAT fat fractionDerivedContinuous (0–1)NoneZ-score
Retropubic gap (mm)DerivedContinuousNoneZ-score

Table 2 (PDF p. 13).

Source key: Structured EHR = extracted directly from structured fields in the Epic EHR (OMOP), including structured Parquet exports; DL segmentation = prostate volume computed from automated deep learning segmentation of the MRI; Derived = calculated from other variables (PSA density = PSA / prostate volume); LLM = extracted from unstructured text (radiology reports, histopathology reports, and clinical notes) using the large language model pipeline described in § 5.5.

Missing Data Handling

In the image-context models, missing values are not imputed. Each clinical variable jj carries an indicator mjm_j, equal to 1 when the variable is recorded and 0 when it is missing, and its contribution to the model is multiplied by mjm_j (§ 8.1.1). A missing variable therefore contributes nothing, and a case with no recorded variable receives exactly the image-only prediction. Image-derived measurements that fail quality control, or that cannot be measured for a case (§ 5.6), are treated as missing in the same way.

LLM-Based Variable Extraction

Several clinical variables listed in Table 2 are not available as structured data fields and must be extracted from unstructured free-text clinical documents. A large language model (LLM)-based natural language processing (NLP) pipeline will be developed for this purpose. The pipeline is designed to maximise extraction reliability and auditability through the following stages.

  1. Source document identification and preparation. Three categories of unstructured text will be used: (i) radiology reports, from which radiologist scores, lesion counts, lesion location (overall laterality, and the side, zone, and level of the index lesion), and the PSA value and prostate volume stated in the report are extracted; (ii) histopathology reports, from which Gleason scores and ISUP grade groups are extracted; and (iii) clinical notes (consultation letters, clinic letters, referral letters, and pre-operative assessments), from which comorbidities, lifestyle factors, family history (including breast and ovarian cancer), history of other cancers, medication use at the time of MRI (5-ARI, aspirin or NSAID), and prior BPH intervention are extracted.

Variables recorded in structured fields, including anthropometric measurements and digital rectal examination findings, are taken from the structured extract (Table 2); prostate volume and the PPAT measurements are computed from the MRI (§ 5.6). All documents are exported from the OMOP-mapped EHR, split into per-patient corpora, and pseudonymised (removal of patient names, NHS numbers, dates of birth, and free-text addresses) prior to LLM processing.

2. Prompt design and extraction schema. Each patient’s documents are submitted to the LLM with a structured JSON schema prompt that (a) enumerates every target variable with its permissible data type and value set, (b) mandates explicit null returns for any variable not mentioned in the source text, preventing the model from inferring or fabricating values, and (c) includes task-specific instructions (e.g., “distinguish between cancer-related and non-cancer family history”, “do not infer a diagnosis from medication alone”). Prompts are developed iteratively on a held-out development set of 50 patients and frozen before production extraction begins. The system prompt and the per-variable extraction schema are reproduced in full in Section A.2 and Table S2.

3. Model selection and execution. The extraction will use a commercial LLM accessed via API (model selection to be finalised based on benchmark performance on the development set; candidates including Claude Sonnet 4.5, Gemini 2.5 Pro, and GPT-5.2). The selected model will process each patient’s notes in a zero-shot setting with the frozen prompt. To ensure reproducibility, the model version, API parameters (temperature = 0, maximum output tokens), and prompt text will be recorded and version-controlled. An automated execution pipeline will handle rate-limiting, retry logic (exponential back-off with a maximum of three retries per patient), and graceful degradation for notes exceeding the model’s context window (truncation with a trailing marker).

4. Output parsing and post-processing. Raw LLM outputs will be parsed as JSON with the following safeguards: (i) responses wrapped in markdown code fences are automatically unwrapped; (ii) malformed JSON triggers one retry with an explicit “return valid JSON only” instruction; (iii) parsed outputs are validated against the extraction schema (type checking, range checking for numerical fields, controlled vocabulary matching for categorical fields). After parsing, rule-based post-processing standardises free-text entries – for example, cancer type descriptions are mapped to a controlled vocabulary using regular-expression rules, and benign conditions erroneously included in cancer fields are filtered out. Internal consistency checks are applied (e.g., if family history of prostate cancer is flagged as true, “prostate cancer” must appear in the cancer types list). Smoking status and alcohol status are extracted as integer codes (Table S2) and one-hot encoded for modelling (Table 2). One-hot encoding discards the order of the seven alcohol bands but allows a non-monotonic association with cSPCa, since the dose–response relation between alcohol and prostate cancer may be non-linear ((Hong et al., 2020)). Figure 3 works through the sequence for one document, including a value that is discarded because its quotation cannot be located.

How a value is extracted from free text and what prevents one being invented

Figure 3. How a value is extracted from free text and what prevents one being invented. The document is an illustrative synthetic example, not a real patient record. Each non-null value must be returned with a supporting quotation, which is then located in the source document after normalisation of case and whitespace; a value whose quotation cannot be found is discarded rather than accepted, as shown for copd. The three states true, false and null are distinct: absence of mention is null, never false.

5. Duplicate extraction and stability assessment. A random 10% sample of patients (minimum 300) will undergo duplicate extraction: the frozen prompt will be run a second time with the primary model, and once with a second model from the candidate set. Per-variable agreement between runs will be quantified using Cohen’s κ\kappa for categorical variables and the intraclass correlation coefficient for continuous variables. Two forms of instability are distinguished. Run-to-run disagreement under the same model indicates residual non-determinism despite temperature 0; variables falling below κ=0.90\kappa= 0.90 will be examined for prompt ambiguity and the prompt revised before it is frozen. Between-model disagreement indicates model-specific interpretation of the prompt; it is reported descriptively and used to bound the model-dependence

of downstream results. This assessment is performed on the development set before the prompt is frozen, so it does not consume the validation sample described below.

6. Validation against manual chart review. A random sample of at least 200 patients (stratified by csPCa status) will undergo independent manual chart review by two clinically trained annotators. Inter-annotator agreement will be quantified using Cohen’s κ\kappa. LLM extractions will then be compared against the consensus annotation to compute per-variable precision, recall, and F1F_1 score. Variables with F1<0.80F_1 < 0.80 will be flagged; depending on the nature of the errors, these variables will be either excluded from analysis, subjected to additional manual correction, or retained with a sensitivity analysis assessing their impact on model performance.

7. Adjudicated re-review of discordant variables. Cases in which the LLM output and the two annotators do not all agree, together with all variables failing the F1F_1 threshold, will undergo a second review in a smaller adjudication set of 50 patients drawn from the validation sample. Review is performed by a third clinically trained reviewer, blinded to both the LLM output and the initial annotations, who assigns an independent value from the source documents; this adjudicated value becomes the reference standard for those cases. Each discrepancy is classified as source ambiguity, annotator error, or extraction error, and counts by category are reported in Table S3. The classification matters because only extraction error is attributable to the pipeline and correctable by prompt revision, whereas source ambiguity sets a ceiling on the accuracy any extraction method could reach for that variable.

VariablenSame-model agreementBetween-model agreementF1F_1 vs consensusDiscrepancies by cause
One row per variable in Table S2; completed on study execution.

Table S3. Reporting template for duplicate extraction and adjudicated re-review. One row per variable.

8. Auditability and traceability. Every extracted value will be linked to its source document identifier, enabling downstream auditing. The full extraction pipeline - prompt templates, model version, post-processing rules, and validation results - will be documented and released as supplementary material to support reproducibility.

9. Handling of extraction failures. Patients for whom extraction fails after all retries (e.g., persistent API errors, unparseable responses) will be coded as missing across all LLM-derived variables. These missing values are handled as described in § 5.4.

Image-Derived Variable Measurement

Prostate volume and the periprostatic adipose tissue (PPAT) measurements in Table 2 are computed automatically from the T2W series. Like the variables extracted from free text (§ 5.5), they enter the image-context model as scalar clinical predictors. They differ from the imaging predictors of § 5.2, which are image channels passed to the image encoder. All measurements are computed on the resampled T2W volume (§ 5.2, step 3) before cropping, which covers a 128×128×60128 \times128 \times60 mm field of view centred on the prostate.

Anatomical segmentation. The prostate and surrounding structures (bladder, rectum, bowel, pelvic bones, iliac vessels, and pelvic muscles) are segmented with the pretrained TotalSegmentator MRI models ((Wasserthal et al., 2023; D’Antonoli et al., 2025)), without retraining or fine-tuning on UCLH data.

Adipose tissue classification. No pretrained model for segmenting PPAT on T2W images was available, so adipose tissue, the high-signal class on non-fat-suppressed T2W images, is classified by signal intensity: after N4 bias-field correction ((Tustison et al., 2010)), a threshold is set for each case by Otsu’s method ((Otsu, 1979)), and segmented non-adipose structures are removed. Because the threshold is set on an arbitrary intensity scale, it does not transfer to other acquisition protocols. Periprostatic adipose tissue. PPAT is included as an additional, exploratory predictor. It has been hypothesised that immune-active cells within PPAT regulate disease development and progression (Gucalp et al., 2017; Finley et al., 2009), and periprostatic fat thickness on MRI has been associated with high-grade prostate cancer (Cao et al., 2017). PPAT is defined as adipose tissue within 15 mm of the prostate surface over the axial extent of the gland; bands of 10 mm and 20 mm are used in sensitivity analysis (§ 8.9).

Measurements. Six measurements are derived from these automated segmentations (Table 2, group III); PSA density is then calculated from prostate volume.

  • Prostate volume: the volume of the prostate mask, except in truncated cases (see below).

  • PPAT volume: the volume of PPAT at 15 mm.

  • PPAT / prostate volume ratio: PPAT volume divided by prostate volume.

  • PPAT fat fraction: the share of the 15 mm periprostatic band, after removal of other segmented structures, that is classified as adipose. It is a proportion of voxels classified by signal intensity, not a physical fat fraction measured with a fat-specific sequence.

  • Retropubic gap: on the mid-gland axial slice, within a strip 10 mm either side of the gland midline, the anteroposterior distance from the anterior surface of the prostate to the posterior surface of the pubic bone. Where no pubic bone is segmented anterior to the gland on that slice, up to two adjacent slices on each side are used instead.

  • PPAT thickness: the adipose depth within the retropubic gap, equal to the gap multiplied by the proportion of the strip classified as adipose.

Quality control. The image-derived measurements of a case are set to missing if the prostate mask is empty, is fragmented into disconnected components, or has a volume outside 10–250 mL; if PPAT volume is below 0.5 mL; or if the PPAT fat fraction lies outside 0.02–0.95. The retropubic gap and PPAT thickness are also set to missing where no pubic bone is segmented anterior to the gland. Overlay images of the masks at apex, mid-gland, and base are generated for every case for visual review. The case itself remains in the cohort, and the number of cases failing each check is reported.

Gland truncation. The 60 mm craniocaudal extent of the field of view does not cover the whole gland in larger prostates, so the segmented volume underestimates the true volume. Cases in which the prostate mask reaches the superior or inferior boundary of the field of view are flagged, and their number is reported. For these cases, prostate volume, and therefore PSA density, is taken from the volume stated in the radiology report; where the report states no volume, the segmented volume is retained. PPAT volume and the measurements derived from it remain underestimated in flagged cases, and their effect is examined in a sensitivity analysis (§ 8.9).

Validation. Prostate volume from segmentation is compared with the volume stated in the radiology report in all cases where a volume is stated, using the intraclass correlation coefficient and Bland–Altman analysis. No reference PPAT measurement exists in routine reports, and manual annotation is labour-intensive, so PPAT segmentation is compared with manual annotation by a trained reader in a random sample of 50 cases, stratified by scanner and gland size, reporting the Dice coefficient and relative volume error.

Data Linkage and Storage

All data sources are linked at the patient level using pseudonymised identifiers generated during the OMOP extraction process. An auditable list of all data items will be maintained, with statistical disclosure control applied to prevent re-identification through unique attribute constellations. Data are stored in the UCL Data Safe Haven or UCLH Azure landing zones with role-based access control (RBAC). GPU computing resources are provisioned within these secure environments for model training.

Sample Size

The study is expected to include approximately 7,171 eligible MRI cases from 5,208 patients with linked clinical data, of whom a subset will have corresponding preprocessed MRI images available for multimodal analysis.

CharacteristicValue
Total eligible MRI cases7,171
Total patients5,208
csPCa positive (ISUP ≥2\ge2)2,448 (34.1%)
csPCa negative4,723 (65.9%)
Median age at MRI (IQR)66 (60–73) years

Table 3. Summary of cohort characteristics. Counts and the age distribution are given per MRI case unless stated otherwise.

Sample Size Justification

Formal sample size calculations for prediction models follow the method of Riley et al. (Riley et al., [5]). With an anticipated outcome prevalence of 34%, ≥20\ge20 candidate predictor parameters (imaging features + clinical variables), a target Cox-Snell R2R^2 of 0.15, and a maximum allowable shrinkage factor of 0.9, the minimum sample size required is approximately 1,200 patients. Our available cohort of >7,000>7{,}000 patients substantially exceeds this minimum, providing adequate statistical power for both primary analysis and subgroup evaluations.

Statistical Analysis Plan

Descriptive Statistics

Baseline characteristics will be summarised by csPCa status, using means (SD) or medians (IQR) for continuous variables and frequencies (%) for categorical variables. Missingness will be reported per variable and separately for the adaptation, validation, and test sets, since missingness rates range from 8% to 78% across the clinical predictors (Table 8).

Bias TypeRiskExpected ImpactMitigation and Remaining Issues
Information biasVariable MRI quality across scanners, protocols, and time; inter-reader variability in the radiologist score, and possible variation in the scale usedLowStandardised preprocessing; as a pioneering protocol, remaining scanner-, protocol-, and reader-related effects will be further assessed through external validation
Measurement bias (image-derived features)Adipose tissue classified by signal intensity rather than by a fat-specific sequence; seminal vesicle fluid may be counted as fat near the gland base; volumes underestimated where the gland exceeds the field of viewModerateQuality control and segmentation validation (§ 5.6); sensitivity analyses on band width and gland truncation (§ 8.9); thresholds are cohort-specific and not transferable to other protocols
ConfoundingUnmeasured confounders (e.g., prior negative biopsy)LowAdjustment using available covariates and sensitivity analyses; residual confounding remains an open research question
Missing data biasDifferential missingness (8–78% across variables); LLM-extracted variables may have systematic extraction failuresHighMissing-data handling is variable-dependent and follows the procedures described in § 5.4 and 5.5, including missingness-aware modelling, sensitivity analyses, and validation of LLM-derived variables against manual review
Algorithmic biasPerformance disparity across demographic subgroupsLowComparative evaluation across combining strategies (§ 8.1.2), together with subgroup analyses (§ 8.8), will identify potential algorithm-dependent disparities and inform future mitigation and external validation

Table 8. Additional sources of bias and mitigation strategies.

Image-only Model and Data Splitting

The image-only model is PI-CAI-2B, an AI system for the detection of csPCa (ISUP grade group ≥2\ge2) on bpMRI, developed and validated in the SCARLET-1 study ((Saha et al., 2025, 2026)). It is a further development of the top-performing algorithm of the PI-CAI study ((Saha et al., 2024)): a three-dimensional nnU-Net (v1) model ((Isensee et al., 2021)) with a MedNeXt-3XL encoder–decoder of approximately 1.0×1091.0 \times10^9 parameters, trained as five cross-validation folds. whose outputs are averaged at inference. It takes T2W, ADC, and high bb-value DWI as input and returns a voxel-level csPCa likelihood map.

The released system can combine its image score with clinical information. This study uses fold 0 of the released weights in the image-only configuration, in which no clinical variable enters the model. The case-level score is derived from the likelihood map alone: voxel likelihoods below 0.1 are set to zero, each connected lesion candidate is assigned its mean likelihood, and the case score is the highest lesion likelihood. Because the image-only model uses no clinical information, it is the reference against which the contribution of clinical variables is measured.

Development and test cohorts of PI-CAI-2B. As stated in the SCARLET-1 study protocol (Saha et al., 2025), PI-CAI-2B was trained on 18,371 MRI examinations (17,178 patients; 26 cities in 14 countries) drawn from two cohorts:

  • Primary training and internal testing cohort: 10,207 examinations (9,129 patients) acquired from 2012 to 2021 at four tertiary centres in the Netherlands and Norway, originally collected for the PI-CAI study (Saha et al., 2024). The PI-CAI training set of 9,107 examinations is used for training; its 1,000-examination test set serves as an internal test cohort.

  • Secondary training and internal testing cohort: 10,264 examinations (10,149 patients) acquired from 2012 to 2024 in 22 cities in 13 countries in Europe, Asia, North America, and Africa, collected through the EU Horizon projects ProCAncer-I and COMFORT and eight independent centres. Of these, 9,264 examinations are used for training and 1,000 are held out as a second internal test cohort.

The SCARLET-1 study tested PI-CAI-2B externally on the PRIME (Ng et al., 2025), STHLM3-MRI (Eklund et al., 2021), and IP1-PROSTAGRAM (Eldred-Evans et al., 2021) cohorts. The figures above are those of the study protocol; the published report gives the final composition of 22,445 examinations from 21,252 men (Saha et al., 2026). The internal test cohorts of the SCARLET-1 study are distinct from the UCLH internal test set of the present study.

The weights of PI-CAI-2B are fixed throughout this study. UCLH, ReIMAGINE, and BiopsyUS are not named among its development cohorts, and PRIME, STHLM3-MRI, and IP1-PROSTAGRAM were used in its development study for external testing only. Case-level lists of the development cohorts are not available, so patient-level overlap with the cohorts of this study cannot be excluded; this is reported as a limitation.

The UCLH cohort is then split into an adaptation set (10%), a validation set (10%), and a held-out internal test set (80%), stratified by csPCa status and grouped by patient, so that all cases from one patient fall in the same split. Grouping is necessary because men on active surveillance contribute serial studies that are highly correlated; splitting at case level would place near-duplicate images on both sides of the partition. The split is performed once using a fixed random seed. The adaptation set trains the clinical modulation modules, with all PI-CAI-2B weights frozen; the validation set is used for model selection, for Platt recalibration, and to fix decision thresholds; and the test set is not used at any point during development.

Model Architecture

For consistency, model terminology used throughout the study is defined in Table 4.

TermDefinition
Image-only modelModel using bpMRI predictors alone (T2W, high bb-value DWI, and ADC): PI-CAI-2B in its image-only configuration, with fixed weights (§ 7.2).
Context-only modelModel using clinical predictors without the image encoder. Group III predictors, although computed from the MRI, enter as scalar features and are included. If not specified, this refers to the full clinical predictor set.
Image-context modelAny model combining bpMRI with one or more clinical predictors.
Full clinical predictor setAll clinical predictors included in Table 2.
Full image-context modelImage-context model combining bpMRI with the full clinical predictor set.

Table 4. Model terminology.

Image Encoder

The image encoder is the MedNeXt-3XL network of PI-CAI-2B, fold 0 (§ 7.2), which encodes the three-channel MRI input (T2W, high bb-value DWI, and ADC) prepared as in § 5.2. All of its weights are frozen and are not updated with UCLH data. For image-context models, only the clinical modulation modules of § 8.1.1 are trained.

Clinical Feature Encoder

Clinical context enters the frozen image model by feature-wise linear modulation (FiLM) ((Perez et al., 2018)), applied per variable. Each clinical variable jj has its own small MLP, which maps its value cjc_j to a channel-wise scale aj(cj)a_j(c_j) and shift bj(cj)b_j(c_j). A signal hh of the image model is modulated as

h′=h+∑jmj[aj(cj)h+bj(cj)],h' = h + \sum_j m_j \left[a_j(c_j)h + b_j(c_j)\right],

where mjm_j indicates whether variable jj is recorded (§ 5.4). The output layer of every MLP is initialised to zero, so modulation starts from the image-only model.

Combining Strategies

Three image-context combining strategies will be compared. They use the same per-variable modulation and differ only in the signal hh that it modulates (Table 5). Beyond its technical interest, the comparison may indicate how clinical context interacts with imaging findings, and so generate hypotheses for further study.

StrategyModulated signal hh
Early combiningOutput of the first encoder block: image feature maps at full spatial resolution.
Mid combiningOutput of the encoder bottleneck: the most compressed image representation.
Late combiningCase-level score of the image model, on the logit scale.

Table 5. Image-context combining strategies.

These strategies represent different ways in which clinical information may contribute to diagnostic reasoning. Early combining lets clinical context change how imaging features are formed. Mid combining lets it adjust the summary of the image. Late combining changes only the final risk estimate, as clinical risk calculators or nomogram-based decision support adjust risk at the decision level.

Training and Calibration

Only the modulation modules are trained, on the adaptation set, with a binary cross-entropy loss. For early and mid combining, the case score of the full inference pipeline cannot be differentiated, so the loss is computed on a proxy case score taken from the top-64 lesion logits. Model selection uses the validation set. Every model, including the image-only model, is then recalibrated by Platt scaling on the validation set, and this calibration is fixed before the test set is evaluated.

Performance Metrics

The primary objective is a comparison, so the primary estimand is the difference between the full image-context model and the image-only model on the held-out UCLH test set, reported with a 95% confidence interval rather than as two separate point estimates. Because both models are applied to the same cases, all comparisons are paired: differences in AUC-ROC are tested using DeLong’s method (DeLong et al., 1988), and differences in every other metric are obtained by paired bootstrap resampling of cases with 2,000 replicates. The context-only model is reported alongside as a reference for the performance attainable without imaging, and is not the subject of a formal comparison.

A model may raise discrimination while worsening calibration, or improve average discrimination without improving performance at the operating point where a biopsy decision is actually made, and these are different failures with different clinical consequences. The metrics in Table 6 are therefore prespecified as a panel spanning four distinct questions - overall discrimination, performance at a clinically relevant operating point, calibration, and net clinical benefit - and a claim that clinical context improves detection requires improvement in discrimination without degradation in calibration.

MetricDefinitionQuestion it answers
AUC-ROCArea under the receiver operating characteristic curveOverall discrimination, independent of threshold
AUPRCArea under the precision-recall curveDiscrimination in the positive class, which ROC can flatter
Sensitivity, specificityAt the prespecified operating points (95% and 90% sensitivity)Performance where the biopsy decision is made
PPV, NPVPredictive values at the same operating pointsHow a prediction should change management
CalibrationCalibration plot, calibration slope and intercept, Brier scoreWhether predicted risks can be believed as probabilities
Net benefitDecision curve analysis across clinically plausible threshold probabilitiesWhether adding clinical context is worth acting on

Table 6. Prespecified panel of performance metrics. All are reported for every model.

Every metric in the panel is reported for every model, in both directions, irrespective of whether the result favours the multimodal model. No metric is selected, added, or dropped after the test set has been examined. This constraint is what allows a panel of metrics to be more informative than a single one.

Threshold-dependent metrics require a prespecified operating point. Sensitivity, specificity, PPV, and NPV are reported at the threshold giving 95% sensitivity in the validation set, reflecting the clinical priority of not missing csPCa, and additionally at the threshold giving 90% sensitivity. Thresholds are fixed on the validation set and applied unchanged to the test set.

Only the primary comparison is confirmatory. All secondary objectives, including the ablation analyses, are estimation exercises reported with confidence intervals, both without and with adjustment for multiple comparisons; given the number of comparisons involved, individual ablation results are interpreted as ranking evidence.

Image-only Model Evaluation

The image-only model is evaluated with frozen weights, preprocessing, and inference settings throughout; no evaluation cohort, internal or external, is used to update the model or its decision thresholds. All evaluations use the case-level score of the model after Platt recalibration on the validation set (§ 8.1.3), together with the metrics in Table 6.

External Testing

The image-only model and the image-context models will be evaluated on five external cohorts (Table 7). PRIME, RelMAGINE, and BiopsyUS come from diagnostic settings; STHLM3-MRI and IP1-PROSTAGRAM come from population-based screening. All weights and decision thresholds are fixed after development on UCLH data, and no external cohort is used to update them.

CohortSetting and populationClinical predictorsOutcome
PRIME ((Ng et al., 2025))Primary diagnosis. International multicentre randomised trial comparing biparametric with multiparametric MRI. External test cohort of the SCARLET-1 study.Age, PSA, prostate volume, PSA density, ethnicity, DRE, family history of prostate cancer, comorbidities, obesityBiopsy grade group; unbiopsied men are labelled only as the trial reference standard allows
ReIMAGINE ((Marsden et al., 2022))Primary diagnosis. UK multicentre prospective cohort of men referred for MRI with suspected prostate cancer and a suspicious MRI (score ≥3\ge3).PSA, prostate volume from the MRI report; PSA density derivedBiopsy Gleason grade; the overall or maximal grade rule is fixed before analysis
BiopsyUS ((Urological Research Network, 2023; Min et al., 2025))Diagnosis. US registry of men undergoing MRI/ultrasound-guided biopsy (NCT02381990); 1,898 patients, 1,976 studies.Age, derived from dates; prostate volume from segmentationCore-level ISUP grade group; case label from the highest grade
STHLM3-MRI ((Eklund et al., 2021))Population-based screening, Stockholm, Sweden. External test cohort of the SCARLET-1 study.Age, PSA, prostate volumeGrade group ≥2\ge2 by the trial reference standard
IP1-PROSTAGRAM ((Eldred-Evans et al., 2021))Population-based screening, London, UK. External test cohort of the SCARLET-1 study.Age, PSA, prostate volume, ethnicityGrade group ≥2\ge2 by the trial reference standard; few positive cases limit precision

Table 7. External testing cohorts. Clinical predictors are those recorded in each cohort, before mapping to the definitions of Table 2.

The cohorts differ in which clinical predictors were recorded. A predictor that a cohort did not record is treated as missing and contributes nothing to the image-context models (§ 5.4). Each cohort therefore tests the models under its own pattern of available clinical information. Only predictors whose definition and timing match those of Table 2 are mapped; other fields are treated as missing.

Eligible cases have T2W, ADC, and high bb-value DWI and an outcome that can be mapped to ISUP grade group ≥2\ge2. Performance is reported separately for each cohort, together with the number of eligible cases, the number of csPCa-positive cases, and the completeness of each clinical predictor. The two screening cohorts are included subject to confirmation of local data access; if either is unavailable, this is reported as a deviation from the protocol.

Ablation Studies

To quantify the contribution of individual clinical predictors to multimodal prediction, add-one-in (AOI) and leave-one-out (LOO) analyses will be performed separately for each combining strategy:

  • Add-one-in: The image-only model serves as the reference. Each clinical predictor is added individually to bpMRI using the specified combining strategy. Its effect is assessed by the change in model performance relative to the image-only model.

  • Leave-one-out: The full image-context model serves as the reference. Each clinical predictor is removed individually while all remaining clinical predictors are retained. Its effect is assessed by the change in model performance relative to the full image-context model.

Each ablation experiment constitutes an independent full training run to avoid masking effects.

Determinants of Variable Contribution

The incremental value of each clinical predictor, estimated by the add-one-in analysis, will be related to three characteristics of that predictor: its source (structured extract, derivation,

automated image analysis, or extraction from free text), its missingness rate in the adaptation set, and, for free-text-derived variables, its validated extraction reliability (Table S3). The association will be summarised descriptively, principally as incremental performance plotted against missingness rate with points coloured by source; given the small number of predictors, it will not be formally tested.

This analysis is exploratory. It asks whether variables that are harder to obtain reliably also contribute less, which bears on which clinical information is worth the cost of capturing in a future prospective system.

Interpretability Analysis

SHAP (SHapley Additive exPlanations) (Lundberg & Lee, 2017) values will be computed on the test set to quantify the contribution of each clinical feature to individual predictions. Summary plots (beeswarm and bar plots) will be presented for each combining strategy.

Subgroup Analyses

Subgroup analyses ask whether the contribution of clinical context differs between groups of patients, scans, or lesions. The subgroups and their cut-off points are fixed before the test set is analysed. Clinical thresholds are used wherever an established one exists; where none exists, the cut-off points are the tertiles of the adaptation set.

  • Patient characteristics:

    • Age: <60<60, 60–69, ≥70\ge70 years.

    • Ethnicity: White, Black, Asian, Other/Not stated. The last subgroup combines the Mixed, Other, and Unknown categories of Table 2.

    • BMI: <25<25, 25–29.9, ≥30\ge30 kg/m2^{2}.

    • Smoking status: never, former, current.

    • Diabetes: present, absent.

  • Inherited risk: family history of prostate cancer (present, absent); family history of breast or ovarian cancer (present, absent).

  • PSA and examination:

    • PSA level: <4<4, 4–10, 10–20, >20>20 ng/mL.

    • PSA density: <0.10<0.10, 0.10–0.19, ≥0.20\ge0.20 ng/mL/mL.

    • DRE finding: normal, abnormal.

  • Treatments that alter PSA or the gland: 5-ARI use at MRI (yes, no); prior BPH intervention (yes, no).

  • Radiologist-derived context:

    • Radiologist score: 1–2, 3, 4, 5.

    • Number of lesions with radiologist score ≥3\ge3: 0, 1, ≥2\ge2.

    • Index lesion zone: peripheral zone, transition zone, other or multiple zones. Restricted to cases with a reported lesion.

  • Image-derived measurements:

    • Prostate volume: <40<40, 40–79, ≥80\ge80 mL.

  • PPAT measurements, each divided at tertiles: PPAT volume, PPAT / prostate volume ratio, PPAT thickness, PPAT fat fraction, and retropubic gap.

  • Image-only model score: low-score (below the median score of the image-only model in the adaptation set) and high-score (at or above that median).

  • Acquisition: field strength (1.5T, 3T); high bb-value series acquired or computed (§ 5.2).

For a variable that can be missing, cases in which it was not recorded form a separate not recorded subgroup. This subgroup is reported because missingness itself may carry information. Subgroups are defined by the recorded values of the variables, whether or not those variables are inputs to the model under evaluation.

Subgrouping by image-only model score uses the output of one of the models being compared. Selecting on this score restricts its range within each subgroup, so the discrimination of the image-only model within a subgroup is expected to be lower than that in the whole cohort, and the difference between models is interpreted accordingly.

Within each subgroup, every model is evaluated with the metrics of Table 6. The difference between the full image-context model and the image-only model is reported with a 95% confidence interval, estimated as in the primary comparison. Whether this difference varies across the levels of a subgrouping variable is assessed by an interaction estimate obtained by paired bootstrap resampling.

A subgroup with fewer than 30 csPCa-positive or 30 csPCa-negative cases is merged with the adjacent level, or is reported descriptively where no adjacent level exists. All subgroup analyses are exploratory. They are reported in full, without adjustment for multiple comparisons, and are not used to support claims about the primary objective.

Sensitivity Analyses

  1. Verification bias correction: Inverse probability weighting (IPW) to account for differential biopsy rates between radiologist score categories.

  1. Complete case analysis: Repeating primary analyses restricted to patients with <20%<20\% missing clinical variables.

  1. Alternative outcome definitions: Using any-grade prostate cancer (ISUP ≥1\ge1) as an alternative outcome.

  1. Pathology linkage interval: Repeating the labelling procedure with intervals of 6 and 12 months in place of 3 months, to quantify the effect of the linkage window on the proportion of cases labelled by histopathology and on model performance. Relabelling does not require the image encoder to be retrained.

  1. PPAT band width: Repeating the analyses with PPAT defined at 10 mm and 20 mm from the prostate surface in place of 15 mm.

  1. Gland truncation: Repeating the analyses after excluding cases in which the prostate mask reaches the boundary of the field of view (§ 5.6).

Software

All analyses will be performed in Python 3.10+ using PyTorch (deep learning), scikit-learn (classical ML comparisons), and SHAP (interpretability). Statistical comparisons between AUCs will use DeLong’s test (DeLong et al., 1988). Code and trained model weights will be made available upon publication.

Bias Assessment

This study is subject to several domain-specific biases that are intrinsic to the prostate cancer diagnostic pathway and the retrospective study design. These are discussed below alongside general methodological biases.

PSA Referral Bias

The cohort is conditioned on referral for prostate MRI, which in clinical practice is triggered primarily by elevated or rising PSA. This creates a PSA referral bias: patients entering the cohort have, by definition, already been flagged by PSA testing, and the distribution of PSA values in this referred population does not reflect the general male population. Consequently, the predictive value of PSA within this cohort may be attenuated compared with a screening population (“PSA has already done its job” by the time MRI is performed). Models trained on this cohort may overestimate the marginal contribution of imaging relative to PSA, and performance metrics may not generalise to settings where PSA-based pre-selection has not occurred (e.g., opportunistic MRI screening).

Patient Selection Bias

UCLH is a high-volume tertiary referral centre with particular expertise in prostate MRI. The patient population is therefore enriched for complex cases, higher radiologist scores, and a higher baseline prevalence of csPCa (34%) compared with community or screening settings. Additionally, the ethnic and socioeconomic composition of the UCLH catchment area (central London) may not be representative of the wider UK or international populations. These factors limit the direct generalisability of model performance estimates. External validation on datasets from community hospitals and diverse populations will be essential before clinical deployment.

Classification Task Bias

The outcome definition (csPCa as ISUP ≥2\ge2 on biopsy) introduces a classification task bias with several components:

  • Verification bias: Not all patients undergo biopsy; those with a radiologist score of 1–2 are less likely to be biopsied, leading to under-ascertainment of csPCa in low-risk groups. The magnitude of this bias will be assessed using inverse probability weighting (IPW), as specified in § 8.9, with weighted and unweighted model performance compared.

  • Biopsy sampling error: Targeted and systematic biopsies may miss significant cancer, with studies reporting approximately 5-16% of csPCa being missed when systematic biopsy is omitted and around 9% misclassification of clinically significant disease with MRI-targeted biopsy alone (Ahdhoot et al. (2020); Lee et al. (2022)).

  • Gleason grading variability: Inter-observer agreement for Gleason grading, particularly at the 3+3 vs. 3+4 boundary, is imperfect. The binary threshold (ISUP ≥2\ge2) is clinically standard but introduces classification noise at the decision boundary. A UK reproducibility study reported 78% agreement with the consensus Gleason score and an overall inter-observer κ\kappa of 0.54, with lower agreement for Gleason score 7 (κ=0.44\kappa= 0.44) (Melia et al. (2006)).

  • Temporal label evolution: Imaging interpretation, biopsy practice, and pathology reporting may evolve over the study period, resulting in temporal heterogeneity in outcome ascertainment.

Other Methodological Biases

Several methodological biases may influence the estimated contribution of clinical context and the interpretation of model performance. Their anticipated impact on this study and corresponding mitigation strategies are summarised below.

Ethical Considerations

Ethical Approval

This study has been approved under the UCLH Data Access Process for Research (DAP-R), which provides a generic favourable ethical opinion (IRAS No. 299136). Individual patient consent is not required as this is a retrospective analysis of routinely collected, anonymised data. The study sponsor is UCLH.

Data Protection and Privacy

  • All data will be pseudonymised at extraction via the OMOP ES pipeline and stored in the UCL Data Safe Haven or UCLH Azure landing zones with RBAC.

  • Patients registered on the National Data Opt-Out will be automatically excluded.

  • DICOM tags will be anonymised prior to image conversion and analysis.

  • Statistical disclosure control techniques will be applied to prevent re-identification through unique attribute constellations.

  • The study complies with the UK General Data Protection Regulation (UK GDPR) and the Data Protection Act 2018. The lawful basis for processing is Article 6(1)(e) (public task) and Article 9(2)(j) (scientific research).

Supplementary Material

Radiology Report Keyword Rules

The keyword rules in Table S1 apply only at step 4 of the outcome labelling procedure (§ 5), that is, only to cases for which no histopathology, no recorded lesion count, and no assessment score is available. They are a last-resort rule set applied to a minority of cases, and they are deliberately conservative: a case is labelled csPCa-negative only when the report contains an explicit statement of absence, and any expression of diagnostic uncertainty prevents the label from being assigned.

Rules are applied to the Conclusion or Impression section of the report where one can be identified, and to the full report text otherwise, since the concluding statement carries the reporting radiologist’s overall assessment. Matching is case-insensitive and follows whitespace and punctuation normalisation.

Exclusion patterns (E1–E8) are evaluated before inclusion patterns (N1–N9). A case matching any exclusion pattern is not labelled at this step and is excluded from the cohort, because step 4 is the final opportunity for ascertainment. Negation is resolved using a scope-limited negation detection method of the NegEx family rather than by literal string matching, so that a negation term applies only to findings within the same clause; this prevents a phrase such as “no extracapsular extension” from negating a suspicious lesion described in an adjacent sentence.

The rule set will be frozen, with a version identifier, before it is applied to the study cohort. Any subsequent amendment will be recorded with its date, rationale, and the number of cases whose label changes as a result.

IDPattern (case-insensitive)Rationale
N9no MRI evidence of (significant)?diseaseExplicit absence of disease on MRI

Table 10.

LLM Extraction Schema and Prompt

Table S2 gives the extraction schema submitted to the model, one row per target variable. The schema is enforced twice: it is stated in the prompt, and parsed outputs are validated against it before any value enters the analysis dataset (§ 5.5). The system prompt reproduced below is applied unchanged to every patient; the per-variable instructions in Table S2 are appended to it as the JSON schema description.

You extract structured clinical data from prostate cancer records.

Rules:

1. Return a single JSON object matching the schema. No prose, no markdown.

2. Return null for any field not stated in the source text. Never infer, estimate, or carry a value forward from another field.

3. Return false only where the text explicitly records absence. Absence of mention is null, not false.

4. Where a value appears more than once with different values, apply the aggregation rule given in the field description (for example, maximum for radiologist scores, highest for Gleason score).

5. Do not infer a diagnosis from medication alone.

6. Quote the source verbatim in the accompanying _evidence field for every non-null value. If you cannot quote it, return null.

7. Report dates in ISO 8601 (YYYY-MM-DD). Return null for partial dates.

Rule 6 requires the model to return a supporting quotation alongside each extracted value. Values whose quotation cannot be located in the source document by exact string match are set to null during post-processing. This converts an unverifiable assertion into a detectable failure and provides the source linkage described in § 5.5.

VariablePermitted valuesExtraction instruction
From radiology reports
Radiologist scoreInteger 1–5Return the highest level-of-suspicion score stated for any lesion, whether reported as a PI-RADS category or as a Likert score. Where several lesions are scored, return the maximum. Record which scale was used in the paired score_scale field. Do not infer a score from a narrative description of suspicion.

Table S2. Per-variable extraction schema, covering only those variables sourced from free text (Table 2, source LLM) together with the report fields used for outcome labelling. Every field additionally accepts null, and every field has a paired _evidence field holding the supporting quotation.

VariablePermitted valuesExtraction instruction
Lesion countInteger ≥0\ge0Count lesions explicitly enumerated or individually described. Where the report states that no lesion is present, return 0. Where lesions are referred to collectively without enumeration, return null.
From histopathology reports
Gleason scoreString, a+bReturn the highest Gleason score across all cores in the specimen. Where primary and secondary patterns are reported separately, combine them in the stated order.
ISUP grade groupInteger 1–5Return the grade group as stated. Where only a Gleason score is given, return null; the mapping is applied during post-processing rather than by the model.
From clinical notes
Family history of prostate cancerBooleanReturn true only where a blood relative with prostate cancer is recorded. Return false only where family history is explicitly documented as negative. Relatives by marriage do not count.
Family history of any cancerBooleanAs above, for any malignancy in a blood relative. Where true, list the cancer types in the paired family_cancer_types array.
History of other cancersBooleanReturn true where the patient has a personal history of a malignancy other than prostate cancer. Benign neoplasms and in-situ lesions do not count. Where true, list types in other_cancer_types.
DiabetesBooleanReturn true where recorded as a current or past diagnosis, including type 1 and type 2. Do not infer from glucose-lowering medication alone.
HypertensionBooleanReturn true where recorded as a diagnosis. Do not infer from antihypertensive medication or from isolated blood pressure readings.
Cardiovascular diseaseBooleanReturn true for ischaemic heart disease, myocardial infarction, heart failure, atrial fibrillation, stroke, or peripheral arterial disease. Hypertension alone does not count.
COPDBooleanReturn true where recorded as a diagnosis. Emphysema and chronic bronchitis count; asthma does not.
AsthmaBooleanReturn true where recorded as a diagnosis.
Chronic kidney diseaseBooleanReturn true where recorded as a diagnosis or where a CKD stage is stated. Do not infer from a single creatinine or eGFR value.
Smoking statusInteger 0–20 = never smoker; 1 = former smoker; 2 = current smoker. Where the text records only “smoker” without qualification, return 2. The integer code is one-hot encoded during post-processing.

Table 1 (PDF p. 34).

VariablePermitted valuesExtraction instruction
Alcohol statusInteger 0–6Ordinal band of reported weekly consumption in UK units: 0 = none; 1 = 1–7; 2 = 8–14; 3 = 15–21; 4 = 22–35; 5 = 36–50; 6 = >50. Where consumption is described qualitatively without a quantity, return null. The integer code is one-hot encoded during post-processing.

Table S2 (PDF p. 35).

Duplicate Extraction and Adjudication

Table S3 is the reporting template for the two checking procedures described in § 5.5: duplicate extraction, which quantifies the stability of the extraction process, and adjudicated re-review, which tests the reference standard against which the process is judged. Both are reported per variable, since reliability differs markedly between variables that are stated explicitly in structured-like text and those that must be inferred from narrative.

Discrepancies are classified as source ambiguity, annotator error, or extraction error. The distinction matters because only extraction error is attributable to the pipeline and correctable by prompt revision; source ambiguity sets a ceiling on the accuracy any extraction method could achieve for that variable, and is reported as such rather than counted against the model.

References

  1. [1]Sung H, Ferlay J, Siegel RL, et al. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J Clin. 2021;71(3):209-249.DOI
  2. [2]Ahmed HU, El-Shater Boısaly A, Brown LC, et al. Diagnostic accuracy of multi-parametric MRI and TRUS biopsy in prostate cancer (PROMIS): a paired validating confirmatory study. Lancet. 2017;389(10071):815-822.DOI
  3. [3]Kasiviswanathan V, Rannikko AS, Borghi M, et al. MRI-Targeted or Standard Biopsy for Prostate-Cancer Diagnosis. N Engl J Med. 2018;378(19):1767-1777.
  4. [5]Riley RD, Ensor J, Snell KIE, et al. Calculating the sample size required for developing a clinical prediction model. BMJ. 2020;368:m441.DOI
  5. [6]Collins GS, Moons KGM, Dhiman P, et al. TRIPOD+AI statement: updated reporting guidance for clinical prediction models that use regression or machine learning methods. BMJ. 2024;385:e078378.DOI

Paper details

vlnc:W_g42c4ecw
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents