WorksComputer Science
CONTEXT-PCa: Evaluating the role of clinical context in MRI-based AI for detecting clinically significant prostate cancer (study protocol)
Abstract
The objective of this study is to investigate the role of clinical variables - their breadth, source, and quality - in the development, utility, and generalisability of an AI system for bi-parametric MRI (bpMRI)-based diagnosis of clinically significant prostate cancer (csPCa). Clinical context is added to the published PI-CAI-2B image-only model, whose weights are fixed. The study cohort comprises 6,022 biparametric MRI cases from 4,540 patients at University College London Hospitals NHS Foundation Trust (2019-2024), of which 4,834 cases (3,632 patients) are held out for internal testing. Forty-three clinical variables are drawn from structured health records, clinical letters, MRI reports, and automated image measurements. The primary endpoint is the difference in the area under the receiver operating characteristic curve (AUROC) between the image-context and image-only models on the internal test set. Secondary analyses compare combining strategies, the contribution of individual variables, and performance across subgroups. All models are further tested on five external cohorts from primary diagnosis and population-based screening.
STUDY PROTOCOL - OBSERVATIONAL RETROSPECTIVE STUDY
*Corresponding author: Yipeng Hu ([email protected])
Background and Rationale
Prostate cancer is the most commonly diagnosed male malignancy worldwide, with 1,414,249 new cases and 375,000 deaths annually (2020 GLOBOCAN estimates) [1]. Diagnosis increasingly relies on magnetic resonance imaging (MRI) followed by targeted biopsy, a pathway supported by clinical trials (PROMIS [2], PRECISION [3]) and prospective studies (MRI-FIRST (Rouvière et al., 2019) and 4M (van der Leest et al., 2019)). Bi-parametric MRI (bpMRI), which omits gadolinium-based contrast enhancement in favour of T2-weighted and diffusion-weighted sequences alone, has gained traction as a faster, cheaper, and contrast-free alternative with comparable diagnostic accuracy. In the UK alone, approximately 200,000 men undergo prostate MRI each year.
The Prostate Imaging-Reporting and Data System version 2.1 (PI-RADS v2.1) (Turkbey et al., 2019) provides a structured framework for reporting prostate MRI, yet interpretation remains subjective, radiologist-dependent, and constrained by a global shortage of trained readers. Clinically significant prostate cancer (csPCa), defined following the PI-CAI challenge labelling procedure (Saha et al., 2024) as ISUP grade group (Gleason score ) in any core, is one of the primary targets in clinical detection pathways because it is associated with with an increased risk of adverse oncological outcomes and generally requires closer surveillance or consideration of active treatment (Epstein et al., 2016; Cornford et al., 2024).
Clinical information available within the prostate cancer diagnostic pathway is highly heterogeneous. Established predictors such as PSA, PSA density, prostate volume, age, DRE findings, radiologist score, ethnicity, and family history are routinely used for risk assessment (Roobol et al., 2013; Ankerst et al., 2014; Turkbey et al., 2019; Cornford et al., 2024). Other factors, including longitudinal PSA change, BMI, lifestyle behaviours, laboratory abnormalities, comorbidities, and periprostatic adipose tissue (PPAT), are suspected to be associated with the diagnosis and prediction of prostate cancer and may provide additional clinical context, although their predictive contribution may be more population- and setting-dependent.
Despite their established clinical relevance, the added value of clinical variables within MRI-based AI systems remains uncertain. Existing studies have reported mixed results, with some showing no statistically significant benefit from adding clinical information and others demonstrating clear improvements; moreover, performance can depend substantially on how clinical and imaging information are integrated (Roest et al., 2023, 2024; Woźnicki et al., 2020; Hiremath et al., 2021). This leaves unresolved which clinical variables provide genuine complementary value beyond imaging, which may introduce redundant or dataset-specific associations that impair generalisability, and how heterogeneous clinical and imaging information should best be integrated within AI models.
Objectives
This study investigates the role of clinical variables - their breadth, source, and quality - in the development, utility, and generalisability of an AI system for bi-parametric MRI (bpMRI)-based diagnosis of clinically significant prostate cancer (csPCa, ISUP ). The objectives below give effect to that question.
Primary Objective
To determine whether combining clinical variables with bpMRI improves the detection of csPCa relative to bpMRI alone, evaluated on the held-out UCLH test set. Clinical-only models are evaluated alongside as a reference for the performance attainable without imaging.
Secondary Objectives
Quantify the incremental predictive value of individual clinical variables through add-one-in and leave-one-out ablation analyses.
Determine whether the value of clinical context depends on the stage at which it is combined with imaging features - early, mid, or late combining.
Determine whether a clinical variable’s contribution depends on its source and quality: structured extraction, automated image analysis, or extraction from free text; its missingness rate; and, for free-text variables, its validated extraction reliability.
Characterise performance heterogeneity across subgroups defined by age, ethnicity, PSA level, and radiologist score.
Identify which clinical variables drive individual predictions, and whether their contributions are consistent across combining strategies.
Evaluate the image-only and image-context models on external cohorts from primary diagnostic and population screening settings.
Study Design
This is a retrospective cohort study using routinely collected electronic health record (EHR) data and clinically acquired MRI studies from University College London Hospitals (UCLH) NHS Foundation Trust. The study follows a diagnostic prediction model development and internal validation design, reported in accordance with the STROBE statement (von Elm et al., 2007) for observational studies and the TRIPOD+AI guideline [6] for prediction model studies involving AI/machine learning.
Setting and Study Period
Data are drawn from the UCLH Epic EHR system, covering the period from April 2019 (system go-live) to March 2024. UCLH is a tertiary academic centre in central London with a high-volume prostate MRI service.
Clinical prostate MRI is performed across several UCLH sites: the UCLH Main Building, where three scanners are potentially used for prostate MRI; Westmoreland Street Hospital; the Macmillan Cancer Centre; the Grafton Way Building; and the Royal National ENT and Eastman Dental Hospitals. Approximately 12 consultant radiologists reported prostate MRI over the study period. The cohort therefore spans multiple sites, scanners, and reporting radiologists.
Prostate MRI at UCLH is reported using the Likert scale, which records the reporting radiologist’s overall level of suspicion for clinically significant cancer on a subjective five-point scale. Unlike a PI-RADS category, which is assigned from the images alone, a Likert score permits the radiologist to weigh clinical context such as PSA, prostate volume, and prior biopsy history alongside the imaging findings.
Both scales run from 1 to 5, and reporting practice may vary between sites and over the study period. Throughout this protocol, radiologist score denotes the five-point level of suspicion recorded in the report, whether assigned as a PI-RADS category or as a Likert score, and the scale actually used will be recorded alongside the value for every case. Where the distinction bears on interpretation it is stated explicitly.
Data Splitting and Evaluation Paradigm
The image-only model is the published PI-CAI-2B system (Saha et al., 2026), with fixed weights; it was developed on cohorts external to this study (Section 7.2). All development of the clinical components uses a UCLH adaptation data set. The UCLH study cohort is divided into an adaptation set (10%), for adapting the model to the local setting, a validation set (10%), and an internal test set (80%) held out for all primary analyses (Figure 1). External testing uses five further cohorts (Section 7.6).
Study Population
Unit of Analysis
The unit of analysis is the MRI case: one prostate MRI study together with the clinical data available at the time it was acquired and the csPCa status ascertained for it. A patient may contribute more than one case, since men on active surveillance undergo serial imaging. Eligibility criteria are stated at the level at which they act: criteria concerning the person (age, opt-out status) remove the patient and all of their cases, whereas criteria concerning an individual study (prior treatment, non-diagnostic image quality) remove only that case.
Inclusion Criteria
Male patients aged years.
Underwent prostate MRI at UCLH during the study period for initial diagnostic assessment or for active surveillance. Studies performed for staging of already-diagnosed prostate cancer, and studies performed for indications other than prostate cancer detection, are not eligible.
csPCa status ascertainable by the hierarchical labelling procedure of § 5, which is applied in strict priority order: (a) histopathology; then (b) absence of any recorded lesion; then (c) a maximum radiologist score (PI-RADS or Likert) ; then (d) an explicitly negative report. A case is labelled by the first applicable step, and a case for which no step applies is excluded.
Only route (a) can establish csPCa-positive status; routes (b)–(d) can establish only a presumed negative. All csPCa-positive cases therefore derive from histopathology, and the negative class is a mixture of biopsy-confirmed and presumed-benign cases. The consequences are addressed in § 9.
Exclusion Criteria
MRI studies performed after the patient received local prostate cancer treatment (radical prostatectomy, radiotherapy, brachytherapy, HIFU or focal therapy). Only studies postdating the treatment are excluded; earlier studies from the same patient remain eligible. Prior biopsy is not treatment, so studies performed in men on active surveillance who have undergone biopsy alone remain eligible. MRI after androgen deprivation or other systemic therapy is retained and flagged; because such therapy can reduce gland volume, PSA and tumour conspicuity, a sensitivity analysis excluding these cases is reported.
Patients who have opted out via the National Data Opt-Out scheme (NHS England, 2026).
MRI studies with severe artefact rendering images non-diagnostic, as judged by the reporting radiologist.

Figure 1. Flow of patients and MRI cases into the UCLH study cohort, and its division into adaptation, validation, and internal test sets. Exclusion boxes give the patients who had any excluded case, so patient counts in successive boxes need not differ by that number. Graphic text: Adult men seen at UCLH for suspected or known prostate cancer (April 2019 – March 2024) ; National Data Opt-Out: ; No MRI procedure recorded: ; MRI not linked to an imaging study: (); Participants in the PRIME or ReIMAGINE study: (); ; ; Directly indicated as post-treatment MRI or no matched MRI report in the source records; (); ; ; csPCa status not ascertainable; (); ; ; Excluded after review of clinical notes; (); MRI on or after prostate cancer treatment (prostatectomy, radiotherapy, brachytherapy, HIFU or focal therapy): 459; DL prostate segmentation failure ( mL): 31; Not a diagnostic prostate MRI: 2; ; ; No histopathology and maximum Likert score 3: csPCa status not ascertainable; (); Study cohort: MRI before any local prostate cancer treatment; ; ; Retained and flagged: prior BPH surgery 188; androgen deprivation or other systemic therapy before MRI 32; csPCa positive; (48.2%); Histopathology GG : 2,431; Biopsy GG in follow-up notes: 224; No histopathology, maximum Likert 4–5 (presumed positive): 247; csPCa negative; (51.8%); Histopathology: 1,476 (benign or negative biopsy 684; GG1 791; GG on record, labelled negative 1); Negative or GG1 biopsy in follow-up notes: 173; No histopathology: 1,471 (no lesion reported 513; maximum Likert 1–2 958); Patient-level split, stratified by csPCa status; Adaptation (10%): , ; Validation (10%): , ; Internal test (80%): , .
Data Source
Structured clinical data are extracted from the UCLH Epic Data Warehouse via the OMOP Extraction System (OMOP ES), a governance-by-default pipeline developed in collaboration between UCL ARC, DHCT, and the UCLH CRDM team. The OMOP ES applies the following safeguards automatically:
Exclusion of patients registered on the National Data Opt-Out.
Exclusion of sexual health conditions.
Replacement of postcode with Lower Layer Super Output Area (LSOA).
Prostate MRI studies are retrieved from UCLH PACS (Picture Archiving and Communication System) and the Vendor Neutral Archive (VNA). DICOM images are anonymised (removal/pseudonymisation of identifying DICOM tags) and converted to NIfTI format for analysis.
Figure 2 shows the form each source takes and the route by which it reaches the analysis dataset. Date of birth is not retained: the extract carries year of birth only, and age is derived from it and the MRI date. Clinical event dates are retained at full precision because the linkage of pathology to imaging depends on the interval between them (§ 5).

Figure 2. Data sources and the route by which each reaches the analysis dataset. The excerpts are illustrative synthetic examples, not real patient records. Shading marks the report section to which the keyword rules of Table S1 apply, and the placeholders substituted during pseudonymisation.
Variables and Outcomes
Primary Outcome
Presence of clinically significant prostate cancer (csPCa), defined following the PI-CAI challenge labelling procedure (Saha et al. (2024)) as a case-level binary label (0 = no csPCa; 1 = csPCa) derived using the following hierarchical procedure:
Histopathology (highest priority): If a Gleason score from prostate biopsy or radical prostatectomy is available, the case is labelled csPCa-positive when Gleason score (i.e., ISUP grade group ; Gleason ) and csPCa-negative when Gleason score .
Lesion count: If no histopathology is available and the radiology report records no lesion, the case is labelled csPCa-negative.
Radiologist score: If neither histopathology nor lesion count is available and the maximum radiologist score (PI-RADS or Likert) is , the case is labelled csPCa-negative. This follows the PI-CAI assumption that cases scored 1–2 without biopsy are presumed benign, applied here to whichever scale the report used.
Radiology report keywords: If the above conditions are unmet, cases with explicitly negative radiology reports (e.g., “no suspicious lesion”, “screen negative”) are labelled csPCa-negative. The complete rule set, including the exclusion patterns that prevent a label being assigned where the report expresses diagnostic uncertainty, is given in Table S1.
| ID | Pattern (case-insensitive) | Rationale |
| Exclusion patterns (evaluated first; a match prevents any label being assigned) | ||
| E1 | suspicio(us|n), concerning (unnegated) | Any surviving expression of suspicion is incompatible with an explicitly negative report |
| E2 | equivocal, indeterminate, borderline | Diagnostic uncertainty; the case should be resolved by biopsy rather than by rule |
| E3 | cannot (be )?exclude(d)?, not excluded | Explicit non-exclusion of malignancy |
| E4 | recommend(ed)? biopsy, for biopsy, proceed to biopsy | Onward action implying residual suspicion |
| E5 | extracapsular extension, seminal vesicle invasion, ECE, SVI | Staging features of established disease |
| E6 | metasta(sis|ses|tic), lymphadenopathy, bone lesion | Disease beyond the prostate |
| E7 | progression, interval increase, enlarging | Change implying active disease, relevant to active surveillance studies |
| E8 | (probabl(e|y)|likely|favour(s|ed)?) within the same clause as a malignancy term | Hedged positive assertion |
| Inclusion patterns (a match in the absence of any exclusion match labels the case csPCa-negative) | ||
| N1 | no (suspicious|significant|concerning) (lesion|abnormality|focus|area) | Explicit absence of a suspicious finding |
| N2 | no evidence of (clinically significant )?(prostate )?(cancer|malignancy|tumour) | Explicit absence of malignancy |
| N3 | screen(ing)? negative | Local reporting convention for a negative study |
| N4 | normal (prostate )?(MRI|study|appearances) | Global statement of normality |
| N5 | unremarkable (prostate|appearances|study) | Global statement of normality |
| N6 | no (focal )?(lesion|abnormality) (identified|seen|detected|demonstrated|visualised) | Explicit absence of a focal finding |
| N7 | benign (appearances|features|changes) not accompanied by any E-pattern | Positive assertion of benignity |
| N8 | no target(able)? lesion | Explicit absence of a biopsy target |
Table S1. Radiology report keyword rules applied at step 4 of outcome labelling. Patterns are shown in a simplified regular-expression notation; alternation is denoted by a vertical bar and optional elements by a question mark.
Linkage of pathology to imaging. A histopathology result is linked to an MRI case only where the specimen was obtained within 3 months after the MRI and before any intervening prostate cancer treatment. Where a patient has several MRI studies and several specimens, each specimen is linked to the most recent preceding eligible MRI, and each MRI case takes the earliest specimen linked to it. An MRI case with no specimen inside the window is labelled by the remaining steps of the hierarchy rather than by a later specimen, so that disease detected long after the index study is not attributed to it. The sensitivity of the results to this interval is examined in § 8.9.
Alignment with PI-CAI. This procedure aligns with the labelling rules of the PI-CAI challenge (Saha et al., 2024): the priority given to histopathology, the ISUP grade group threshold, and the presumption that unbiopsied cases scored 1–2 are benign are all adopted unchanged.1
Imaging Predictors
Prostate MRI studies at UCLH are acquired on 1.5T and 3T scanners as part of routine clinical care. The bi-parametric MRI protocol comprises three input channels.
| Sequence | Purpose | Key Parameters |
| T2-weighted imaging (T2W) | Anatomical and zonal assessment and lesion morphology | High-resolution T2W imaging |
| High -value diffusion-weighted imaging (HBV) | Detection of restricted diffusion and lesion conspicuity | High -value s/mm |
| Apparent diffusion coefficient (ADC) map | Quantitative assessment of diffusion restriction | Calculated from diffusion-weighted imaging (DWI) acquired at multiple -values |
Table 1. Bi-parametric MRI sequences used as imaging predictors.
Image preprocessing.
Format conversion: DICOM series are converted to NIfTI using dcm2niix. Manufacturer rescaling is applied at conversion so that ADC maps are expressed in mm/s irrespective of vendor; series whose scaling cannot be resolved are excluded. The high -value series is identified from the -value recorded in the standard or vendor-private tags, and whether it was acquired or computed is recorded for each case.
Anonymisation: Identifying tags are removed or pseudonymised following the Basic Application Level Confidentiality Profile of DICOM PS3.15 Annex E. Series liable to carry burned-in annotation, such as secondary captures, are excluded.
Spatial standardisation: T2W, high -value DWI, and ADC are resampled into a common reference frame at a fixed voxel spacing of mm, to a uniform matrix of about the image centre. This reference frame is used for the image-derived measurements (§ 5.6); the input to the image model is prepared separately in step 4.
Model-specific preprocessing: The input to PI-CAI-2B is prepared with the preprocessing released with the model (Saha et al., 2026): the prostate is segmented on T2W; ADC and high -value DWI are registered to T2W; the volumes are cropped to the prostate bounding box with a 10 mm margin, and resampling and intensity normalisation follow the nnU-Net plans of the model (Isensee et al., 2021). The preprocessing is identical to that used to train PI-CAI-2B, necessary for using its fixed weights (§ 8.1).
Quality control: Converted volumes are checked for correct orientation, plausible geometry, and non-empty content. In addition, the median ADC within the segmented prostate is computed per scanner and compared against a physiologically plausible range of approximately –; a systematic departure indicates unresolved vendor scaling rather than a biological difference. Failures are listed and excluded.
Clinical Predictor Variables
| Variable | Source | Type | Transform | Normalisation |
| I. Patient context | ||||
| Age | Structured EHR | Continuous | None | Z-score |
| Ethnicity | Structured EHR | Categorical (6) | One-hot encoded | None |
| Height (cm) | Structured EHR | Continuous | None | Z-score |
| Weight (kg) | Structured EHR | Continuous | None | Z-score |
| BMI (kg/) | Structured EHR | Continuous | Z-score | |
| BMI change | Structured EHR | Continuous | None | Z-score |
| Smoking status | LLM | Categorical (3) | One-hot encoded | None |
| Alcohol status | LLM | Categorical (7) | One-hot encoded | None |
| Family hx any cancer | LLM | Binary | Binary encoded (0/1) | None |
| Family hx prostate ca. | LLM | Binary | Binary encoded (0/1) | None |
| Family hx breast ca. | LLM | Binary | Binary encoded (0/1) | None |
| Family hx ovarian ca. | LLM | Binary | Binary encoded (0/1) | None |
| Hx other cancers | LLM | Binary | Binary encoded (0/1) | None |
| Diabetes | LLM | Binary | Binary encoded (0/1) | None |
| Hypertension | LLM | Binary | Binary encoded (0/1) | None |
| Cardiovascular disease | LLM | Binary | Binary encoded (0/1) | None |
Table 2. Clinical predictor variables, data sources, types, and preprocessing.
| Variable | Source | Type | Transform | Normalisation |
| COPD | LLM | Binary | Binary encoded (0/1) | None |
| Asthma | LLM | Binary | Binary encoded (0/1) | None |
| Chronic kidney disease | LLM | Binary | Binary encoded (0/1) | None |
| DRE finding | Structured EHR | Binary | Binary encoded (0/1) | None |
| PSA, laboratory (ng/mL) | Structured EHR | Continuous | Z-score | |
| PSA change, laboratory | Structured EHR | Continuous | None | Z-score |
| PSA, MRI report (ng/mL) | LLM | Continuous | Z-score | |
| PSA change, MRI report | Derived | Continuous | None | Z-score |
| Blood | Structured EHR | Binary | Binary encoded (0/1) | None |
| Urine | Structured EHR | Binary | Binary encoded (0/1) | None |
| 5-ARI use at MRI | LLM | Binary | Binary encoded (0/1) | None |
| Aspirin / NSAID use at MRI | LLM | Binary | Binary encoded (0/1) | None |
| Prior BPH intervention | LLM | Binary | Binary encoded (0/1) | None |
| II. Radiologist-derived context | ||||
| Radiologist score (1–5) | LLM | Ordinal | None | None (raw) |
| MRI lesion count | LLM | Count | Z-score | |
| Lesion count with radiologist score | LLM | Count | Z-score | |
| Overall lesion laterality | LLM | Categorical | One-hot encoded | None |
| Index lesion side | LLM | Categorical | One-hot encoded | None |
| Index lesion zone | LLM | Multi-label categorical | Multi-hot encoded | None |
Table 9.
| Variable | Source | Type | Transform | Normalisation |
| Index lesion level | LLM | Multi-label categorical | Multi-hot encoded | None |
| III. Quantitative imaging context | ||||
| Prostate volume (mL) | DL segmentation | Continuous | Z-score | |
| PSA density (ng/mL/mL) | Derived | Continuous | Z-score | |
| PPAT volume (mL) | DL segmentation | Continuous | Z-score | |
| PPAT / prostate volume ratio | Derived | Continuous | Z-score | |
| PPAT thickness (mm) | Derived | Continuous | None | Z-score |
| PPAT fat fraction | Derived | Continuous (0–1) | None | Z-score |
| Retropubic gap (mm) | Derived | Continuous | None | Z-score |
Table 2 (PDF p. 13).
Source key: Structured EHR = extracted directly from structured fields in the Epic EHR (OMOP), including structured Parquet exports; DL segmentation = prostate volume computed from automated deep learning segmentation of the MRI; Derived = calculated from other variables (PSA density = PSA / prostate volume); LLM = extracted from unstructured text (radiology reports, histopathology reports, and clinical notes) using the large language model pipeline described in § 5.5.
Missing Data Handling
In the image-context models, missing values are not imputed. Each clinical variable carries an indicator , equal to 1 when the variable is recorded and 0 when it is missing, and its contribution to the model is multiplied by (§ 8.1.1). A missing variable therefore contributes nothing, and a case with no recorded variable receives exactly the image-only prediction. Image-derived measurements that fail quality control, or that cannot be measured for a case (§ 5.6), are treated as missing in the same way.
LLM-Based Variable Extraction
Several clinical variables listed in Table 2 are not available as structured data fields and must be extracted from unstructured free-text clinical documents. A large language model (LLM)-based natural language processing (NLP) pipeline will be developed for this purpose. The pipeline is designed to maximise extraction reliability and auditability through the following stages.
Source document identification and preparation. Three categories of unstructured text will be used: (i) radiology reports, from which radiologist scores, lesion counts, lesion location (overall laterality, and the side, zone, and level of the index lesion), and the PSA value and prostate volume stated in the report are extracted; (ii) histopathology reports, from which Gleason scores and ISUP grade groups are extracted; and (iii) clinical notes (consultation letters, clinic letters, referral letters, and pre-operative assessments), from which comorbidities, lifestyle factors, family history (including breast and ovarian cancer), history of other cancers, medication use at the time of MRI (5-ARI, aspirin or NSAID), and prior BPH intervention are extracted.
Variables recorded in structured fields, including anthropometric measurements and digital rectal examination findings, are taken from the structured extract (Table 2); prostate volume and the PPAT measurements are computed from the MRI (§ 5.6). All documents are exported from the OMOP-mapped EHR, split into per-patient corpora, and pseudonymised (removal of patient names, NHS numbers, dates of birth, and free-text addresses) prior to LLM processing.
2. Prompt design and extraction schema. Each patient’s documents are submitted to the LLM with a structured JSON schema prompt that (a) enumerates every target variable with its permissible data type and value set, (b) mandates explicit null returns for any variable not mentioned in the source text, preventing the model from inferring or fabricating values, and (c) includes task-specific instructions (e.g., “distinguish between cancer-related and non-cancer family history”, “do not infer a diagnosis from medication alone”). Prompts are developed iteratively on a held-out development set of 50 patients and frozen before production extraction begins. The system prompt and the per-variable extraction schema are reproduced in full in Section A.2 and Table S2.
3. Model selection and execution. The extraction will use a commercial LLM accessed via API (model selection to be finalised based on benchmark performance on the development set; candidates including Claude Sonnet 4.5, Gemini 2.5 Pro, and GPT-5.2). The selected model will process each patient’s notes in a zero-shot setting with the frozen prompt. To ensure reproducibility, the model version, API parameters (temperature = 0, maximum output tokens), and prompt text will be recorded and version-controlled. An automated execution pipeline will handle rate-limiting, retry logic (exponential back-off with a maximum of three retries per patient), and graceful degradation for notes exceeding the model’s context window (truncation with a trailing marker).
4. Output parsing and post-processing. Raw LLM outputs will be parsed as JSON with the following safeguards: (i) responses wrapped in markdown code fences are automatically unwrapped; (ii) malformed JSON triggers one retry with an explicit “return valid JSON only” instruction; (iii) parsed outputs are validated against the extraction schema (type checking, range checking for numerical fields, controlled vocabulary matching for categorical fields). After parsing, rule-based post-processing standardises free-text entries – for example, cancer type descriptions are mapped to a controlled vocabulary using regular-expression rules, and benign conditions erroneously included in cancer fields are filtered out. Internal consistency checks are applied (e.g., if family history of prostate cancer is flagged as true, “prostate cancer” must appear in the cancer types list). Smoking status and alcohol status are extracted as integer codes (Table S2) and one-hot encoded for modelling (Table 2). One-hot encoding discards the order of the seven alcohol bands but allows a non-monotonic association with cSPCa, since the dose–response relation between alcohol and prostate cancer may be non-linear ((Hong et al., 2020)). Figure 3 works through the sequence for one document, including a value that is discarded because its quotation cannot be located.

Figure 3. How a value is extracted from free text and what prevents one being invented. The document is an illustrative synthetic example, not a real patient record. Each non-null value must be returned with a supporting quotation, which is then located in the source document after normalisation of case and whitespace; a value whose quotation cannot be found is discarded rather than accepted, as shown for copd. The three states true, false and null are distinct: absence of mention is null, never false.
5. Duplicate extraction and stability assessment. A random 10% sample of patients (minimum 300) will undergo duplicate extraction: the frozen prompt will be run a second time with the primary model, and once with a second model from the candidate set. Per-variable agreement between runs will be quantified using Cohen’s for categorical variables and the intraclass correlation coefficient for continuous variables. Two forms of instability are distinguished. Run-to-run disagreement under the same model indicates residual non-determinism despite temperature 0; variables falling below will be examined for prompt ambiguity and the prompt revised before it is frozen. Between-model disagreement indicates model-specific interpretation of the prompt; it is reported descriptively and used to bound the model-dependence
of downstream results. This assessment is performed on the development set before the prompt is frozen, so it does not consume the validation sample described below.
6. Validation against manual chart review. A random sample of at least 200 patients (stratified by csPCa status) will undergo independent manual chart review by two clinically trained annotators. Inter-annotator agreement will be quantified using Cohen’s . LLM extractions will then be compared against the consensus annotation to compute per-variable precision, recall, and score. Variables with will be flagged; depending on the nature of the errors, these variables will be either excluded from analysis, subjected to additional manual correction, or retained with a sensitivity analysis assessing their impact on model performance.
7. Adjudicated re-review of discordant variables. Cases in which the LLM output and the two annotators do not all agree, together with all variables failing the threshold, will undergo a second review in a smaller adjudication set of 50 patients drawn from the validation sample. Review is performed by a third clinically trained reviewer, blinded to both the LLM output and the initial annotations, who assigns an independent value from the source documents; this adjudicated value becomes the reference standard for those cases. Each discrepancy is classified as source ambiguity, annotator error, or extraction error, and counts by category are reported in Table S3. The classification matters because only extraction error is attributable to the pipeline and correctable by prompt revision, whereas source ambiguity sets a ceiling on the accuracy any extraction method could reach for that variable.
| Variable | n | Same-model agreement | Between-model agreement | vs consensus | Discrepancies by cause |
| One row per variable in Table S2; completed on study execution. |
Table S3. Reporting template for duplicate extraction and adjudicated re-review. One row per variable.
8. Auditability and traceability. Every extracted value will be linked to its source document identifier, enabling downstream auditing. The full extraction pipeline - prompt templates, model version, post-processing rules, and validation results - will be documented and released as supplementary material to support reproducibility.
9. Handling of extraction failures. Patients for whom extraction fails after all retries (e.g., persistent API errors, unparseable responses) will be coded as missing across all LLM-derived variables. These missing values are handled as described in § 5.4.
Image-Derived Variable Measurement
Prostate volume and the periprostatic adipose tissue (PPAT) measurements in Table 2 are computed automatically from the T2W series. Like the variables extracted from free text (§ 5.5), they enter the image-context model as scalar clinical predictors. They differ from the imaging predictors of § 5.2, which are image channels passed to the image encoder. All measurements are computed on the resampled T2W volume (§ 5.2, step 3) before cropping, which covers a mm field of view centred on the prostate.
Anatomical segmentation. The prostate and surrounding structures (bladder, rectum, bowel, pelvic bones, iliac vessels, and pelvic muscles) are segmented with the pretrained TotalSegmentator MRI models ((Wasserthal et al., 2023; D’Antonoli et al., 2025)), without retraining or fine-tuning on UCLH data.
Adipose tissue classification. No pretrained model for segmenting PPAT on T2W images was available, so adipose tissue, the high-signal class on non-fat-suppressed T2W images, is classified by signal intensity: after N4 bias-field correction ((Tustison et al., 2010)), a threshold is set for each case by Otsu’s method ((Otsu, 1979)), and segmented non-adipose structures are removed. Because the threshold is set on an arbitrary intensity scale, it does not transfer to other acquisition protocols. Periprostatic adipose tissue. PPAT is included as an additional, exploratory predictor. It has been hypothesised that immune-active cells within PPAT regulate disease development and progression (Gucalp et al., 2017; Finley et al., 2009), and periprostatic fat thickness on MRI has been associated with high-grade prostate cancer (Cao et al., 2017). PPAT is defined as adipose tissue within 15 mm of the prostate surface over the axial extent of the gland; bands of 10 mm and 20 mm are used in sensitivity analysis (§ 8.9).
Measurements. Six measurements are derived from these automated segmentations (Table 2, group III); PSA density is then calculated from prostate volume.
Prostate volume: the volume of the prostate mask, except in truncated cases (see below).
PPAT volume: the volume of PPAT at 15 mm.
PPAT / prostate volume ratio: PPAT volume divided by prostate volume.
PPAT fat fraction: the share of the 15 mm periprostatic band, after removal of other segmented structures, that is classified as adipose. It is a proportion of voxels classified by signal intensity, not a physical fat fraction measured with a fat-specific sequence.
Retropubic gap: on the mid-gland axial slice, within a strip 10 mm either side of the gland midline, the anteroposterior distance from the anterior surface of the prostate to the posterior surface of the pubic bone. Where no pubic bone is segmented anterior to the gland on that slice, up to two adjacent slices on each side are used instead.
PPAT thickness: the adipose depth within the retropubic gap, equal to the gap multiplied by the proportion of the strip classified as adipose.
Quality control. The image-derived measurements of a case are set to missing if the prostate mask is empty, is fragmented into disconnected components, or has a volume outside 10–250 mL; if PPAT volume is below 0.5 mL; or if the PPAT fat fraction lies outside 0.02–0.95. The retropubic gap and PPAT thickness are also set to missing where no pubic bone is segmented anterior to the gland. Overlay images of the masks at apex, mid-gland, and base are generated for every case for visual review. The case itself remains in the cohort, and the number of cases failing each check is reported.
Gland truncation. The 60 mm craniocaudal extent of the field of view does not cover the whole gland in larger prostates, so the segmented volume underestimates the true volume. Cases in which the prostate mask reaches the superior or inferior boundary of the field of view are flagged, and their number is reported. For these cases, prostate volume, and therefore PSA density, is taken from the volume stated in the radiology report; where the report states no volume, the segmented volume is retained. PPAT volume and the measurements derived from it remain underestimated in flagged cases, and their effect is examined in a sensitivity analysis (§ 8.9).
Validation. Prostate volume from segmentation is compared with the volume stated in the radiology report in all cases where a volume is stated, using the intraclass correlation coefficient and Bland–Altman analysis. No reference PPAT measurement exists in routine reports, and manual annotation is labour-intensive, so PPAT segmentation is compared with manual annotation by a trained reader in a random sample of 50 cases, stratified by scanner and gland size, reporting the Dice coefficient and relative volume error.
Data Linkage and Storage
All data sources are linked at the patient level using pseudonymised identifiers generated during the OMOP extraction process. An auditable list of all data items will be maintained, with statistical disclosure control applied to prevent re-identification through unique attribute constellations. Data are stored in the UCL Data Safe Haven or UCLH Azure landing zones with role-based access control (RBAC). GPU computing resources are provisioned within these secure environments for model training.
Sample Size
The study is expected to include approximately 7,171 eligible MRI cases from 5,208 patients with linked clinical data, of whom a subset will have corresponding preprocessed MRI images available for multimodal analysis.
| Characteristic | Value |
| Total eligible MRI cases | 7,171 |
| Total patients | 5,208 |
| csPCa positive (ISUP ) | 2,448 (34.1%) |
| csPCa negative | 4,723 (65.9%) |
| Median age at MRI (IQR) | 66 (60–73) years |
Table 3. Summary of cohort characteristics. Counts and the age distribution are given per MRI case unless stated otherwise.
Sample Size Justification
Formal sample size calculations for prediction models follow the method of Riley et al. (Riley et al., [5]). With an anticipated outcome prevalence of 34%, candidate predictor parameters (imaging features + clinical variables), a target Cox-Snell of 0.15, and a maximum allowable shrinkage factor of 0.9, the minimum sample size required is approximately 1,200 patients. Our available cohort of patients substantially exceeds this minimum, providing adequate statistical power for both primary analysis and subgroup evaluations.
Statistical Analysis Plan
Descriptive Statistics
Baseline characteristics will be summarised by csPCa status, using means (SD) or medians (IQR) for continuous variables and frequencies (%) for categorical variables. Missingness will be reported per variable and separately for the adaptation, validation, and test sets, since missingness rates range from 8% to 78% across the clinical predictors (Table 8).
| Bias Type | Risk | Expected Impact | Mitigation and Remaining Issues |
| Information bias | Variable MRI quality across scanners, protocols, and time; inter-reader variability in the radiologist score, and possible variation in the scale used | Low | Standardised preprocessing; as a pioneering protocol, remaining scanner-, protocol-, and reader-related effects will be further assessed through external validation |
| Measurement bias (image-derived features) | Adipose tissue classified by signal intensity rather than by a fat-specific sequence; seminal vesicle fluid may be counted as fat near the gland base; volumes underestimated where the gland exceeds the field of view | Moderate | Quality control and segmentation validation (§ 5.6); sensitivity analyses on band width and gland truncation (§ 8.9); thresholds are cohort-specific and not transferable to other protocols |
| Confounding | Unmeasured confounders (e.g., prior negative biopsy) | Low | Adjustment using available covariates and sensitivity analyses; residual confounding remains an open research question |
| Missing data bias | Differential missingness (8–78% across variables); LLM-extracted variables may have systematic extraction failures | High | Missing-data handling is variable-dependent and follows the procedures described in § 5.4 and 5.5, including missingness-aware modelling, sensitivity analyses, and validation of LLM-derived variables against manual review |
| Algorithmic bias | Performance disparity across demographic subgroups | Low | Comparative evaluation across combining strategies (§ 8.1.2), together with subgroup analyses (§ 8.8), will identify potential algorithm-dependent disparities and inform future mitigation and external validation |
Table 8. Additional sources of bias and mitigation strategies.
Image-only Model and Data Splitting
The image-only model is PI-CAI-2B, an AI system for the detection of csPCa (ISUP grade group ) on bpMRI, developed and validated in the SCARLET-1 study ((Saha et al., 2025, 2026)). It is a further development of the top-performing algorithm of the PI-CAI study ((Saha et al., 2024)): a three-dimensional nnU-Net (v1) model ((Isensee et al., 2021)) with a MedNeXt-3XL encoder–decoder of approximately parameters, trained as five cross-validation folds. whose outputs are averaged at inference. It takes T2W, ADC, and high -value DWI as input and returns a voxel-level csPCa likelihood map.
The released system can combine its image score with clinical information. This study uses fold 0 of the released weights in the image-only configuration, in which no clinical variable enters the model. The case-level score is derived from the likelihood map alone: voxel likelihoods below 0.1 are set to zero, each connected lesion candidate is assigned its mean likelihood, and the case score is the highest lesion likelihood. Because the image-only model uses no clinical information, it is the reference against which the contribution of clinical variables is measured.
Development and test cohorts of PI-CAI-2B. As stated in the SCARLET-1 study protocol (Saha et al., 2025), PI-CAI-2B was trained on 18,371 MRI examinations (17,178 patients; 26 cities in 14 countries) drawn from two cohorts:
Primary training and internal testing cohort: 10,207 examinations (9,129 patients) acquired from 2012 to 2021 at four tertiary centres in the Netherlands and Norway, originally collected for the PI-CAI study (Saha et al., 2024). The PI-CAI training set of 9,107 examinations is used for training; its 1,000-examination test set serves as an internal test cohort.
Secondary training and internal testing cohort: 10,264 examinations (10,149 patients) acquired from 2012 to 2024 in 22 cities in 13 countries in Europe, Asia, North America, and Africa, collected through the EU Horizon projects ProCAncer-I and COMFORT and eight independent centres. Of these, 9,264 examinations are used for training and 1,000 are held out as a second internal test cohort.
The SCARLET-1 study tested PI-CAI-2B externally on the PRIME (Ng et al., 2025), STHLM3-MRI (Eklund et al., 2021), and IP1-PROSTAGRAM (Eldred-Evans et al., 2021) cohorts. The figures above are those of the study protocol; the published report gives the final composition of 22,445 examinations from 21,252 men (Saha et al., 2026). The internal test cohorts of the SCARLET-1 study are distinct from the UCLH internal test set of the present study.
The weights of PI-CAI-2B are fixed throughout this study. UCLH, ReIMAGINE, and BiopsyUS are not named among its development cohorts, and PRIME, STHLM3-MRI, and IP1-PROSTAGRAM were used in its development study for external testing only. Case-level lists of the development cohorts are not available, so patient-level overlap with the cohorts of this study cannot be excluded; this is reported as a limitation.
The UCLH cohort is then split into an adaptation set (10%), a validation set (10%), and a held-out internal test set (80%), stratified by csPCa status and grouped by patient, so that all cases from one patient fall in the same split. Grouping is necessary because men on active surveillance contribute serial studies that are highly correlated; splitting at case level would place near-duplicate images on both sides of the partition. The split is performed once using a fixed random seed. The adaptation set trains the clinical modulation modules, with all PI-CAI-2B weights frozen; the validation set is used for model selection, for Platt recalibration, and to fix decision thresholds; and the test set is not used at any point during development.
Model Architecture
For consistency, model terminology used throughout the study is defined in Table 4.
| Term | Definition |
| Image-only model | Model using bpMRI predictors alone (T2W, high -value DWI, and ADC): PI-CAI-2B in its image-only configuration, with fixed weights (§ 7.2). |
| Context-only model | Model using clinical predictors without the image encoder. Group III predictors, although computed from the MRI, enter as scalar features and are included. If not specified, this refers to the full clinical predictor set. |
| Image-context model | Any model combining bpMRI with one or more clinical predictors. |
| Full clinical predictor set | All clinical predictors included in Table 2. |
| Full image-context model | Image-context model combining bpMRI with the full clinical predictor set. |
Table 4. Model terminology.
Image Encoder
The image encoder is the MedNeXt-3XL network of PI-CAI-2B, fold 0 (§ 7.2), which encodes the three-channel MRI input (T2W, high -value DWI, and ADC) prepared as in § 5.2. All of its weights are frozen and are not updated with UCLH data. For image-context models, only the clinical modulation modules of § 8.1.1 are trained.
Clinical Feature Encoder
Clinical context enters the frozen image model by feature-wise linear modulation (FiLM) ((Perez et al., 2018)), applied per variable. Each clinical variable has its own small MLP, which maps its value to a channel-wise scale and shift . A signal of the image model is modulated as
where indicates whether variable is recorded (§ 5.4). The output layer of every MLP is initialised to zero, so modulation starts from the image-only model.
Combining Strategies
Three image-context combining strategies will be compared. They use the same per-variable modulation and differ only in the signal that it modulates (Table 5). Beyond its technical interest, the comparison may indicate how clinical context interacts with imaging findings, and so generate hypotheses for further study.
| Strategy | Modulated signal |
| Early combining | Output of the first encoder block: image feature maps at full spatial resolution. |
| Mid combining | Output of the encoder bottleneck: the most compressed image representation. |
| Late combining | Case-level score of the image model, on the logit scale. |
Table 5. Image-context combining strategies.
These strategies represent different ways in which clinical information may contribute to diagnostic reasoning. Early combining lets clinical context change how imaging features are formed. Mid combining lets it adjust the summary of the image. Late combining changes only the final risk estimate, as clinical risk calculators or nomogram-based decision support adjust risk at the decision level.
Training and Calibration
Only the modulation modules are trained, on the adaptation set, with a binary cross-entropy loss. For early and mid combining, the case score of the full inference pipeline cannot be differentiated, so the loss is computed on a proxy case score taken from the top-64 lesion logits. Model selection uses the validation set. Every model, including the image-only model, is then recalibrated by Platt scaling on the validation set, and this calibration is fixed before the test set is evaluated.
Performance Metrics
The primary objective is a comparison, so the primary estimand is the difference between the full image-context model and the image-only model on the held-out UCLH test set, reported with a 95% confidence interval rather than as two separate point estimates. Because both models are applied to the same cases, all comparisons are paired: differences in AUC-ROC are tested using DeLong’s method (DeLong et al., 1988), and differences in every other metric are obtained by paired bootstrap resampling of cases with 2,000 replicates. The context-only model is reported alongside as a reference for the performance attainable without imaging, and is not the subject of a formal comparison.
A model may raise discrimination while worsening calibration, or improve average discrimination without improving performance at the operating point where a biopsy decision is actually made, and these are different failures with different clinical consequences. The metrics in Table 6 are therefore prespecified as a panel spanning four distinct questions - overall discrimination, performance at a clinically relevant operating point, calibration, and net clinical benefit - and a claim that clinical context improves detection requires improvement in discrimination without degradation in calibration.
| Metric | Definition | Question it answers |
| AUC-ROC | Area under the receiver operating characteristic curve | Overall discrimination, independent of threshold |
| AUPRC | Area under the precision-recall curve | Discrimination in the positive class, which ROC can flatter |
| Sensitivity, specificity | At the prespecified operating points (95% and 90% sensitivity) | Performance where the biopsy decision is made |
| PPV, NPV | Predictive values at the same operating points | How a prediction should change management |
| Calibration | Calibration plot, calibration slope and intercept, Brier score | Whether predicted risks can be believed as probabilities |
| Net benefit | Decision curve analysis across clinically plausible threshold probabilities | Whether adding clinical context is worth acting on |
Table 6. Prespecified panel of performance metrics. All are reported for every model.
Every metric in the panel is reported for every model, in both directions, irrespective of whether the result favours the multimodal model. No metric is selected, added, or dropped after the test set has been examined. This constraint is what allows a panel of metrics to be more informative than a single one.
Threshold-dependent metrics require a prespecified operating point. Sensitivity, specificity, PPV, and NPV are reported at the threshold giving 95% sensitivity in the validation set, reflecting the clinical priority of not missing csPCa, and additionally at the threshold giving 90% sensitivity. Thresholds are fixed on the validation set and applied unchanged to the test set.
Only the primary comparison is confirmatory. All secondary objectives, including the ablation analyses, are estimation exercises reported with confidence intervals, both without and with adjustment for multiple comparisons; given the number of comparisons involved, individual ablation results are interpreted as ranking evidence.
Image-only Model Evaluation
The image-only model is evaluated with frozen weights, preprocessing, and inference settings throughout; no evaluation cohort, internal or external, is used to update the model or its decision thresholds. All evaluations use the case-level score of the model after Platt recalibration on the validation set (§ 8.1.3), together with the metrics in Table 6.
External Testing
The image-only model and the image-context models will be evaluated on five external cohorts (Table 7). PRIME, RelMAGINE, and BiopsyUS come from diagnostic settings; STHLM3-MRI and IP1-PROSTAGRAM come from population-based screening. All weights and decision thresholds are fixed after development on UCLH data, and no external cohort is used to update them.
| Cohort | Setting and population | Clinical predictors | Outcome |
| PRIME ((Ng et al., 2025)) | Primary diagnosis. International multicentre randomised trial comparing biparametric with multiparametric MRI. External test cohort of the SCARLET-1 study. | Age, PSA, prostate volume, PSA density, ethnicity, DRE, family history of prostate cancer, comorbidities, obesity | Biopsy grade group; unbiopsied men are labelled only as the trial reference standard allows |
| ReIMAGINE ((Marsden et al., 2022)) | Primary diagnosis. UK multicentre prospective cohort of men referred for MRI with suspected prostate cancer and a suspicious MRI (score ). | PSA, prostate volume from the MRI report; PSA density derived | Biopsy Gleason grade; the overall or maximal grade rule is fixed before analysis |
| BiopsyUS ((Urological Research Network, 2023; Min et al., 2025)) | Diagnosis. US registry of men undergoing MRI/ultrasound-guided biopsy (NCT02381990); 1,898 patients, 1,976 studies. | Age, derived from dates; prostate volume from segmentation | Core-level ISUP grade group; case label from the highest grade |
| STHLM3-MRI ((Eklund et al., 2021)) | Population-based screening, Stockholm, Sweden. External test cohort of the SCARLET-1 study. | Age, PSA, prostate volume | Grade group by the trial reference standard |
| IP1-PROSTAGRAM ((Eldred-Evans et al., 2021)) | Population-based screening, London, UK. External test cohort of the SCARLET-1 study. | Age, PSA, prostate volume, ethnicity | Grade group by the trial reference standard; few positive cases limit precision |
Table 7. External testing cohorts. Clinical predictors are those recorded in each cohort, before mapping to the definitions of Table 2.
The cohorts differ in which clinical predictors were recorded. A predictor that a cohort did not record is treated as missing and contributes nothing to the image-context models (§ 5.4). Each cohort therefore tests the models under its own pattern of available clinical information. Only predictors whose definition and timing match those of Table 2 are mapped; other fields are treated as missing.
Eligible cases have T2W, ADC, and high -value DWI and an outcome that can be mapped to ISUP grade group . Performance is reported separately for each cohort, together with the number of eligible cases, the number of csPCa-positive cases, and the completeness of each clinical predictor. The two screening cohorts are included subject to confirmation of local data access; if either is unavailable, this is reported as a deviation from the protocol.
Ablation Studies
To quantify the contribution of individual clinical predictors to multimodal prediction, add-one-in (AOI) and leave-one-out (LOO) analyses will be performed separately for each combining strategy:
Add-one-in: The image-only model serves as the reference. Each clinical predictor is added individually to bpMRI using the specified combining strategy. Its effect is assessed by the change in model performance relative to the image-only model.
Leave-one-out: The full image-context model serves as the reference. Each clinical predictor is removed individually while all remaining clinical predictors are retained. Its effect is assessed by the change in model performance relative to the full image-context model.
Each ablation experiment constitutes an independent full training run to avoid masking effects.
Determinants of Variable Contribution
The incremental value of each clinical predictor, estimated by the add-one-in analysis, will be related to three characteristics of that predictor: its source (structured extract, derivation,
automated image analysis, or extraction from free text), its missingness rate in the adaptation set, and, for free-text-derived variables, its validated extraction reliability (Table S3). The association will be summarised descriptively, principally as incremental performance plotted against missingness rate with points coloured by source; given the small number of predictors, it will not be formally tested.
This analysis is exploratory. It asks whether variables that are harder to obtain reliably also contribute less, which bears on which clinical information is worth the cost of capturing in a future prospective system.
Interpretability Analysis
SHAP (SHapley Additive exPlanations) (Lundberg & Lee, 2017) values will be computed on the test set to quantify the contribution of each clinical feature to individual predictions. Summary plots (beeswarm and bar plots) will be presented for each combining strategy.
Subgroup Analyses
Subgroup analyses ask whether the contribution of clinical context differs between groups of patients, scans, or lesions. The subgroups and their cut-off points are fixed before the test set is analysed. Clinical thresholds are used wherever an established one exists; where none exists, the cut-off points are the tertiles of the adaptation set.
Patient characteristics:
Age: , 60–69, years.
Ethnicity: White, Black, Asian, Other/Not stated. The last subgroup combines the Mixed, Other, and Unknown categories of Table 2.
BMI: , 25–29.9, kg/m.
Smoking status: never, former, current.
Diabetes: present, absent.
Inherited risk: family history of prostate cancer (present, absent); family history of breast or ovarian cancer (present, absent).
PSA and examination:
PSA level: , 4–10, 10–20, ng/mL.
PSA density: , 0.10–0.19, ng/mL/mL.
DRE finding: normal, abnormal.
Treatments that alter PSA or the gland: 5-ARI use at MRI (yes, no); prior BPH intervention (yes, no).
Radiologist-derived context:
Radiologist score: 1–2, 3, 4, 5.
Number of lesions with radiologist score : 0, 1, .
Index lesion zone: peripheral zone, transition zone, other or multiple zones. Restricted to cases with a reported lesion.
Image-derived measurements:
Prostate volume: , 40–79, mL.
PPAT measurements, each divided at tertiles: PPAT volume, PPAT / prostate volume ratio, PPAT thickness, PPAT fat fraction, and retropubic gap.
Image-only model score: low-score (below the median score of the image-only model in the adaptation set) and high-score (at or above that median).
Acquisition: field strength (1.5T, 3T); high -value series acquired or computed (§ 5.2).
For a variable that can be missing, cases in which it was not recorded form a separate not recorded subgroup. This subgroup is reported because missingness itself may carry information. Subgroups are defined by the recorded values of the variables, whether or not those variables are inputs to the model under evaluation.
Subgrouping by image-only model score uses the output of one of the models being compared. Selecting on this score restricts its range within each subgroup, so the discrimination of the image-only model within a subgroup is expected to be lower than that in the whole cohort, and the difference between models is interpreted accordingly.
Within each subgroup, every model is evaluated with the metrics of Table 6. The difference between the full image-context model and the image-only model is reported with a 95% confidence interval, estimated as in the primary comparison. Whether this difference varies across the levels of a subgrouping variable is assessed by an interaction estimate obtained by paired bootstrap resampling.
A subgroup with fewer than 30 csPCa-positive or 30 csPCa-negative cases is merged with the adjacent level, or is reported descriptively where no adjacent level exists. All subgroup analyses are exploratory. They are reported in full, without adjustment for multiple comparisons, and are not used to support claims about the primary objective.
Sensitivity Analyses
Verification bias correction: Inverse probability weighting (IPW) to account for differential biopsy rates between radiologist score categories.
Complete case analysis: Repeating primary analyses restricted to patients with missing clinical variables.
Alternative outcome definitions: Using any-grade prostate cancer (ISUP ) as an alternative outcome.
Pathology linkage interval: Repeating the labelling procedure with intervals of 6 and 12 months in place of 3 months, to quantify the effect of the linkage window on the proportion of cases labelled by histopathology and on model performance. Relabelling does not require the image encoder to be retrained.
PPAT band width: Repeating the analyses with PPAT defined at 10 mm and 20 mm from the prostate surface in place of 15 mm.
Gland truncation: Repeating the analyses after excluding cases in which the prostate mask reaches the boundary of the field of view (§ 5.6).
Software
All analyses will be performed in Python 3.10+ using PyTorch (deep learning), scikit-learn (classical ML comparisons), and SHAP (interpretability). Statistical comparisons between AUCs will use DeLong’s test (DeLong et al., 1988). Code and trained model weights will be made available upon publication.
Bias Assessment
This study is subject to several domain-specific biases that are intrinsic to the prostate cancer diagnostic pathway and the retrospective study design. These are discussed below alongside general methodological biases.
PSA Referral Bias
The cohort is conditioned on referral for prostate MRI, which in clinical practice is triggered primarily by elevated or rising PSA. This creates a PSA referral bias: patients entering the cohort have, by definition, already been flagged by PSA testing, and the distribution of PSA values in this referred population does not reflect the general male population. Consequently, the predictive value of PSA within this cohort may be attenuated compared with a screening population (“PSA has already done its job” by the time MRI is performed). Models trained on this cohort may overestimate the marginal contribution of imaging relative to PSA, and performance metrics may not generalise to settings where PSA-based pre-selection has not occurred (e.g., opportunistic MRI screening).
Patient Selection Bias
UCLH is a high-volume tertiary referral centre with particular expertise in prostate MRI. The patient population is therefore enriched for complex cases, higher radiologist scores, and a higher baseline prevalence of csPCa (34%) compared with community or screening settings. Additionally, the ethnic and socioeconomic composition of the UCLH catchment area (central London) may not be representative of the wider UK or international populations. These factors limit the direct generalisability of model performance estimates. External validation on datasets from community hospitals and diverse populations will be essential before clinical deployment.
Classification Task Bias
The outcome definition (csPCa as ISUP on biopsy) introduces a classification task bias with several components:
Verification bias: Not all patients undergo biopsy; those with a radiologist score of 1–2 are less likely to be biopsied, leading to under-ascertainment of csPCa in low-risk groups. The magnitude of this bias will be assessed using inverse probability weighting (IPW), as specified in § 8.9, with weighted and unweighted model performance compared.
Biopsy sampling error: Targeted and systematic biopsies may miss significant cancer, with studies reporting approximately 5-16% of csPCa being missed when systematic biopsy is omitted and around 9% misclassification of clinically significant disease with MRI-targeted biopsy alone (Ahdhoot et al. (2020); Lee et al. (2022)).
Gleason grading variability: Inter-observer agreement for Gleason grading, particularly at the 3+3 vs. 3+4 boundary, is imperfect. The binary threshold (ISUP ) is clinically standard but introduces classification noise at the decision boundary. A UK reproducibility study reported 78% agreement with the consensus Gleason score and an overall inter-observer of 0.54, with lower agreement for Gleason score 7 () (Melia et al. (2006)).
Temporal label evolution: Imaging interpretation, biopsy practice, and pathology reporting may evolve over the study period, resulting in temporal heterogeneity in outcome ascertainment.
Other Methodological Biases
Several methodological biases may influence the estimated contribution of clinical context and the interpretation of model performance. Their anticipated impact on this study and corresponding mitigation strategies are summarised below.
Ethical Considerations
Ethical Approval
This study has been approved under the UCLH Data Access Process for Research (DAP-R), which provides a generic favourable ethical opinion (IRAS No. 299136). Individual patient consent is not required as this is a retrospective analysis of routinely collected, anonymised data. The study sponsor is UCLH.
Data Protection and Privacy
All data will be pseudonymised at extraction via the OMOP ES pipeline and stored in the UCL Data Safe Haven or UCLH Azure landing zones with RBAC.
Patients registered on the National Data Opt-Out will be automatically excluded.
DICOM tags will be anonymised prior to image conversion and analysis.
Statistical disclosure control techniques will be applied to prevent re-identification through unique attribute constellations.
The study complies with the UK General Data Protection Regulation (UK GDPR) and the Data Protection Act 2018. The lawful basis for processing is Article 6(1)(e) (public task) and Article 9(2)(j) (scientific research).
Supplementary Material
Radiology Report Keyword Rules
The keyword rules in Table S1 apply only at step 4 of the outcome labelling procedure (§ 5), that is, only to cases for which no histopathology, no recorded lesion count, and no assessment score is available. They are a last-resort rule set applied to a minority of cases, and they are deliberately conservative: a case is labelled csPCa-negative only when the report contains an explicit statement of absence, and any expression of diagnostic uncertainty prevents the label from being assigned.
Rules are applied to the Conclusion or Impression section of the report where one can be identified, and to the full report text otherwise, since the concluding statement carries the reporting radiologist’s overall assessment. Matching is case-insensitive and follows whitespace and punctuation normalisation.
Exclusion patterns (E1–E8) are evaluated before inclusion patterns (N1–N9). A case matching any exclusion pattern is not labelled at this step and is excluded from the cohort, because step 4 is the final opportunity for ascertainment. Negation is resolved using a scope-limited negation detection method of the NegEx family rather than by literal string matching, so that a negation term applies only to findings within the same clause; this prevents a phrase such as “no extracapsular extension” from negating a suspicious lesion described in an adjacent sentence.
The rule set will be frozen, with a version identifier, before it is applied to the study cohort. Any subsequent amendment will be recorded with its date, rationale, and the number of cases whose label changes as a result.
| ID | Pattern (case-insensitive) | Rationale |
| N9 | no MRI evidence of (significant)?disease | Explicit absence of disease on MRI |
Table 10.
LLM Extraction Schema and Prompt
Table S2 gives the extraction schema submitted to the model, one row per target variable. The schema is enforced twice: it is stated in the prompt, and parsed outputs are validated against it before any value enters the analysis dataset (§ 5.5). The system prompt reproduced below is applied unchanged to every patient; the per-variable instructions in Table S2 are appended to it as the JSON schema description.
You extract structured clinical data from prostate cancer records.
Rules:
1. Return a single JSON object matching the schema. No prose, no markdown.
2. Return null for any field not stated in the source text. Never infer, estimate, or carry a value forward from another field.
3. Return false only where the text explicitly records absence. Absence of mention is null, not false.
4. Where a value appears more than once with different values, apply the aggregation rule given in the field description (for example, maximum for radiologist scores, highest for Gleason score).
5. Do not infer a diagnosis from medication alone.
6. Quote the source verbatim in the accompanying _evidence field for every non-null value. If you cannot quote it, return null.
7. Report dates in ISO 8601 (YYYY-MM-DD). Return null for partial dates.
Rule 6 requires the model to return a supporting quotation alongside each extracted value. Values whose quotation cannot be located in the source document by exact string match are set to null during post-processing. This converts an unverifiable assertion into a detectable failure and provides the source linkage described in § 5.5.
| Variable | Permitted values | Extraction instruction |
| From radiology reports | ||
| Radiologist score | Integer 1–5 | Return the highest level-of-suspicion score stated for any lesion, whether reported as a PI-RADS category or as a Likert score. Where several lesions are scored, return the maximum. Record which scale was used in the paired score_scale field. Do not infer a score from a narrative description of suspicion. |
Table S2. Per-variable extraction schema, covering only those variables sourced from free text (Table 2, source LLM) together with the report fields used for outcome labelling. Every field additionally accepts null, and every field has a paired _evidence field holding the supporting quotation.
| Variable | Permitted values | Extraction instruction |
| Lesion count | Integer | Count lesions explicitly enumerated or individually described. Where the report states that no lesion is present, return 0. Where lesions are referred to collectively without enumeration, return null. |
| From histopathology reports | ||
| Gleason score | String, a+b | Return the highest Gleason score across all cores in the specimen. Where primary and secondary patterns are reported separately, combine them in the stated order. |
| ISUP grade group | Integer 1–5 | Return the grade group as stated. Where only a Gleason score is given, return null; the mapping is applied during post-processing rather than by the model. |
| From clinical notes | ||
| Family history of prostate cancer | Boolean | Return true only where a blood relative with prostate cancer is recorded. Return false only where family history is explicitly documented as negative. Relatives by marriage do not count. |
| Family history of any cancer | Boolean | As above, for any malignancy in a blood relative. Where true, list the cancer types in the paired family_cancer_types array. |
| History of other cancers | Boolean | Return true where the patient has a personal history of a malignancy other than prostate cancer. Benign neoplasms and in-situ lesions do not count. Where true, list types in other_cancer_types. |
| Diabetes | Boolean | Return true where recorded as a current or past diagnosis, including type 1 and type 2. Do not infer from glucose-lowering medication alone. |
| Hypertension | Boolean | Return true where recorded as a diagnosis. Do not infer from antihypertensive medication or from isolated blood pressure readings. |
| Cardiovascular disease | Boolean | Return true for ischaemic heart disease, myocardial infarction, heart failure, atrial fibrillation, stroke, or peripheral arterial disease. Hypertension alone does not count. |
| COPD | Boolean | Return true where recorded as a diagnosis. Emphysema and chronic bronchitis count; asthma does not. |
| Asthma | Boolean | Return true where recorded as a diagnosis. |
| Chronic kidney disease | Boolean | Return true where recorded as a diagnosis or where a CKD stage is stated. Do not infer from a single creatinine or eGFR value. |
| Smoking status | Integer 0–2 | 0 = never smoker; 1 = former smoker; 2 = current smoker. Where the text records only “smoker” without qualification, return 2. The integer code is one-hot encoded during post-processing. |
Table 1 (PDF p. 34).
| Variable | Permitted values | Extraction instruction |
| Alcohol status | Integer 0–6 | Ordinal band of reported weekly consumption in UK units: 0 = none; 1 = 1–7; 2 = 8–14; 3 = 15–21; 4 = 22–35; 5 = 36–50; 6 = >50. Where consumption is described qualitatively without a quantity, return null. The integer code is one-hot encoded during post-processing. |
Table S2 (PDF p. 35).
Duplicate Extraction and Adjudication
Table S3 is the reporting template for the two checking procedures described in § 5.5: duplicate extraction, which quantifies the stability of the extraction process, and adjudicated re-review, which tests the reference standard against which the process is judged. Both are reported per variable, since reliability differs markedly between variables that are stated explicitly in structured-like text and those that must be inferred from narrative.
Discrepancies are classified as source ambiguity, annotator error, or extraction error. The distinction matters because only extraction error is attributable to the pipeline and correctable by prompt revision; source ambiguity sets a ceiling on the accuracy any extraction method could achieve for that variable, and is reported as such rather than counted against the model.
References
- [1]Sung H, Ferlay J, Siegel RL, et al. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J Clin. 2021;71(3):209-249.DOI
- [2]Ahmed HU, El-Shater Boısaly A, Brown LC, et al. Diagnostic accuracy of multi-parametric MRI and TRUS biopsy in prostate cancer (PROMIS): a paired validating confirmatory study. Lancet. 2017;389(10071):815-822.DOI
- [3]Kasiviswanathan V, Rannikko AS, Borghi M, et al. MRI-Targeted or Standard Biopsy for Prostate-Cancer Diagnosis. N Engl J Med. 2018;378(19):1767-1777.
- [5]Riley RD, Ensor J, Snell KIE, et al. Calculating the sample size required for developing a clinical prediction model. BMJ. 2020;368:m441.DOI
- [6]Collins GS, Moons KGM, Dhiman P, et al. TRIPOD+AI statement: updated reporting guidance for clinical prediction models that use regression or machine learning methods. BMJ. 2024;385:e078378.DOI