betaFeedback

WorksComputer Science

How Deep is the Signal? Time-Context up to 30 Years for Colorectal Cancer Prediction in a Primary-Care EHR Model

Abstract

EHR foundation models read medical events within a fixed context window, and long context has so far been measured only in tokens. How far back in time a model must look to capture the earliest predictive signal is unknown. We introduce time-context, the calendar lookback window before the prediction point, and provide, to our knowledge, the first systematic evaluation of decade-scale time-context for an EHR foundation model. We pretrained a decoder-only transformer (34.6M parameters) on 1.4B coded events from 3.6M primary-care patients in North West London and fine-tuned it to predict incident colorectal cancer under a 12-month pre-diagnostic buffer. A frozen cohort of 1,008 cases and 9,582 matched controls, all registered for at least 30 years, was evaluated at 15 nested lookback windows from 2 to 30 years, so that every truncation is compared within the same patients. Discrimination rose from an AUROC of 0.551 at 2 years to 0.590 at 18 years, then plateaued, with about 80% of the gain reached within 10 years; an XGBoost baseline converged to the same level (peak AUROC 0.589). A simulation further suggests that rotary position embeddings (RoPE) over elapsed time alias beyond about 20 years at the default base frequency, motivating a retuned θθ. Time-context is a measurable design axis for clinical prediction models, distinct from token-context, and should be reported.

Data and Code Availability The Discover dataset, a de-identified research mirror of the Whole Systems Integrated Care (WSIC) North West London database, is not publicly available; access is granted to approved researchers by the Discover-NOW Data Access Committee. Code may be available from the authors on reasonable request, subject to approval by the data controller and Imperial College London.

Institutional Review Board (IRB) The study falls under the Discover-NOW research database ethics and HRA approval (REC 23/WM/0196) and was authorised by the North West London Data Access Committee; all analyses ran in a secure environment without export of patient-level data.

Introduction

Lookback depth – how much of a patient’s past a prediction model is allowed to see – is a weakly scrutinised design choice ((Pungitore and Subbian, 2023)), often constrained by technical limits and by shallow hospital datasets. In longitudinally sparse EHRs, calendar history is a correlated but distinct axis from the number of tokens a model reads, and it remains largely unexplored. Wornow et al. (2025) varied the context window of EHR foundation models from 512 to 16,384 tokens and showed that larger token counts improve prediction. However, a patient with dense recent contact can exhaust a large context window within a few years, while a sparsely recorded patient’s whole life can fit in a few hundred tokens. The questions that matter operationally – record retention, data curation, screening strategies, guidelines for clinical prediction models – are all denominated in years. The median context window of EHR models has roughly quadrupled since 2019 ([4]), making long medical sequences tractable. Yet the scarcity of accessible primary-care databases, highlighted in a recent taxonomy of EHR event-stream modelling ((Zhu et al., 2026)), has kept researchers on secondary-care data and prevented the field from exploiting long-range signal. The same review identifies long context as an open challenge, and information sufficiency, i.e. how much history a reliable inference requires, as an unresolved question. Previous studies used fixed lookback windows of at most three years and did not encode event timing, using classical models [5] or note encoders (Mohammadi et al., 2026).

The field has meanwhile moved towards generative decoder-only architectures in addition to classical encoders, with models such as Foresight [7] and SurvivEHR [3] built on causal pretraining, an approach shown to work at health-system scale (Rajamohan et al., 2025). Delphi-2M (Shmatko et al., 2025) models lifetime disease trajectories but from sparser data, including registries, rather than granular clinical event streams. The closest prior work on our data source selected three-year pathways with a 1-month buffer to predict lung cancer, using an encoder-only model in the BERT lineage [2] (Wang et al. 2024). Following the current state of the art, we trained a LLaMA-like decoder-only model (Llama Team, 2024), deliberately small (34.6M parameters, against 1.6B for Rajamohan et al. (2025)) to fit our timeline, compute, and training sample.

CRC is a well-suited probe of how far back predictive signal carries for a specific disease in primary-care EHRs, as the adenoma–carcinoma sequence can run from about 10 to 26 years before diagnosis [9]. A curve still rising at deep windows would indicate long-dwell prodromal signal of direct interest for early detection, whereas an early plateau would give a citable bound for data curation.

Our contributions are: (1) time-context, a design axis distinct from token-context, evaluated for the first time at decade scale through a CRC predictive-signal depth curve; (2) a within-patient nested design that isolates calendar time on a fixed cohort at fifteen windows; (3) the use of primary-care data, known to be understudied in the field, with more than 30 years of history (1990–2026); and (4) simulation results suggesting that RoPE over elapsed weeks at the default θ=10,000\theta= 10{,}000 may alias attention similarity beyond ∼20\sim20-year gaps, and that a retuned θ\theta restores near-monotonic decay up to 35 years.

Data and Design

Data: Whole Systems Integrated Care (WSIC) North West London primary care database. A frozen extract (24 August 2026) was restricted to events from 1 January 1990, comprising ∼\sim6M patients and ∼\sim2B events coded in SNOMED CT (observations) and dm+d (prescriptions).

Incident CRC cases were defined by a first-ever code among 73 SNOMED CT concepts descending from Malignant neoplasm of large intestine (363510005), excluding secondary, metastatic, recurrent and non-carcinoma tumours, on or after 1 January 1990; this diagnosis date is the index date. Of 18,104 incident cases, 15,527 were registered at index, and 1,249 had at least 30 years of continuous registration before it; the deep cases (Figure 4). In each of the fine-tuning, validation and evaluation partitions, cases were matched to alive, CRC-free controls on sex, birth year (±2 years) and available continuous registration depth. Controls inherited their case’s index date, and no patient appears in two partitions. Patients with fewer than five events were excluded, and realised matching ratios are reported after this threshold. The evaluation cohort is a frozen ∼\sim80% hash split of the deep cases: 1,008 cases and 9,582 matched controls (1:9.51), excluded from all preprocessing statistics and from training. The remaining 241 deep cases and all non-deep cases entered fine-tuning with full-history inputs (13,490 cases, 37,321 controls, 1:2.77), except 10% of cases with 22–29 years of history, which formed the validation cohort (284 cases, 2,778 controls, 1:9.78).

Figure 4. Derivation of the pretraining, fine-tuning, validation and evaluation cohorts from the WSIC extract. Controls were matched to cases within each partition.

A uniform 12-month buffer before the index date was removed for every patient outside pretraining. It conservatively prevents leakage of the diagnostic work-up and isolates pre-clinical, long-range signal; its length exceeds published timings of the pre-diagnostic code surge for CRC, up to 8–10 months [8] (Moull et al. 2022). The end of the buffer is the index date and prediction point.

Evaluation patients were scored at fifteen nested time windows of 2, 4, …, 30 years (Figure 5), obtained by removing all events older than the window start, counted back from the index date. This design keeps everything constant except the depth of history: the same 10,590 patients, index dates, and the same model applied to every window. Validation patients were scored at six windows from 2 to 22 years. The primary metric is the AUROC; the AUPRC is secondary. Patients were resampled with replacement 1,000 times and each resample was scored at all fifteen windows; 95% CIs are the 2.5th–97.5th percentiles, and between-window differences (Δ\DeltaAUROC) are computed within each resample, so they are paired. The plateau depth is the shallowest window whose AUROC is within 0.001 of the maximum, whose paired Δ\DeltaAUROC against the 30-year window has a 95% CI containing 0, and whose point estimate of that difference lies within ±0.01\pm0.01.

Figure 5. Nested design with 15 lookback windows applied to the same 10,590 evaluation patients.

Baseline: An XGBoost classifier [1] on per-code event counts was trained using the same training data and nested evaluation as the transformer (Appendix § 10).

Model and Implementation

Model: Decoder-only transformer (causal masking, RMSNorm, SwiGLU), 34.6M parameters, d=512d = 512, 8 layers, 8 heads, context 8,192 tokens, full attention.

Each event is encoded as a triplet of feature, value and time, as reviewed by Zhu et al. (2026) and used in SurvivEHR (Gadd et al. [3]). Each of the three parallel channels is projected to 512 dimensions and summed element-wise (Figure 6). The feature embedding uses a code lookup table, values use a small trained MLP over the z-score and a presence flag (plus an acute/repeat qualifier embedding for prescriptions), and time is a sinusoidal encoding of age at event (sixteen bands with periods of ∼\sim1.5–630 years, Vaswani et al. (2017)). This tuple keeps sequences short enough for full attention and fitting within the 8,192-token context window: the median evaluation patient reaches ∼\sim1,200 tokens at 30 years and the 99th percentile among deep cases ∼\sim5,100.

Figure 6. Event representation and model. Each event of an illustrative (not real) history is embedded as the sum of code, value and age components; elapsed time between events drives the rotary position embedding. A [CLS] token closes the sequence, and its final hidden state is the only input to the CRC head.

We apply Rotary Position Embedding (RoPE) over elapsed weeks rather than token index (Su et al. (2024)), as in other time-aware EHR transformers (Zhang and Li, 2025}). Because attention between two events depends on their time interval, irregular sampling is handled natively, and same-day events are rotation-identical. No absolute calendar date is encoded, since era-shift recognition would contaminate the depth curve; the age channel also carries temporal signal. Under the default base θ=10,000\theta= 10{,}000, RoPE aliases at decade scale: attention similarity between a reference event and one 30 years earlier rebounds above the 20-year value. Sweeping θ∈[104,105]\theta\in[10^4, 10^5], we select θ=40,750\theta= 40{,}750, which maximises similarity spread across 3 months–35 years subject to near-monotonic decay (Appendix A).

A [CLS] token closes every sequence; it is embedded like other events, with its own code, no value, and age at the index date. During fine-tuning only, a dedicated learned vector is added to it. Under causal attention, its final hidden state summarises the whole history and ensures that every window contains at least one token.

After preprocessing, our vocabulary contained 18,733 tokens: 17,184 SNOMED CT concepts from four hierarchies (finding, procedure, observable entity, situation) plus 1,546 dm+d Virtual Therapeutic Moiety (VTM) codes from prescriptions, with [CLS]/[PAD]/[UNK] reserved. Numeric values are z-scored per code on pretraining-population statistics and winsorised at the 1st/99th percentiles; prescriptions are qualified as acute or repeat. See Appendix § 9 for preprocessing specifications.

Pretraining used a 70% sample of eligible patients (3.6M patients, 1.4B events), with a median sequence length of 150 tokens, and excluding all downstream cohorts and patients with fewer than five events or no birth date. The trunk was first trained on half of this sample for 40,000 steps, then continued on the remaining non-overlapping half for 6,500 steps. The pretraining objective is next-event prediction (causal masking): given a patient’s history to event tt, predict event t+1t+1. The task is fully self-supervised and disease-agnostic. See Appendix § 10 for training specifications.

We fully fine-tuned the model for CRC discrimination, training a linear head on the final-norm [CLS] representation jointly with all trunk parameters after one frozen-trunk warm-up epoch. Hyperparameters and checkpoints were selected by mean AUROC across six validation windows. The model was trained once on full-history inputs and applied unchanged to all fifteen evaluation windows, so that they differ only in their input.

Results

Transformer discrimination rose with time-context up to 18 years and plateaued thereafter, with 80% of the total gain (0.032 of 0.040 AUROC) realised within the first 10 years (Figure 1, Appendix § 11).

Figure 1. Evaluation AUROC (95% CI) by lookback window for the transformer and XGBoost.

AUROC was 0.551 (0.532–0.569) at 2 years, 0.583 (0.564–0.600) at 10 years, and reached a peak of 0.590 (0.572–0.608) at 18 years; all deeper windows stayed within ±0.01\pm0.01 of the 30-year value (0.580, 0.561–0.599). Under the above rule, the plateau depth was 18 years: Δ(30y−18y)=−0.010\Delta(30\mathrm{y}{-}18\mathrm{y}) = -0.010 (−0.022-0.022–0.002), consistent with no further gain. The shallowest window remains clearly below the plateau with Δ(18y−2y)=+0.040\Delta(18\mathrm{y}{-}2\mathrm{y}) = +0.040 (0.018–0.061). AUPRC tracked the same shape, from 0.113 at 2 years to 0.132 at 18 years (prevalence 0.095).

XGBoost on per-code counts traced a curve of similar shape with comparable performance: AUROC 0.528 (0.510–0.548) at 2 years, rising to a peak of 0.589 (0.569–0.607) at 14 years, and flat thereafter (30 years: 0.584, 0.566–0.602). The transformer was marginally ahead at shallow windows (2-year paired Δ\Delta = +0.022, 0.000–0.043), but the two model classes converged from about 12 years and were statistically indistinguishable at depth (30-year paired Δ\Delta = −0.004, −0.026–0.017). The transformer’s advantage is concentrated where records are sparse and disappears once enough events accumulate for code counts alone to carry the signal.

History beyond 18 years did not improve discrimination, consistent with a saturating signal rather than an effect of diminishing record density alone. Median tokens per evaluation patient grew from ∼\sim90 at 2 years to ∼\sim1,200 at 30 years, although the tokens added per 2-year step fell from ∼\sim180 to ∼\sim10 (Figure 2); between 18 and 24 years, each step still added several tens of tokens per patient without any gain. The plateau appears gradual rather than a sharp cut-off, and under data shift the curve is best read as a lower bound on the long-range signal available.

Figure 2. Cumulative and marginal token count per lookback window for evaluation patients.

Absolute discrimination was modest for both models, consistent with a conservative 12-month buffer, disease-agnostic preprocessing without cancer-specific feature selection, and noisy primary-care data. Above all, matching cases and controls on age and sex (along with registration depth) neu

tralises two of the strongest CRC predictors, so these figures are not comparable with clinical CRC models developed with cohort designs, such as QCancer (Hippisley-Cox and Coupland, [6]). Our 34.6M-parameter model is probably under-sized: a larger trunk, task-specific features and longer training may lift absolute discrimination; the depth shape, replicated by an independent model class, is however unlikely to change.

Discussion and Conclusion

Calendar-time lookback depth is a distinct axis from token-context length, and it can be measured directly. For incident colorectal cancer in primary care, most of the attainable discrimination is reached by ∼\sim18 years of lookback history, with no detectable gain beyond, a timescale compatible with the known adenoma-carcinoma dwell time. While specific to two model classes, a selected cohort and coded primary-care data under a 12-month buffer, these results suggest that a transformer can capture long-range pre-diagnostic signal that may reflect underlying disease biology, with early detection potential. The transformer and the count-based baseline converge in both performance and curve shape, although the transformer peaks later (18 vs 14 years), suggesting that the measured signal is largely a property of the data rather than of a specific architecture. Absolute discrimination is modest for both models, which may indicate an approximate signal ceiling for disease-agnostic primary-care codes at a buffered pre-diagnostic horizon in a matched design. The main limitations are: (1) this is a single-region study, albeit across diverse London practices and populations; (2) registration depth is counted from 1990 so every deep-cohort index date falls in 2020–2026, the evaluation cohort is therefore strongly selected (older, residentially stable, likely less deprived); (3) events recorded per year increased from ∼5M\sim5\mathrm{M} in the 1990s to ∼130M\sim130\mathrm{M} in the 2020s, and the number of distinct SNOMED CT codes per year rose ∼2.5\sim2.5-fold, making older history sparser and more coarsely coded; (4) some codes have approximate dates, and registration depth was computed at year resolution, potentially blurring the 30-year window; and (5) results come from a single training run, and architectural choices need ablations.

Future work includes expanding downstream tasks to other tumour site such as lung and pancreas to compare depth signatures, evaluating at natural prevalence, scaling model and data, refining preprocessing, adding an explainability analysis, and comparing architectures for long-range dependencies.

Acknowledgments

This work was supported by the NIHR Imperial Biomedical Research Centre (BRC) and by Discover-NOW, the health data research hub providing access to WSIC. A.R.’s visiting research position was funded by Assistance Publique – Hôpitaux de Paris (AP-HP). The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care.

RoPE base-frequency retuning for long-range gaps

Attention similarity between two events should decay monotonically as the temporal gap increases. To verify this, we drew 20,000 random vectors in simulation (head dimension 64), applied RoPE rotations using elapsed time in weeks for gaps ranging from 3 months to 35 years, and measured the mean rotated similarity as a function of elapsed time (see Figure 3). We represent time in weeks so that a 35-year gap spans ∼1,800\sim1{,}800 position units, comparable in magnitude to the token-index ranges RoPE was designed for; in days the same gap would be ∼12,800\sim12{,}800. Any rise in the curve indicates aliasing, i.e. distant events regaining spurious similarity once the faster rotations complete full turns. After sweeping θ∈[104,105]\theta\in[10^4, 10^5], we screened θ∈{10,000; 40,750; 75,000; 100,000}\theta\in\{10{,}000;\ 40{,}750;\ 75{,}000;\ 100{,}000\} and scored each candidate by the number of non-monotonic rebounds and by the dynamic range of the similarity curve. At the default θ=10,000\theta= 10{,}000, similarity rebounds beyond ∼20\sim20 years: events 25–35 years apart appear more alike than events 20 years apart; whereas θ=40,750\theta= 40{,}750 yields near-monotonic decay with the widest dynamic range and was retained for all experiments. While this method allows us to justify our parameter choice given our long context window, further ablations are needed to demonstrate its impact on performance.

Figure 3. Simulated mean RoPE attention similarity between random query/key vectors as a function of elapsed time (0.25–35 years), for four base frequencies θ\theta.

Study design and model figures

This appendix shows the cohort flowchart (Figure 4), the evaluation design (Figure 5), and the event representation and model (Figure 6).

Data preprocessing

Preprocessing ran in SQL on Snowflake within a secure NHS sandbox.

Case ascertainment and cohorts. The first CRC code was identified over the whole record before the 1990 restriction was applied; restricting first would have mistaken a follow-up code for the incident code in 191 prevalent cases. Under a ≥5\geq5 events threshold, ∼5%\sim5\% of the cases and ∼12%\sim12\% of the controls were excluded in fine-tuning and validation cohorts, versus 0% of cases and ∼6%\sim6\% of controls for evaluation. Fine-tuning used 1:3 matched controls as a compromise between class balance and sample size, the loss was reweighted accordingly. Predicted probabilities are therefore not calibrated, and all reported metrics are computed on the separately matched frozen evaluation cohort.

Vocabulary curation: Codes below a frequency floor are mapped to the nearest frequent SNOMED ancestor or dropped (800 patients for findings/procedures; 300 for observable/situations; 50 for dm+d VTMs), and ∼\sim2,000 administrative codes are removed on ontological grounds. The four retained hierarchies span 90% of coded nonprescription events, with 99.98% within-hierarchy coverage; 91.2% of prescriptions resolve to a dm+d VTM code. We harmonised dm+d drug concepts across the pre/post-2018 re-identification onto SNOMED product identifiers, removing coding artefacts (e.g., molecule codes splitting or fluctuating across the 2018 boundary) that would otherwise confound deep windows. After unit harmonisation, 96% of numeric values are retained; missing values use a learned embedding, and prescriptions carry an acute/repeat qualifier.

Within-day repetitions of the same code (order not recoverable) are collapsed to one event, with median retained if value-bearing, reducing the stream by 25% and avoiding a known source of inflated EHR metrics (Rajamohan et al., 2025; Wornow et al., 2025).

One evaluation patient exceeded the 8,192-token context and was left-truncated.

CohortCasesControlsRatioInput
Pretraining3.6M patients–Full
Fine-tuning13,49037,3211:2.77Full
Validation2842,7781:9.782–22y (6)
Evaluation1,0089,5821:9.512–30y (15)

Table 1. Cohorts used for pretraining, fine-tuning, model selection and evaluation, after the five-event threshold (number of windows).

Model training specifications

All training ran on a single NVIDIA T4 GPU (16 GB) in PyTorch, with fp16 and seed 43 throughout.

Pretraining. Next-event cross-entropy over the 18,733-token vocabulary (output weights tied to the input embedding), with Gaussian negative log-likelihood for values where present and regression on log time-to-next-event, the two auxiliary terms weighted 0.1 each. (1) 40,000 steps on the first half of the pretraining sample: sequences up to 8,192 tokens, batches formed to a fixed budget of 16,384 tokens with length bucketing, AdamW (lr 4.5×10−44.5 \times10^{-4},

weight decay 0.1, β=(0.9,0.95)\beta= (0.9, 0.95), cosine schedule with 2,000-step linear warmup, gradient clipping 1.0, fp16. (2) Continuation for 6,500 optimiser steps on the remaining non-overlapping half with 8-fold gradient accumulation (∼\sim131,000 tokens per step), rewarmed to lr 4.5×10−44.5 \times10^{-4} (400-step warmup, cosine decay), fp16. Final token-level cross-entropy 3.78 against ln⁡(18,733)=9.84\ln(18,733) = 9.84 at initialisation.

Fewer than 0.1% of pretraining patients had a CRC code; these entered no downstream cohort (e.g. prevalent before 1990 or not registered at diagnosis).

Fine-tuning. lr 2×10−52 \times10^{-5} (trunk) and 10−310^{-3} (head), weight decay 0.05, dropout 0.2, gradient clipping 1.0, one warm-up epoch with frozen trunk, loss weighted by the control:case ratio; best mean validation AUROC at epoch 7.

XGBoost baseline. Depth 6, learning rate 0.05, subsample 0.8, colsample-by-tree 0.5, minimum child weight 0.5, scale pos weight set to the realised control:case ratio; up to 1,000 trees with early stopping (50 rounds) on validation AUROC at the deepest validation window (22 years).

Discrimination by calendar-time lookback

Full AUROC and AUPRC results are given in Table 2 and Table 3.

Table 2: AUROC (95% CI) by calendar-time lookback depth for the transformer and the XGBoost baseline on the frozen evaluation cohort (n=10,590n = 10,590; 1,008 cases). Confidence intervals are 2.5th–97.5th percentiles of 1,000 patient-level bootstrap replicates. Peaks in bold; for XGBoost, 14 and 24 years differ by 0.0001 and are both shown.

Window (y)TransformerXGBoost
20.5505 (0.5320–0.5691)0.5281 (0.5101–0.5481)
40.5625 (0.5433–0.5812)0.5445 (0.5266–0.5650)
60.5730 (0.5530–0.5922)0.5553 (0.5359–0.5731)
80.5756 (0.5555–0.5950)0.5625 (0.5437–0.5809)
100.5825 (0.5637–0.6003)0.5718 (0.5533–0.5902)
120.5840 (0.5643–0.6020)0.5795 (0.5596–0.5981)
140.5814 (0.5615–0.6012)0.5886 (0.5690–0.6067)
160.5868 (0.5679–0.6048)0.5881 (0.5695–0.6066)
180.5901 (0.5717–0.6080)0.5857 (0.5673–0.6032)
200.5785 (0.5604–0.5964)0.5880 (0.5698–0.6050)
220.5841 (0.5668–0.6024)0.5880 (0.5706–0.6054)
240.5791 (0.5606–0.5974)0.5887 (0.5706–0.6059)
260.5787 (0.5600–0.5965)0.5863 (0.5689–0.6036)
280.5807 (0.5622–0.5990)0.5848 (0.5671–0.6019)
300.5802 (0.5612–0.5990)0.5844 (0.5661–0.6017)

Table 2.

Table 3: AUPRC (95% CI) by calendar-time lookback depth, same cohort and bootstrap as Table 2. AUPRC is reported against the realised evaluation prevalence (0.095) and is not comparable to population incidence. Peaks in bold.

Window (y)TransformerXGBoost
20.1128 (0.1026–0.1245)0.1082 (0.0982–0.1200)
40.1169 (0.1062–0.1289)0.1133 (0.1022–0.1261)
60.1206 (0.1098–0.1323)0.1162 (0.1060–0.1289)
80.1239 (0.1118–0.1367)0.1176 (0.1074–0.1308)
100.1259 (0.1141–0.1385)0.1193 (0.1093–0.1322)
120.1265 (0.1144–0.1393)0.1225 (0.1120–0.1346)
140.1272 (0.1153–0.1406)0.1256 (0.1151–0.1380)
160.1293 (0.1173–0.1430)0.1252 (0.1148–0.1381)
180.1323 (0.1197–0.1469)0.1235 (0.1133–0.1361)
200.1270 (0.1157–0.1410)0.1244 (0.1136–0.1367)
220.1278 (0.1163–0.1420)0.1242 (0.1141–0.1368)
240.1284 (0.1164–0.1425)0.1243 (0.1143–0.1362)
260.1273 (0.1153–0.1414)0.1240 (0.1142–0.1362)
280.1285 (0.1169–0.1431)0.1245 (0.1144–0.1367)
300.1284 (0.1166–0.1441)0.1239 (0.1141–0.1363)

Table 3.

References

  1. [1]Tianqi Chen and Carlos Guestrin. XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 785–794, 2016. doi: 10.1145/2939672.2939785.DOI
  2. [2]Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 4171–4186, 2019. doi: 10.18653/v1/N19-1423.DOI
  3. [3]Charles Gadd, Krishna Gokhale, Aditya Acharya, Jennifer Cooper, Francesca Crowe, Leah Fitzsimmons, Thomas Jackson, Krishnarajah Nirantharakumar, and Christopher Yau. SurvivEHR: a competing risks, time-to-event foundation model for multiple long-term conditions from primary care electronic health records. npj Digital Medicine, 9:546, 2026. doi: 10.1038/s41746-026-02709-z.DOI
  4. [4]Lin Lawrence Guo, Sofia E. Arciniegas, Adam P. Yan, Jason Fries, George A. Tomlinson, and Lillian Sung. Systematic review of foundation models for structured electronic health records. Journal of the American Medical Informatics Association, 33 (6):1190–1198, 2026. doi: 10.1093/jamia/ocag033.DOI
  5. [5]Jill Hardin and Jenna M. Reps. Evaluating the impact of covariate lookback times on performance of patient-level prediction models. BMC Medical Research Methodology, 21(1):180, 2021. doi: 10.1186/s12874-021-01370-2.DOI
  6. [6]Julia Hippisley-Cox and Carol Coupland. Identifying patients with suspected colorectal cancer in primary care: derivation and validation of an algorithm. British Journal of General Practice, 62 (594):e29–e37, 2012. doi: 10.3399/bjgp12X616346.
  7. [7]Zeljko Kraljevic, Dan Bean, Anthony Shek, Rebecca Bendayan, Harry Hemingway, Joshua Au Yeung, Alexander Deng, Alfred Balston, Jack Ross, Esther Idowu, James T. Teo, and Richard J. B. Dobson. Foresight—a generative pretrained transformer for modelling of patient timelines using electronic health records: a retrospective modelling study. The Lancet Digital Health, 6(4):e281–e290, 2024. doi: 10.1016/S2589-7500(24)00025-6.DOI
  8. [8]Josephina G. Kuiper, Myrthe P. P. van Herk-Sukel, Valery E. P. P. Lemmens, Ernst J. Kuipers, and Ron M. C. Herings. A steep increase in healthcare seeking behaviour in the last months before colorectal cancer diagnosis. BMC Family Practice, 22(1):121, 2021. doi: 10.1186/s12875-021-01482-0.DOI
  9. [9]Karen M. Kuntz, Iris Lansdorp-Vogelaar, Carolyn M. Rutter, Amy B. Knudsen, Marjolein van Ballegooijen, James E. Savarino, Eric J. Feuer, and Ann G. Zauber. A systematic comparison of microsimulation models of colorectal cancer: the role of assumptions about adenoma progression. Medical Decision Making, 31(4):530–539, 2011. doi: 10.1177/0272989X11408730.DOI

Paper details

vlnc:W_7jwz9huh
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 12

downloads · 2

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents