betaFeedback

WorksComputer Science

When Copying Is Hard to Beat: A Negative Result in Clinical Assessment Forecasting

Abstract

We report a negative result in next-day clinical assessment forecasting: the evaluated models show no consistent advantage over strong persistence-based baselines. Clinicians can view the previous day's questionnaire answers, a plausible source of documentation persistence. Predicting 45 items on 29,715 ICU day-pairs with patient-disjoint splits, a physiology-informed gated model improves item-macro-F1 over a one-line clinical rule by +0.0019 [+0.0006,+0.0031], while losing accuracy. No macro-F1 advantage is established when the physiology window ends at the previous form's reference time (+0.0001 [-0.0006,+0.0008]), or when scoring excludes the 68.5% of cells derived from parent answers; combining both changes yields a small disadvantage. Four checkpoints across three families underperform persistence, and a fine-tuned one differs from it on only two cells. A categorical read-out raises changed-cell accuracy eightfold, but a state-only read-out scores higher overall than forecast-only decoding and matches a transition matrix. This negative result highlights how the documentation process, input cutoff, observation mask and decoding rule shape conclusions about forecasting recorded assessments.

Introduction

We report a negative result: the evaluated predictors do not consistently improve on strong simple baselines for next-day clinical assessment forecasting. Clinicians completing the questionnaire can view the previous day’s answers, making persistence directly relevant to the documentation workflow. Access to prior answers could contribute to label stability alongside genuine clinical stability; the data cannot separate these effects. We therefore evaluate forecasts of recorded assessments, whose temporal consistency may partly reflect how they are collected.

We predict the next day’s values of 45 recorded items – diagnostic judgments, infection localization, organ dysfunction – against persistence, a one-line organ-acuity rule, a per-item transition matrix, supervised encoders, baseline-gated predictors and pretrained forecasters. Persistence is a strong baseline [3], and labels of this kind have been used for sepsis prediction [7]. The strongest gated model gains +0.0019 item-macro-F1 over the rule, interval excluding zero, but loses accuracy. Two task definitions qualify this gain. The input cutoff: the form carries a fixed daily reference time, so the usual physiology window overlaps the interval the target form describes. The observation mask: the instrument is conditional, and 68.5% of scored cells are materialized from a parent answer rather than entered. Under either, no macro-F1 advantage is established; under both the model is significantly worse. Only that summary flips – the gate loses accuracy and gains destination accuracy on changed cells in all five settings (Appendix § 6.5).

Pretrained forecasters fare worse – all three fall below copying, fine-tuned Chronos-Bolt differs from it on 2 of 281,952 cells, and a state-only read-out scores higher overall than forecast-based decoding. These findings document a negative result and show why the recording process and evaluation choices must be made explicit.

Task and evaluation

Cohort and assessment process. The export comprises 585,791 item entries from 3,736 ICU encounters and 39,148 questionnaire days on the surgical ICU of University Medicine Mannheim, Heidelberg University, over roughly six years; the instrument is the published Ground Truth for Sepsis Questionnaire [5]. Requiring at least four completed items on both days of a consecutive pair retains 29,715 day-pairs from 2,789 encounters. The export identifies encounters only; person-level identity comes from a separate table mapping these to 2,694 patients. Splitting by encounter – the convention in the work we audit – is not patient-disjoint: 24 patients had encounters in both training and test. Grouping each patient into one fold gives 18,688/4,708/6,319 day-pairs and 1,724/431/539 patients, with zero overlap (Appendix § 6); no result from the encounter split is carried over. The 45 targets comprise 25 binary and 20 multiclass items, not all ordinal. Each form covers a fixed 14:00–14:00 interval, is completed by a single attending physician per visit, and the tablet displays that physician’s preceding entries, which may contribute to documentation persistence. A changed value is not deterioration – “worsened” on successive days is an unchanged label – so we speak of a recorded assessment change.

Information available to the predictor. For item jj the target is yD+1,jy_{D+1,j} and persistence predicts yD,jy_{D,j}. The form timestamp is always 14:00 – the interval’s reference time, not an entry or signing time – and no timestamp recovers when a form was written, so when a physiology-based prediction could have been issued is not settleable from these data. Consecutive forms are exactly 24.00 h apart for 98.9% of pairs. Day-pairs are required to fall on consecutive calendar dates, a separate condition from an exact 24-hour timestamp difference; pairs failing it are excluded rather than scored. We therefore report two physiology settings over 24 h of 107 clinical variables: a late cutoff ending six hours before the target form’s reference time, which overlaps the interval that form assesses, and a matched cutoff ending at the preceding form’s reference time. Microbiology enters at the time each event became known.

Baselines and learned predictors. The rule copies the previous form but maps organ acuity from new to acute; it cannot express resolution to none. The transition matrix is a per-item first-order conditional arg⁡max⁡cP(yD+1=c∣j,yD)\arg\max_c P(y_{D+1}=c \mid j,y_D), fitted on training data without smoothing. Organ-acuity cells reading new on two consecutive days were corrected to acute on the later day; among stable cells the rule can err only on such cells, so after correction its false-change rate is zero by construction, and every row is retrained and rescored on uncorrected labels (Appendix § 6). Supervised models use item-specific cross-entropy. A gated variant mixes model logits zj(x)z_j(x) with baseline logits bj(yD)b_j(y_D),

z~j=γj(x)zj(x)+(1−γj(x))bj(yD),γj(x)∈[0,1],(1)\tilde{z}_j=\gamma_j(x)z_j(x)+(1-\gamma_j(x))b_j(y_D),\qquad\gamma_j(x)\in[0,1], \tag*{(1)}

over logits, not labels (Figure 2). The baseline path is persistence, or a confidence-thresholded per-item transition table which – verified by enumeration – installs exactly the eight organ-acuity new→acute redirections and so coincides with the rule.

The gate, as evaluated

Figure 2. The gate, as evaluated. Encoders and head produce per-item logits zj(x)z_j(x); a learned per-item, input-dependent γj(x)∈[0,1]\gamma_j(x) \in[0,1] mixes them in logit space (Eq. (1)) with a parameter-free baseline path (dashed) that either copies day DD or applies the organ-acuity rule. Because that path carries no parameters, γ(x)≡0\gamma(x) \equiv0 reproduces the baseline exactly: the baseline is a selectable special case of the model, not an external competitor, so falling below it is a statement about what the fitted model chose, not about the hypothesis class.

State accuracy and change detection. A scored cell is an item observed on both days, a change being undefined when day DD is missing; which cells qualify is itself a variable (§ 3). Test contains 281,952 cells from 539 patients, r=3.81%r = 3.81\% changing. With C=1[yD+1,j≠yD,j]∈{0,1}C = \mathbb{1}[y_{D+1,j} \ne y_{D,j}] \in\{0,1\} we report accuracy, item-macro-F1, AccΔ=P(y^=yD+1∣C=1)\mathrm{Acc}_{\Delta} = P(\hat{y} = y_{D+1} \mid C = 1) – destination accuracy on changed cells, not change detection – and FCR=P(y^≠yD∣C=0)\mathrm{FCR} = P(\hat{y} \ne y_D \mid C = 0), related exactly by

Acc−Accpers=rAccΔ−(1−r)FCR,(2)\mathrm{Acc} - \mathrm{Acc}_{\mathrm{pers}} = r\mathrm{Acc}_{\Delta} - (1-r)\mathrm{FCR}, \tag*{(2)}

so gains on changed cells must exceed errors on stable ones; macro-F1 obeys no such decomposition. Change scores are reported as AUROC and average precision against an item-identity baseline. Uncertainty is a paired bootstrap over patients; a difference is established only when its interval excludes zero. Full definitions are in Appendix A.

Results

A macro-F1 gain, at a cost in accuracy. Persistence attains 96.19% accuracy while predicting no changed cell by construction; the rule and the transition matrix reach 96.32% and 96.34%, recovering 3.49% and 7.23% of changed cells (Table 1). The transition matrix’s edge over the rule is +0.000164 [−0.000025, +0.000346] and is not established. Against that, the macro-F1-selected gate over the rule reaches 0.8730 macro-F1 and 4.43% AccΔ\mathrm{Acc}_{\Delta} – an improvement on item-macro-F1 while losing accuracy: +0.0019 [+0.0006, +0.0031] macro-F1, against −0.0022 [−0.0026, −0.0019] accuracy and a false-change rate rising from 0.0000 to 0.0027. Gating over plain persistence is worse on every count, including AccΔ\mathrm{Acc}_{\Delta} 0.01460.0146 lower than the rule [−0.0204-0.0204, −0.0086-0.0086]: departing from a weaker baseline is not departing usefully.

Table 1: Test point estimates, all rows from one evaluation pass on the patient-disjoint split, one mask and one label version. Row groups give the inputs; the late cutoff is defined in § 2 and the matched cutoff is in Table 8. AccΔ\mathrm{Acc}_{\Delta} is destination accuracy on changed cells, FCR the false-change rate on stable cells. Neural rows are five-seed means; ensembles are in Table 6. Best per column in bold.

PredictorAccuracyMacro-F1AccΔ\mathrm{Acc}_{\Delta}FCR
Questionnaire input only
Persistence (copy)0.96190.87050.00000.0000
Persistence + rule0.96320.87110.03490.0000
Transition matrix0.96340.86810.07230.0013
Questionnaire + physiology, late cutoff
GRU without baseline gate0.93390.80330.20740.0373
Gate over persistence0.95570.86870.02030.0072
Gate over rule, macro-F1 selected0.96090.87300.04430.0027
Change-weighted objective0.92930.84250.12530.0388

Table 1.

PredictorLate cutoffMatched cutoff
AccuracyMacro-F1AccuracyMacro-F1
Gate over persistence0.95570.86870.95570.8688
Gate over rule, macro-F1 selected0.96090.87300.96180.8712
GRU without baseline gate0.93390.80330.93300.7998
Change-weighted objective0.92930.84250.93360.8460

Table 8. Input-cutoff sensitivity. The late cutoff ends the physiology window six hours before the target form’s reference time and therefore overlaps the interval that form assesses; the matched cutoff ends it at the preceding form’s reference time. Models retrained on each.

The macro-F1 gain depends on task definition. Table 2 varies the input cutoff, the observation mask and the label version, singly and together, retraining wherever the training data changes. Ending the physiology window at the previous form’s reference time leaves +0.0001+0.0001 [−0.0006-0.0006, +0.0008+0.0008]; scoring only explicitly recorded cells leaves 0.00000.0000 [−0.0001-0.0001, +0.0001+0.0001]; under both the gate is significantly worse than the rule, though by a very small margin: −0.54×10−4-0.54 \times10^{-4} [−0.82-0.82, −0.29-0.29] ×10−4\times10^{-4}. A non-significant interval is not evidence of equality, so the claim is precise: no macro-F1 advantage is established under either alternative. Two effects are stable across all five settings, each interval excluding zero – the gate loses accuracy, and it gains destination accuracy on changed cells, from +0.0094+0.0094 [+0.0075+0.0075, +0.0114+0.0114] at the late cutoff to +0.0016+0.0016 [+0.0011+0.0011, +0.0021+0.0021] at the matched one (all five in Appendix A.5). The trade-off is reproducible; which side of it the item-macro-F1 summary lands on is not.

Table 2: The gate against the rule under each evaluation choice: differences of five-seed means with paired patient-clustered intervals, each contrast taken within its own setting. Late ends the physiology window 6 h before the target form and so inside the interval it describes; matched at the previous form’s reference time. Filled scores all cells, explicit only those a clinician entered. Entries marked * exclude zero; the last row is scaled by 10−410^{-4}. Explicit-mask macro-F1 rests on thin class support (Appendix A.6).

Cutoff / maskLabelsMacro-F1 Δ\DeltaAccuracy Δ\Delta
late / filledcorrected+0.0019+0.0019 [+0.0006+0.0006, +0.0031+0.0031] *−0.0022-0.0022 [−0.0026-0.0026, −0.0019-0.0019] *
late / filledraw+0.0018+0.0018 [−0.0011-0.0011, +0.0029+0.0029]−0.0033-0.0033 [−0.0036-0.0036, −0.0028-0.0028] *
matched / filledcorrected+0.0001+0.0001 [−0.0006-0.0006, +0.0008+0.0008]−0.0014-0.0014 [−0.0016-0.0016, −0.0012-0.0012] *
late / explicitcorrected0.00000.0000 [−0.0001-0.0001, +0.0001+0.0001]−0.0005-0.0005 [−0.0007-0.0007, −0.0003-0.0003] *
matched / explicitcorrected−0.54-0.54 [−0.82-0.82, −0.29-0.29] ×10−4\times10^{-4} *−1.29-1.29 [−1.87-1.87, −0.76-0.76] ×10−4\times10^{-4} *

Table 2.

Pretrained forecasters stay below copying; fine-tuning converges onto it. Chronos-Bolt [1], Chronos-2 [2] and the evaluated MIRA checkpoint [4, 6], given up to seven days of history, all fall below persistence (0.9599/0.85420.9599/0.8542 down to 0.9500/0.82310.9500/0.8231), agreeing with it on 98.498.4–99.7%99.7\% of cells. Fine-tuning does not surpass persistence on accuracy or macro-F1: fine-tuned Chronos-Bolt small differs from the copy on 2 of 281,952 scored cells, and MIRA improves markedly on its zero-shot scores (0.9500/0.82310.9500/0.8231 to 0.9607/0.85850.9607/0.8585) while still falling short. In these fine-tuning runs, predictions remained close to persistence. We claim the narrower reading: these checkpoints, under our decoding, do not beat copying here. A structural mismatch with categorical, rarely-changing fields would need a natively discrete-state control, which we do not evaluate.

A state-only read-out is a strong control. These models emit numbers and the targets are categories, so every row above also reflects a decoding choice: threshold the median, round and clip. Replacing it with a per-item multinomial map from the nine predicted quantiles to classes – fitted on train, regularization chosen on validation, forecasts frozen – moves AccΔ\mathrm{Acc}_{\Delta} from 1.18%1.18\% to 9.51%9.51\%, an eightfold change with nothing about the model altered. The comparison that bounds it is a control. The same read-out given only the item’s current state, with no pretrained features, reaches 0.9634/0.86810.9634/0.8681 with AccΔ\mathrm{Acc}_{\Delta} 7.23%7.23\%: higher on both aggregate metrics, though it recovers fewer changed cells than the quantile read-out (7.23%7.23\% against 9.51%9.51\%). It also matches the transition matrix of Table 1 on every reported metric, as expected, since a per-item multinomial on a one-hot current state is a first-order transition table. This is bounded: the forecast-based read-out does not beat a state-only one on the aggregate metrics, so such an adaptation needs this control before its transition gains are credited to the model. Adding forecast quantiles to the current-state read-out slightly increases macro-F1 (+0.0006+0.0006 [+0.0004+0.0004, +0.0008+0.0008]) but reduces changed-cell accuracy (−0.0036-0.0036 [−0.0051-0.0051, −0.0022-0.0022]); both paired patient-bootstrap intervals exclude zero (Appendix B).

A pooled change score mostly identifies the question, not the patient. A score conditioned on item and current value reaches pooled AUROC 0.83100.8310 on all cells and 0.91660.9166 on explicit cells, but each item’s marginal change frequency alone – using nothing about the patient – gives $0.7858รีเมียร์ and 0.8640, and within an item it falls to 0.6497 and 0.7469 (Appendix § 8): ranking summaries on different populations, not a decomposition into shares of explained signal.

Discussion and limitations

We find no consistent advantage over strong simple baselines across metrics and settings. The gate’s item-macro-F1 gain over the rule costs accuracy and is not established under either alternative.

The mask is the sharpest case. The instrument is conditional, so one “no organ dysfunction” answer materializes many zero cells; excluding them raises the change rate from 3.81% to 5.96% and lowers persistence’s accuracy from 96.19% to 94.04%. The convention is defensible – the derived values are logically correct – but it defines a different evaluation population, and which one is intended should be stated rather than inherited from a preprocessing default. Appendix § 6.6 states which we would score.

This single-center study concerns recorded assessments. Displayed prior answers may contribute to persistence (Figure 1), but the data do not separate documentation effects from clinical stability. The instrument changed once during the study period, and one target, suspicion of infection, is not comparable across the two versions (Appendix § 6). One severity item is relative to other ward patients, and label correction removes the stable cells on which the rule would err. There are 539 test patients and 150 positive new-acuity cells; candidates were searched on an earlier split with reported patient leakage. We establish neither unpredictability nor equivalence to the rule. This negative result is specific to the evaluated models and workflow; the input cutoff, observation mask and decoding rule must be reported and varied.

Figure 1. How the previous day enters the new form. Each item’s previous value is carried into the form the physician completes, so most rows are confirmed and a few are overridden. This is the mechanism behind the persistence baseline: it is not only a statistical reference point but part of the recording process, and the data cannot separate documentation persistence from genuine clinical stability. Schematic – the instrument itself is not shown. Two of the nine rows shown here differ; in the test data 3.81% of scored cells do.

Acknowledgements

Marius Fracarolli is funded by the Helmholtz Information & Data Science School for Health (HIDSS4Health), Karlsruhe/Heidelberg, Germany. We thank the clinical staff of the intensive care unit at University Medicine Mannheim for recording the assessments this study analyses, and the physicians and clinical staff who answered our questions about them.

Task specification and reproducibility

Cohort and labels

Table 3 gives the flow from export to analysis cohort, reproduced by report.py cohort. The prior version of this work reported 3,020 retained encounters; that figure is not reproducible from any split and is corrected here to 2,789 – the number of encounters contributing at least one consecutive day-pair. The day-pair count is unchanged.

QuantityCount
Source item entries / encounters / questionnaire days585,791 / 3,736 / 39,148
Questionnaire versions (v1 / v2 / missing)213,455 / 372,334 / 2
Items completed per form, median (IQR)13 (9–18)
Retained consecutive day-pairs / encounters / patients29,715 / 2,789 / 2,694
Training day-pairs / encounters / patients18,688 / 1,775 / 1,724
Validation day-pairs / encounters / patients4,708 / 453 / 431
Test day-pairs / encounters / patients6,319 / 561 / 539
Patient overlap between any two folds0
Scored test cells, both days observed281,952
Changed test cells (prevalence)10,754 (3.81%)

Table 3. Cohort flow and split composition. Patient identity comes from the demographics export’s lifetime number, which the questionnaire export does not carry; 15 encounters mapped to more than one lifetime number and were assigned to the first. Splitting by encounter is not patient-disjoint.

Splitting is by patient, 20% of patients to test and 20% of the remainder to validation, seed 42, not stratified. For reference, the encounter-based split used previously placed 24 patients in both training and test, 17 in training and validation, and 7 in validation and test.

The 45-item output comprises four judgment items, nine infection-site indicators, and 32 organ-related fields; the last block contains eight presence, eight acuity, eight cause, seven chronicity, and one capillary-leak field. Infection-site suspicion and confirmation were merged into binary presence; source-control fields were excluded. Cause categories are nominal: no ordinal distance is imposed on them anywhere in the evaluation or in the categorical decoding of Appendix § 7. The instrument is published: Lindner et al. [5] introduce the questionnaire, describe its structure and report interrater analyses for selected items, and Schamoni et al. [7] use its labels for sepsis prediction. What is not published, and is specified here instead, is the transformation from the recorded entries to the 45 modeling targets: the pivot of conditional items into per-organ and per-site fields, the hierarchy zero-fill, the harmonization of the two questionnaire versions, and the class encodings. Interrater analysis of selected published items does not extend to those derived targets.

Questionnaire versions. The instrument changed once, and the two versions are temporally disjoint: v1 covers 2016-06-01 to 2017-08-27 and v2 2017-08-28 onwards, so v1 is 24.2% of day-level records and 29.2% of test day-pairs under the patient-random split. v2 retired 15 item identifiers, none of which are among the 45 targets. One target draws on both codings – capillary leak, recorded as a sub-field in v1 and a standalone item in v2 – and is harmonized to a single binary field. One target is not comparable across the boundary: the suspicion of infection item admits a confirmed class that exists only in v2, where it is the modal value (v1: 92.3/7.7/0.0%; v2: 28.9/10.1/61.0%). Pooling cannot repair this and it inflates any train-early/test-late contrast on that item, so we report it as a limitation. It does not carry the headline: recomputing the gate-versus-rule macro-F1 contrast from the saved predictions with that item dropped from the item average changes it by +0.0001.

Ambiguous patient identifiers. Fifteen of the 9,460 mapped encounters carry more than one lifetime number in the demographics export, and each was assigned to the first. Five of the fifteen fall in the retained cohort, and for all five the alternative identifier belongs to a patient in the same fold, so no assignment among the recorded alternatives would place an encounter on both sides of a split. Patient-disjointness therefore does not depend on the tie-break.

Prediction time and preprocessing

Every form’s recorded timestamp is 14:00, for 100.0% of 585,791 item entries: it is the reference time of the 14:00–14:00 interval the form describes, not the moment of entry or signing. No entry or signing time exists in the export, so the interval between a form being written and becoming available cannot be measured, and no analysis here can establish it. Consecutive forms are 24.00 h apart for 98.9% of consecutive pairs.

Two physiology cutoffs are therefore reported. The late cutoff uses [tD+1−30 h, tD+1−6 h][t_{D+1} - 30\ \mathrm{h},\ t_{D+1} - 6\ \mathrm{h}], which is the setting of the earlier work; it ends six hours before the target form’s reference time and so overlaps the interval that form assesses. The matched cutoff uses [tD+1−48 h, tD+1−24 h][t_{D+1} - 48\ \mathrm{h},\ t_{D+1} - 24\ \mathrm{h}], ending exactly at the preceding form’s reference time. Physiology comprises 107 clinical variables at hourly resolution with an explicit observed/missing channel per variable; microbiology events are placed at the time they became known (cultures at collection, results and positivity at result arrival). All normalization statistics are fitted on training data only.

The label correction rewrites organ-acuity cells reading new on two consecutive calendar days to acute on the later day. It is decided against the original series, conditions only on the preceding day, and is applied to the day-level table before pairing, so a day carries one value whether it is used as an input or as a target; a day-DD input is never corrected using day-D+1D+1 information. Corrections affect 27 organ-acuity cells cohort-wide – 13 training, 6 validation and 8 test under the patient-disjoint split – verified by differencing the corrected and uncorrected target arrays on row-aligned splits. Before correction, 27 of 2,470 organ-acuity pairs beginning at new were new–new (1.09%); after it, 0 of 2,447. The denominator falls because correcting day D+1D+1 removes the following pair from those beginning at new, and 4 of the 27 corrected cells had no following pair.

The correction interacts with the rule and this must be read carefully. The rule maps new to acute, so the only stable cells on which it can err are those reading new on both days – exactly the cells the correction rewrites. After correction, 0 of the 2,447 organ-acuity pairs whose first day is new remain new–new, so the rule cannot produce a false change on these items at all, and its FCR of zero in Table 1 is an arithmetic consequence of the label version, not evidence that new–new is clinically impossible. The clinicians’ judgment that the form’s wording forbids it is the reason for the correction; it is not independently confirmed by the rule’s performance. Because the operation changes the target, rescoring alone would not measure its effect, so Table 7 reports every row retrained and rescored on uncorrected labels.

PredictorCorrected labelsRaw labels
AccuracyMacro-F1AccuracyMacro-F1
Persistence0.96190.87050.96190.8719
Persistence + rule0.96320.87110.96320.8711
Transition matrix0.96340.86810.96330.8680
Gate over rule, macro-F1 selected0.96090.87300.95990.8729

Table 7. Label sensitivity. Every row retrained and rescored on uncorrected labels, so the comparison reflects the training signal as well as the scoring target. On macro-F1 the correction leaves the rule unchanged and the gated model nearly so, although the gate’s accuracy moves by 0.0010; what it moves substantially is persistence, and with it the ordering of the two simple baselines.

The sensitivity is instructive and runs opposite to the intuition that a correction helps the corrected-for predictor. The rule’s macro-F1 is unchanged (0.8711) and the gated model’s nearly so (0.8730 against 0.8729), though the gate’s accuracy does move, 0.9609 to 0.9599. What the correction moves is persistence, from 0.8719 on raw labels to 0.8705 on corrected: the new–new cells were cells persistence scored correct for free, and removing them costs it exactly those. The consequence is that the ordering of the two simple baselines depends on the label version – on raw labels persistence outscores the rule on macro-F1 (0.8719 against 0.8711), on corrected labels the rule outscores persistence – and the paired interval for that difference, +0.0007 [−0.0001, +0.0018] on raw labels, does not exclude zero. On raw labels the rule also makes 5 false changes where on corrected labels it makes none, confirming that the zero is a property of the label version rather than of the rule. Any claim that the rule improves on persistence should therefore be read as conditional on an authorized correction that removes the cells where the rule would be wrong. Consecutive new entries appear to be a recording error: new denotes a dysfunction first entered within the preceding 24 hours, so two in a row cannot both be correct. We proposed the correction on that basis and the attending physicians who complete the questionnaire accepted it. It was then applied programmatically to 27 cells, not by case-by-case chart review, so the rule was adjudicated but individual cells were not. The correction is load-bearing – it is what makes the rule’s false-change rate zero – so Table 7 reports every row retrained and rescored without it.

Models and selection

All learned rows are GRU encoders over the physiology sequence with concatenation fusion, an item-wise output head, batch size and learning rate as listed below, dropout, the Adam optimiser, an epoch cap of 200 and early stopping on validation macro-F1 with patience six. Seeds are 42, 123, 456, 789 and 1337 for every row. The gate mixes logits as in Equation (1); binary baseline logits are ±c\pm c and multiclass baseline logits assign cc to the baseline class and zero elsewhere, with c=8c = 8 unless stated. The thresholded transition table used by the gate over rule rows is estimated on the training split only, with minimum support 10 and purity threshold τ=0.5\tau= 0.5; a state is redirected only when its modal successor differs from itself. Enumerating its entries (report.py markovmap) shows it contains precisely the eight organ-acuity new→acute redirections, so it is the rule and not a larger table – a data-driven baseline given no clinical knowledge recovers the clinicians’ rule exactly. This is verified rather than inferred from the override count that the training log reports. A saturated baseline makes departures expensive, which is an inductive bias rather than evidence that optimization is solved.

Rowbaseline pathwidthlayersfflrbatchdrop
GRU without baseline gatenone12825125×10−45 \times10^{-4}640.05
Gate over persistencecopy, c=8c = 8, λ=0.5\lambda= 0.512825125×10−45 \times10^{-4}640.05
Gate over rule (a)table, c=8c = 8, λ=0.5\lambda= 0.56435122×10−32 \times10^{-3}320.3
Gate over rule (b)table, c=8c = 8, λ=0.5\lambda= 0.56435121×10−41 \times10^{-4}320.3
Gate over rule (c)table, c=4c = 4, λ=0.5\lambda= 0.512825125×10−45 \times10^{-4}640.05
Change-weighted objectivetable, c=8c = 8, λ=0.5\lambda= 0.512825125×10−45 \times10^{-4}640.05

Table 4. Configuration of every learned row. All use a GRU physiology encoder with concatenation fusion, Adam, an epoch cap of 200, early stopping on validation macro-F1 with patience 6, uniform loss balance, the both-days mask, and seeds 42/123/456/789/1337. cc is the baseline logit scale; λ\lambda weights an auxiliary loss that supervises the gate γj(x)\gamma_j(x) against whether the cell actually changed, added to the item-wise cross-entropy; γj(x)\gamma_j(x) is produced by a sigmoid head over the fused representation, initialized at zero logit. Candidates a–c differ in learning rate, width, depth, dropout, batch size and baseline scale, as listed; the reported gate over rule row is whichever maximizes validation macro-F1, selected inside the evaluation script before test is read.

The change-weighted row adds a reweighting of changed cells by a factor of five in the loss; no other row reweights. The transition table used as a baseline path is refitted on the training split of whichever campaign a row belongs to, never carried across campaigns.

Selection protocol, stated exactly. The hyperparameter search reported in the earlier version of this work was carried out on the encounter-based split and is not repeated here. What is repeated is the selection: the configurations that were validation-optimal in that search were retrained on the patient-disjoint split with five seeds each, and the configuration reported in Table 1 is the one maximizing validation macro-F1 among them. Test was read once, afterwards, by a single evaluation script. This is a validation-based selection informed by an earlier exploratory search on a different split; it is not a preregistered protocol and is not described as one. The full run manifest – run directory, configuration, seed, selected checkpoint, input cutoff, label version and aggregation for every row – is emitted as fmts_manifest.json by the evaluation script and released with the code.

Aggregation. Neural rows in Table 1 are means over the five seeds, with standard deviations in the released metrics file. The majority-vote ensemble over the same five seeds is reported separately in Table 6. Means and ensembles are never combined in one row, and five seeds are not treated as five independent test cohorts.

PredictorSeed mean ± sdEnsemble
AccuracyMacro-F1AccuracyMacro-F1
Gate over persistence0.9557 ± 0.00060.8687 ± 0.00110.95900.8721
Gate over rule, macro-F1 selected0.9609 ± 0.00080.8730 ± 0.00160.96240.8749
GRU without baseline gate0.9339 ± 0.00130.8033 ± 0.00120.93990.8154
Change-weighted objective0.9293 ± 0.00400.8425 ± 0.00630.93890.8540

Table 6. Seed mean ± sd against the majority-vote ensemble over the same five seeds. The two are different estimators and are not interchangeable: every number in the main text, every interval and Figure 3 use the seed mean. The ensemble columns appear only here. An earlier version of this work reported the mean in its tables while bootstrapping the ensemble, which is why the distinction is made explicitly.

Metric definitions and uncertainty

Macro-F1 is the mean over items of the per-item class-macro F1. Each item’s class set is fixed from the training split and held constant on the test split and inside every bootstrap resample, so a class that happens to be absent from a resample contributes an F1 of zero rather than being dropped from the average; zero division is scored as zero. Changed-cell macro-F1 repeats that calculation on cells with C=1C = 1; it is a class-macro F1 restricted to changed cells, not the F1 of a binary change detector, and the two are never combined into one claim. Average precision is sklearn.average_precision_score. Micro-F1 equals accuracy for single-label multiclass targets and is therefore omitted.

Equation 2 follows by partitioning correct predictions into changed and unchanged cells: on unchanged cells every departure from persistence is an error. Because both Acc⁡Δ\operatorname{Acc}_{\Delta} and FCR are reported on the same mask, the identity is checkable from Table 1; Figure 3 does not use it, and instead counts departures directly from the paired per-cell predictions.

Figure 3. Departures from persistence per 1,000 of the 281,952 scored test cells, counted directly from the paired per-cell predictions on the common evaluation mask: corrected (persistence wrong, model right), broken (persistence right, model wrong), and futile (wrong either way). These are counts, not reconstructions from aggregate identities. Note the scale: the two rows without a baseline gate break 35–37 cells per 1,000 to correct 5–8.

Uncertainty is a paired bootstrap over patients: 2,000 resamples of the 539 test patients with replacement, all days and items of a sampled patient kept together, and the same resamples used for every predictor so that differences remain paired. Intervals are 95% percentile intervals of the resampled paired difference (2.5th and 97.5th percentiles), uncorrected. Resampling day-pairs, as the earlier version did, preserves the dependence among the 45 items of a form – the whole form moves together – but not the dependence between several day-pairs of the same patient, which is why the patient is the resampling unit here. Seed variability, patient-sampling uncertainty and ensemble performance are reported as three separate quantities. We describe a difference as established only when its interval excludes zero, and use no other significance language.

Changed-cell accuracy across settings

The claim that the gate improves destination accuracy on changed cells in every setting rests on all five contrasts, not only the two quoted in the main text.

Table 5: Acc⁡Δ\operatorname{Acc}_{\Delta} of the selected gate against the rule, under each setting. Differences of five-seed means with paired patient-clustered bootstrap intervals; every interval excludes zero.

Cutoff / maskLabelsAcc⁡Δ\operatorname{Acc}_{\Delta} difference
late / filledcorrected+0.0094 [+0.0075, +0.0114]
late / filledraw+0.0160 [+0.0134, +0.0186]
matched / filledcorrected+0.0016 [+0.0011, +0.0021]
late / explicitcorrected+0.0109 [+0.0085, +0.0134]
matched / explicitcorrected+0.0004 [+0.0001, +0.0006]

Table 5.

The observation mask

A cell is explicit when the clinician entered a value for that field on both days of the pair; the derived cells excluded are those the hierarchy fill materializes from a parent answer. Transferring that mask onto the main cohort, matched pair by pair on encounter and date, retains 88,924 of 281,952 scored cells (31.5%) over the same 6,319 day-pairs and 539 patients, and raises the change rate from 3.81% to 5.96%. These rows are a rescoring: identical predictions, targets and models, with only the set of scored cells changed, so nothing is retrained or reselected.

The explicit subset is a different forecasting population, not the filled one with easy negatives removed. It retains 5,296 of the 10,754 changed cells, so it discards 51% of all transitions along with the stable cells: requiring an explicit entry on both days excludes, in particular, a transition from a parent-derived negative to a newly documented positive, of which 1,207 occur on the site and organ-presence items. Neither mask is automatically the more clinically valid task, and we do not argue that the explicit one is; the point is that the comparison changes with the choice.

Which mask we would score. Asked to choose, we would report the filled mask as the primary population with the explicit mask beside it, and we state that as a judgement rather than a property of the data. A derived zero is a real clinical assertion – the clinician denied the parent condition – and a forecaster deployed on this instrument would have to produce those cells too. But the explicit mask is the harder population and the more informative one, and it discards 51% of the transitions rather than only easy negatives. Neither alone is sufficient: the pair is the result, which is why every contrast in this paper is reported within a mask.

Why we do not compare macro-F1 across masks. Excluding derived cells removes at least one observed class from 40 of the 45 items and leaves 30 with a single class: the nine infection-site fields fall from two observed classes to one, because a site is recorded only when ticked, and most organ fields do likewise. Under our fixed-class-set convention an absent class contributes an F1 of zero, so persistence’s item-macro-F1 would read 0.5349; counting only the classes present in each population instead gives 0.9446, because a single-class item is then trivially perfect. A spread of 0.41 induced by the scoring convention alone, on the same predictions, is the clearest possible demonstration that the statistic is not comparable between masks. Neither figure measures predictive skill. Item-macro-F1 is therefore not comparable between masks and we do not use it that way: the cross-mask statement is made on accuracy, which is well defined in both populations, and macro-F1 is retained only for paired comparisons within a mask, where both predictors face identical class support.

Aggregation, label sensitivity and temporal transfer

In these runs the ensemble is better than the seed mean on every row, by more than the seed standard deviation in each case. We report that as an observation rather than a guarantee of majority voting; it is not extra evidence about the method, and it is the reason the two are never mixed within a row: reporting an ensemble beside means would flatter whichever rows happened to be ensembled.

Against the rule, the macro-F1 difference of the selected gate is +0.0019 [+0.0006, +0.0031] at the late cutoff and +0.0001 [−0.0006, +0.0008] at the matched cutoff; destination accuracy on changed cells goes from +0.0094 [+0.0075, +0.0114] to +0.0016 [+0.0011, +0.0021]. The macro-F1 advantage is established only at the cutoff that draws on information from inside the assessed interval, and is not established at the matched cutoff. The changed-cell improvement is established at both: +0.0016 [+0.0011, +0.0021] even at the matched cutoff.

Per-item results

Table 9 reports all 45 items separately. The aggregate gain is concentrated: of the +0.0019 item-macro-F1 the gate adds over the rule, +0.0027 comes from the eight organ acute or new items and −0.0008 from the remaining 37. Across all 45 items the gate is better on 9, worse on 16 and unchanged on 20 at the four decimals shown. Persistence and persistence + rule coincide on 37 of 45 items, the rule acting only on the new-acuity items. Forecastability varies widely: patient’s development in last 24 h is hardest (macro-F1 0.5819 under persistence, with 30.2% of its cells changing), together with the organ acute or new items (0.628–0.646), while the chronic fields sit at 0.978–0.992 with under 0.5% of cells changing. The mean over items of each macro-F1 column is the corresponding aggregate in Table 1.

Itemnnchg%Persist.RuleGateΔ\DeltaAccΔ\Delta
Daily rating of illness severity6,31921.590.72340.72340.7245+0.00110.012
Patient’s development in last 24 h6,31730.220.58190.58190.5846+0.00260.016
Working diagnosis6,31911.810.84890.84890.8479-0.00100.001
Suspicion of infection6,1696.840.86970.86970.8675-0.00220.016
Focus of infection: abdominal6,2703.080.96650.96650.9660-0.00050.006
Focus of infection: endocarditis6,2700.110.87010.87010.87010.00000.000
Focus of infection: blood stream6,2701.450.95150.95150.95150.00000.000
Focus of infection: skin6,2700.620.97300.97300.97300.00000.000
Focus of infection: intracranial/meningeal6,2700.610.97000.97000.97000.00000.000
Focus of infection: catheter6,2700.810.93030.93030.93030.00000.000
Focus of infection: thoracic6,2703.590.96080.96080.96080.00000.000
Focus of infection: urogenital6,2700.540.94340.94340.94340.00000.000
Focus of infection: bone/joint6,2700.180.98520.98520.98520.00000.000
Capillary leak6,2453.960.95160.95160.9507-0.00080.014
Lung: present6,2634.060.95060.95060.9504-0.00020.002
Lung: acute or new6,2635.750.63060.63370.6794+0.04570.276
Lung: cause6,2634.610.91750.91750.91750.00000.000
Lung: chronic6,2630.320.97880.97880.97880.00000.000
Kidney: present6,2632.620.97340.97340.97340.00000.000
Kidney: acute or new6,2633.820.64560.64830.6674+0.01910.254
Kidney: cause6,2633.080.95020.95020.9501-0.00010.002
Kidney: chronic6,2630.490.98400.98400.98400.00000.000
Brain: present6,2634.230.95760.95760.95760.00000.000
Brain: acute or new6,2635.170.63620.63960.6431+0.00350.199
Brain: cause6,2634.810.93440.93440.93440.00000.000
Brain: chronic6,2630.420.97750.97750.97750.00000.000
Heart: present6,2633.580.95640.95640.9560-0.00040.008
Heart: acute or new6,2634.760.62810.63320.6559+0.02270.197
Heart: cause6,2633.700.85100.85100.8495-0.00150.012
Heart: chronic6,2630.480.98670.98670.98670.00000.000
Liver: present6,2632.550.95230.95230.9504-0.00190.033
Liver: acute or new6,2632.920.62880.63290.6434+0.01040.142
Liver: cause6,2632.270.93140.93140.9269-0.00450.021
Itemnnchg%Persist.RuleGateΔ\DeltaAccΔ\Delta
Liver: chronic6,2630.260.97990.97990.97990.00000.000
Gastrointestinal: present6,2634.140.95540.95540.95540.00000.000
Gastrointestinal: acute or new6,2634.840.63560.63850.6525+0.01400.160
Gastrointestinal: cause6,2634.120.85950.85950.8594-0.00010.001
Gastrointestinal: chronic6,2630.100.99100.99100.99100.00000.000
Coagulation system: present6,2633.210.95280.95280.9524-0.00050.005
Coagulation system: acute or new6,2633.910.63120.63630.6462+0.00990.168
Coagulation system: cause6,2633.160.90340.90340.9020-0.00140.006
Coagulation system: chronic6,2630.140.99160.99160.99160.00000.000
Bone marrow: present6,2630.930.95860.95860.9562-0.00230.003
Bone marrow: acute or new6,2630.960.63850.63950.6362-0.00330.037
Bone marrow: cause6,2630.460.67790.67790.6566-0.02140.000

Table 9. Per-item test results, all 45 items. nn is the item’s scored cells (observed on both days) and chg% the share of them that change; macro-F1 is the per-item class-macro F1 whose mean over items is the aggregate in Table 1, so each F1 column averages to that row’s published value. Δ\Delta is gate − rule. AccΔ\Delta is the gate’s destination accuracy on that item’s changed cells. The gate columns are the mean over five seeds.

Ethics and availability

Institutional review board. This study analyses retrospectively collected routine-care data from the intensive care unit of University Medicine Mannheim, Heidelberg University, Germany. Retrospective analysis was approved by the Medical Ethics Commission II of the Medical Faculty Mannheim, Heidelberg University, Germany, with waiver of patient consent (reference numbers 2016-800R-MA, 2016-840R-MA and 2023-851).

Data and code availability. The questionnaire and clinical data are single-center routine-care records and cannot be made publicly available. The analysis code – preprocessing, splitting, training, and the single evaluation script that produces every number in this paper – is publicly available at https://github.com/StatNLP/FMTS_2026_When-Copying-Is-Hard-to-Beat and included as supplemental material. The aggregate evaluation artefacts that back Table 1 and Figure 3 – metrics, patient-clustered intervals, departure counts, and the run manifest recording the input cutoff, label version, mask, reporting object and seeds of every row – are written by that script and included. Per-cell predictions are not included: they carry the recorded questionnaire values and patient identifiers for every scored day, and the persistence row is identical to the day-DD record, so releasing them would release the data itself.

LLM usage. The main ideas are by the authors. After completing the initial manuscript, Anthropic’s Claude (Opus 4.8, Opus 5 and Opus 5.5), OpenAI’s Astra, and paperreview.ai provided feedback and suggestions. Claude also assisted with developing, documenting, and extending the code. No patient-level data from the database were shared with or processed by these tools, in accordance with the applicable data-use agreements.

Additional pretrained-forecaster results

Histories are up to seven days of questionnaire values (mean 5.51 observed days on test). Chronos-Bolt processes items individually; Chronos-2 processes them jointly; the MIRA checkpoint is evaluated itemwise with observation timestamps, padding absent days with NaN rather than a repeated value. Exact checkpoints, by repository and commit:

Table 10: Evaluated checkpoints. The MIRA weights are the MIRA-Mode open-source reproduction, whose training corpus differs from the model described in the original paper [4]; its pretraining claims are therefore not attributed to these weights [6]. Chronos-2’s support for categorical covariates is not categorical target modeling [2].

RowRepositoryRevision
Chronos-Bolt smallamazon/chronos-bolt-small772f3d25
Chronos-Bolt baseamazon/chronos-bolt-base5d9f166d
Chronos-2amazon/chronos-229ec3766
MIRAMIRA-Mode/MIRA932caadc

Table 10.

Fine-tuning protocol. Both fine-tuned rows adapt all parameters of the released checkpoint with AdamW (weight decay 0.01), learning rate 10−410^{-4}, batch 256, for up to 10 epochs over 835,133 training and 209,673 validation item-series, using each implementation’s native objective – Chronos-Bolt’s own quantile loss, MIRA’s native loss – rather than a categorical one. The epoch is chosen on validation loss; test is never consulted during fine-tuning. All pretrained-forecaster rows, zero-shot and fine-tuned, are single runs, not seed means; only the supervised rows of Table 11 are averaged over five seeds. Fine-tuned Chronos-Bolt small differs from the copy on exactly 2 of the 281,952 scored cells.

ModelAccuracyMacro-F1AccΔ_{\Delta}F1Δ_{\Delta}Agreement
Persistence (harness check)0.96190.87050.00000.0000100.0%
Chronos-Bolt small, zero-shot0.95990.85420.00700.001399.7%
Chronos-Bolt base, zero-shot0.96010.85320.01180.004199.7%
Chronos-2, zero-shot0.95700.84950.01930.011099.3%
MIRA checkpoint, zero-shot0.95000.82310.04330.022098.4%
Chronos-Bolt small, fine-tuned0.96190.87050.00000.0000100.0%
MIRA checkpoint, fine-tuned0.96070.85850.06950.005899.3%
Chronos-Bolt base, read-out on forecast quantiles0.95940.82360.09510.020898.9%
control: same read-out, current state only0.96340.86810.07230.020499.5%
control: state + forecast quantiles0.96330.86870.06860.020399.6%

Table 11. Pretrained-forecaster results on the patient-disjoint split and the common mask. F1ΔF1_{\Delta} is changed-cell macro-F1. Agreement is the fraction of scored cells matching persistence.

Numerical versus categorical decoding. The rows above decode a numerical forecast by thresholding the median at 0.5 for binary items and rounding-and-clipping it to the class range for multiclass items. That imposes a metric on the label space, which is defensible for an ordinal sensitivity and not for a nominal cause. As a bounded sensitivity check we keep the model’s predictive distribution and map its nine forecast quantiles to class probabilities with a per-item multinomial logistic regression, fitted on the training split with its regularization strength chosen on validation. The comparison is decoding against decoding on identical forecasts, so a difference is attributable to the read-out rather than to a stronger model, and no ordinal distance is imposed on nominal categories. On identical frozen forecasts, the categorical read-out moves destination accuracy on changed cells from 1.18% to 9.51% – an eightfold increase – while macro-F1 falls from 0.8532 to 0.8236 and accuracy is essentially unchanged (0.9601 to 0.9594). The conclusion that these forecasters barely anticipate transitions is therefore substantially a statement about collapsing a predictive distribution to its median and rounding it, not only about the forecasts themselves. Two controls bound what that shows. Giving the same head only the item’s current state, with no pretrained features, reaches 0.9634/0.8681 with Acc⁡Δ\operatorname{Acc}_{\Delta} 7.23% – higher on both aggregate metrics, and numerically the transition matrix of Table 1, which it is. Giving it the state and the quantiles reaches 0.9633/0.8687 with Acc⁡Δ\operatorname{Acc}_{\Delta} 6.86%. Against the state-only control these differences are small but resolvable: macro-F1 +0.0006 [+0.0004, +0.0008], Acc⁡Δ\operatorname{Acc}_{\Delta} −0.0036 [−0.0051, −0.0022], accuracy 0.0000 [−0.0001, +0.0000], paired and clustered on the 539 test patients. Adding the forecasts to the current state therefore slightly increases macro-F1 and reduces changed-cell accuracy, both resolvably. We do not claim the forecasts are uninformative given the state, nor that the net effect is unhelpful: it is a trade-off between the two metrics, of the same kind the rest of the paper documents, and which side matters depends on what the forecast is for. The forecast-only read-out, by contrast, does recover more changed cells than the state-only control (Acc⁡Δ\operatorname{Acc}_{\Delta} +0.0228 [+0.0170, +0.0287]) at a large cost in both aggregate metrics. All three fitted rows use training labels, are not zero-shot, are not comparable with the rows above them, and cover Chronos-Bolt base only.

Change-score analysis and temporal holdout

The empirical change score is a fitted frequency baseline, not a data-free predictor: for each (item, current value) pair it estimates the training-split frequency of C=1C = 1. It is fitted on the training split only, uses a cell only where that cell has at least five training observations, falls back to the global training change rate for unseen pairs, and applies no smoothing. An item-only score estimates the corresponding marginal. Within an (item, state) stratum the lookup assigns one score to every patient, so it cannot rank patients inside a stratum; the within-item figure is the support-weighted mean of per-item AUROCs over items with both outcomes present in test, and it measures discrimination between current states, not patient-specific forecasting.

QuantityRandom splitChronological split
Scored cells281,952250,682
Change prevalence3.81%3.89%
Pooled AUROC (item + state)0.83100.8149
Pooled average precision0.27490.2789
Item identity only, AUROC0.7858—
Within-item (state only), AUROC0.6497—
Paired difference, item+state − item only+0.0452—
Rule-covered transitions removed, AUROC0.8249—
Rule-covered transitions removed, AP0.2274—

Table 12. Change score on the random and the chronological split. Both are patient-disjoint; the chronological split places all of a patient’s day-pairs in one date range and drops patients who straddle a cutoff (below). The paired difference has 95% patient-bootstrap interval [+0.0408, +0.0498]. The rule-covered rows remove the new→acutenew\to acute transitions the one-line rule already produces, which are not new information for a reader who has that rule. A dash marks a decomposition we computed on the random split only; it is missing, not zero.

Chronological holdout, and what it measures. Assigning each patient by the date of their first day-pair keeps patients out of two folds but does not separate the folds in time: a patient first seen in 2016 and readmitted in 2021 carries 2021 data into training. Under that design our folds spanned 2016-06-01–2022-01-12 (train) against 2020-12-14–2022-03-30 (test), an overlap of thirteen months. We therefore require every day-pair of a patient to fall on one side of each cutoff and drop patients who straddle one: 38 of 2,694 patients (1.4%), giving 18,738/4,432/5,577 pairs over 2016-06-01– 2019-10-08, 2019-10-11–2020-11-29 and 2020-12-02–2022-03-30, with no overlap and no shared patients. The lookup and all fitted preprocessing statistics are refitted within the training period.

Strict separation makes the comparison sharper and its interpretation narrower. Training now lies entirely before the COVID-19 period and test entirely within it, and the cohort changes substantially: between the training and test folds the prevalence of recorded gastrointestinal dysfunction rises from 26.5% to 53.8% and cerebral dysfunction from 42.3% to 69.2%. This row therefore measures robustness to a large case-mix shift, not to elapsed time alone, and the questionnaire-version boundary lies inside the training fold, which inflates the apparent shift on the infection item specifically. What does not move is the property the paper is about: the cell change rate is 3.80%, 3.99% and 3.89% across the three folds, so persistence dominance is not an artefact of a quieter pre-pandemic cohort.

PredictorAccuracyMacro-F1AccΔ_{\Delta}FCR
Persistence0.96110.86410.00000.0000
Persistence + rule0.96240.86450.03240.0000
Transition matrix0.96290.86170.07530.0012
GRU without baseline gate0.92220.78150.19990.0486
Gate over persistence0.95510.86270.02000.0070
Gate over rule, macro-F1 selected0.95940.86660.04850.0037
Change-weighted objective0.93480.84090.12140.0323

Table 13. Chronological holdout, point forecasters, five-seed means, on the strictly separated split. This row is heavily qualified and we do not rest any claim on it. Training lies entirely before the COVID-19 period and test entirely within it; the cohort is markedly sicker in test (recorded gastrointestinal dysfunction 26.5%→\to53.8%, cerebral 42.3%→\to69.2%); and the questionnaire-version boundary falls inside the training period, so one target is coded differently either side of it. What the comparison shows is robustness to a large case-mix change rather than to elapsed time. The property the paper concerns does not move: the cell change rate is 3.80%, 3.99% and 3.89% across the three folds, and against the rule the gated model behaves as on the patient-random split, +0.0020+0.0020 macro-F1 [+0.0006+0.0006, +0.0036+0.0036] at −0.0030-0.0030 accuracy [−0.0036-0.0036, −0.0024-0.0024].

Neither AUROC nor these retrospective labels establishes the benefit of asking a clinician to review a field, so alert-budget and decision-curve claims are excluded. The point-forecast and binary-change metrics are also not combined into a single claim that a model predicts whether but not what: they are different tasks with different baselines, and we report them separately.

References

  1. [1]Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Schhur, Syama Sundar Rangapuram, Sebastian Pineda Arango, Shubham Kapoor, Jasper Zschiegner, Danielle C. Maddix, Hao Wang, Michael W. Mahoney, Kari Torkkola, Andrew Gordon Wilson, Michael Bohlke-Schneider, and Yuyang Wang. Chronos: Learning the language of time series. Transactions on Machine Learning Research, 2024. URL https://openreview.net/forum?id=gerNCVqqtR.arxiv.org/abs/2403.07815
  2. [2]Abdul Fatir Ansari, Oleksandr Schhur, Jaris Küken, Andreas Auer, Boran Han, Pedro Mercado, Syama Sundar Rangapuram, Huibin Shen, Lorenzo Stella, Xiyuan Zhang, Mononito Goswami, Shubham Kapoor, Danielle C. Maddix, Pablo Guerron, Tony Hu, Junming Yin, Nick Erickson, Prateek Mutalik Desai, Hao Wang, Huzefa Rangwala, George Karypis, Yuyang Wang, and Michael Bohlke-Schneider. Chronos-2: From univariate to universal forecasting. arXiv:2510.15821, 2025. URL https://arxiv.org/abs/2510.15821.
  3. [3]Haniska Hewamalage, Klaus Ackermann, and Christoph Bergmeir. Forecast evaluation for data scientists: common pitfalls and best practices. Data Mining and Knowledge Discovery, 37: 788–832, 2023. doi: https://doi.org/10.1007/s10618-022-00894-5.
  4. [4]Hao Li, Bowen Deng, Chang Xu, Zhiyuan Feng, Viktor Schlegel, Yu-Hao Huang, Yizheng Sun, Jingyuan Sun, Kailai Yang, Yiyao Yu, and Jiang Bian. MIRA: Medical time series foundation model for real-world health data. arXiv:2506.07584, 2025. URL https://arxiv.org/abs/2506.07584.
  5. [5]Holger A. Lindner, Shigehiko Schamoni, Thomas Kirschning, Corinna Worm, Bianka Hahn, Franz-Simon Centner, Jochen J. Schoettler, Michael Hagmann, Jörg Krebs, Dennis Mangold, Stephanie Nitsch, Stefan Riezler, Manfred Thiel, and Verena Schneider-Lindner. Ground truth labels challenge the validity of sepsis consensus definitions in critical illness. Journal of Translational Medicine, 20(27), 2022. doi: 10.1186/s12967-022-03228-7.DOI
  6. [6]MIRA-Mode. MIRA: Open-source reproduction (model card). Hugging Face model repository MIRA-Mode/MIRA, revision 932caadc, 2025. URL https://huggingface.co/MIRA-Mode/MIRA. Accessed 10 September 2026.
  7. [7]Shigehiko Schamoni, Holger A. Lindner, Verena Schneider-Lindner, Manfred Thiel, and Stefan Riezler. Leveraging implicit expert knowledge for non-circular machine learning in sepsis prediction. Artificial Intelligence in Medicine, 100:101725, 2019. doi: 10.1016/j.artmed.2019.101725.DOI

Paper details

vlnc:W_wkefyedh
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY-SA 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents