betaFeedback

WorksComputer Science

Which Model, For Which Variable? Per-Variable Forecast Combination in Sparse ICU Time Series

Abstract

Clinical forecasting papers typically report a single best architecture. We show that on multivariate ICU physiology this question conceals variable-specific differences: no architecture is best on every variable, and which model wins changes from variable to variable. Across five ICU databases (MIMIC-III, MIMIC-IV, eICU, HiRID, and UMM (University Medicine Mannheim), a German single-center ICU) we forecast 94 channels over a 24-hour horizon with a pool of 14 models, and combine them with weights fitted per variable. On test, the combination beats the best single model on all five databases by 6.1-11.0% mean squared error (mean 8.7%). The gain is concentrated where clinical data is thinnest: the sparsest third of channels improves by 11.9% and laboratory values by 7.5%, against 3.1% for the densest third. Rank-based harmonic weighting of the best five models per variable, with no fitted weights at all, recovers about four-fifths of the gain. Extending the pool afterwards with a per-channel gradient-boosted tree, ARIMA and ETS makes the tree the validation-selected best single model on all five databases; per-variable combination still improves on it by 2.9-5.9% (significant on four of five), while equal weighting of the same members does not. As on univariate benchmarks, learned combination beats simple averaging - but only once the weights are constrained. We also report which model to prefer when, and show that the ordering changes with database size.

Data and Code Availability All code for harmonization, training, evaluation and combination is publicly available at https://github.com/StatNLP/ML4H_2026_Per-Variable-Forecast-Combination. The four public databases (MIMIC-III, MIMIC-IV, eICU and HiRID) are available via PhysioNet under their respective data use agreements; UMM is a single-center clinical database that cannot be shared.

Institutional Review Board (IRB) While IRB approval is not required for the scientific usage of the four public databases, retrospective analysis of the UMM data was approved by the Medical Ethics Commission II of the Medical Faculty Mannheim, Heidelberg University, Germany, with waiver of patient consent (reference numbers 2016-800R-MA, 2016-840R-MA and 2023-851).

Introduction

A clinical time-series forecasting paper typically ends with a table in which one architecture has the lowest error, and that architecture is described as the best model for the task. This paper argues that on multivariate ICU physiology the single-best-model framing is suboptimal. The target is not one series but a 94-channel state vector mixing daily laboratory values, continuously sampled vital signs, medications that are administered, not measured, and derived scores. The observation process is extremely sparse — 8.8–18.6 % of (variable, hour) cells carry a measurement — and the sparsity is itself informative, because sicker patients are sampled more often. Furthermore, the channels are heterogeneous in a way univariate benchmarks are not: a creatinine trajectory and a noradrenaline infusion rate have almost nothing in common as forecasting problems. Asking which architecture is best therefore assumes one inductive bias dominates the whole panel. We show that fitting combination weights separately for each variable — a per-variable stacked generalization — recovers a consistent 6.1–11.0 % error reduction over the best single model on every database we examine, on a test set spent exactly once.

Contributions.

  1. Per-variable forecast combination for sparse clinical panels. We formulate stacking with variable-specific weights, shared across windows and forecast horizons rather than tied to a series, and give an efficient estimator that makes the full combiner ladder a seven-minute CPU job over cached predictions.

  1. A five-database held-out confirmation. Per-variable forecast combination improves over the best single model on all five databases (6.1–11.0 %), with the combiner chosen on validation before test was touched.

  1. Sparse channels gain most. Improvement tracks channel sparsity: averaged over the five databases the sparsest third of channels gains 11.9 % against 3.1 % for the densest. We also show that the metric conventionally used for such panels is dominated by near-unobserved channels, and report it with a minimum-support filter.

  1. Guidance on which model to use when, from per-channel win shares: no model is uniformly preferable, and the leader changes between databases. A dense autoregressive decoder is the most consistent choice at any size; the set-function encoder STraTS leads on HiRID and MIMIC-IV and wins fewest channels on the smallest and densest database, where an architecture built for irregular sampling has least to exploit; linear models are competitive only on the small databases; and the zero-shot foundation models are included for their decorrelation and not their accuracy. Adding a gradient-boosted tree changes the picture: it wins most cells on every database (§ 5.3), and per-variable combination still improves on it.

Related Work

Forecast combination. That combinations of forecasts outperform their members is one of the oldest results in the field ([2]), and the persistent finding that simple combinations beat estimated ones — the forecast combination puzzle — has survived decades of competition evidence ([25, 26]). Stacked generalization ([35]) and stacked regressions ([5]) established the modern framing, with the key practical requirement that the combiner be fitted on out-of-fold predictions; greedy ensemble selection ([6]) is the constrained alternative we also evaluate. The ambiguity decomposition ([24]) makes the mechanism exact. For a weighted average yˉ=∑mwmy^(m)\bar{y} = \sum_m w_m\hat{y}^{(m)} with ∑mwm=1\sum_m w_m = 1, under squared error,

(yˉ−y)2=∑mwm(y^(m)−y)2−∑mwm(y^(m)−yˉ)2,(\bar{y}-y)^2 = \sum_m w_m(\hat{y}^{(m)}-y)^2 - \sum_m w_m(\hat{y}^{(m)}-\bar{y})^2,

an identity, not a bound: ensemble error is mean member error minus the ambiguity, the weighted variance of the members about their own mean. Combination therefore pays exactly in proportion to member disagreement, and we report the ambiguity term directly (Figure F1). Perrone and Cooper [28] take the further step of choosing weights from the members’ error covariance and not their individual errors, showing that the resulting combination beats every member — the argument our per-channel Gram matrix implements. Hansen and Salamon [14] give the original variance-reduction result and Ganaie et al. [11] survey the deep-learning instantiation. Closest to us in setting, Schamoni et al. [31] ensemble neural networks for early sepsis prediction in the ICU, so disagreement-weighted combination is already validated on clinical time series; we extend it from a single target to a 94-channel panel with per-channel weights.

Figure F1. Per-(variable, hour) structure on UMM validation, the densest database and so the one where most channels clear the support filter. Top: which architecture wins each cell. Center: ambiguity share, the weighted spread of the members about their own mean under equal weights; it bounds the gain of the equal-weight average only, not of a fitted combiner. Bottom: mean pairwise residual correlation. Cells with fewer than 200 observations in either the fitting or the scoring half are grayed, so a winner picked from a handful of observations cannot be over-read. No architecture owns the grid: the winner changes across variables and, less often, along the horizon.

Stacking for time series. Recent work ([3]) has systematized stacking for forecasting at benchmark scale, establishing out-of-fold protocol requirements and comparing combiner families. Their conclusion that learned stacking beats both individual models and simple aggregation agrees with ours. Their evidence base is 50 univariate, general-purpose datasets ([1, 36]) with no clinical application, filtered so that enough data exists to train base and stacker models, and their unit of prediction is the series itself, so there is no feature axis along which weights could be tied. Our channels are 94 per window, 8.8–18.6 % observed, and neither independent nor comparably supported. Whether stacking survives clinical sparsity is therefore open. § 5.2 shows that it does.

Clinical time-series forecasting. Irregular sampling has motivated architectures built around observation triplets instead of dense grids (Tipirneni and Reddy, [34]), and the cross-database reproduction of Staniek et al. [33] sets our pool and its 4-hourly, 24-hour-horizon protocol. Zero-shot foundation models (Woo et al., [36]; Podest et al., [29]) are a natural addition and we include two; persistence remains the incumbent baseline (Haugsdal et al., [15]; Hewamalage et al., [16]).

Data and Setup

Databases

We use five ICU databases: MIMIC-III (Johnson et al., [22]), MIMIC-IV (Johnson et al., [21]), eICU (Pollard et al., [30]), HiRID (Hyland et al., [19]), and UMM (University Medicine Mannheim, Heidelberg University), a German single-center ICU whose data are not publicly available. All five are harmonized onto a shared 94-channel schema and gridded to at most one entry per (variable, hour) cell, with unrecorded cells marked unobserved in the mask; windows carry a 24-hour input and a 24-hour horizon and are drawn from the first seven ICU days at a 4-hour stride. Fixed at one seed, splits are 64/16/20 (train/validation/test) by patient (by admission for HiRID, where readmissions are not linkable), so no stay appears in two splits.

Table 1 makes the central difficulty explicit. Even the densest database leaves four of every five target cells unobserved, and the databases differ a lot in size: eICU has 17.2 times UMM’s stays. Sparsity determines where combination pays, and size determines whether the combiner can be fitted at all.

stayspatientswindowsdensitycharted
MIMIC-III28 51422 338431 11111.0%85
MIMIC-IV46 22035 706693 54811.4%89
eICU77 11563 8591 065 0288.8%79
HiRID8 6218 621136 68313.3%66
UMM4 4904 22897 06118.6%80

Table 1. Databases after harmonization. Stays and windows are totals over all splits; the 64/16/20 breakdown is in Appendix § 9. Density is the fraction of (variable, hour) target cells carrying an observation; charted is how many of the 94 channels the database ever records. Patients are unique individuals; HiRID releases one record per admission with no cross-admission linkage, so its stays and patients coincide by construction.

Evaluation metric

Write y^nhf\hat{y}_{nhf} for the prediction of window n∈{1..N}n \in\{1..N\} at horizon step h∈{1..24}h \in\{1..24\} for channel f∈{1..94}f \in\{1..94\}, ynhfy_{nhf} for the observed value, and mnhf∈{0,1}m_{nhf} \in\{0,1\} for the mask, which is 1 when that cell carries an observation. Let CC be the set of channels the database ever charts and ∣C∣|C| its size. All losses are masked to observed cells, and we report the per-channel mean

MSEper−var=1∣C∣∑f∈C∑n,hmnhf(y^nhf−ynhf)2∑n,hmnhf,\mathrm{MSE}_{\mathrm{per-var}} = \frac{1}{|C|}\sum_{f \in C}\frac{\sum_{n,h}m_{nhf}\left(\hat{y}_{nhf}-y_{nhf}\right)^2}{\sum_{n,h}m_{nhf}},

with nn running over windows and hh over the 24 horizon steps. On test, error grows 1.2–1.9× from h=1h=1 to $h=24 while the combination gain stays positive but falls by half or more (Appendix I). Each channel contributes its own mean squared error, and the channels are then averaged with equal weight. Values are z-scored per channel using training-split statistics, so an error of 1.0 is one training standard deviation of that channel and the numbers are comparable across channels.

We use squared error because the ambiguity decomposition that motivates combination (§ 2) is an identity only under squared error; it does not hold for MAE (mean absolute error), and probabilistic scores such as CRPS (continuous ranked probability score) would require the members to emit distributions, which several of ours do not. Weighting channels, not observations, suits a clinical panel — a rare but decision-relevant assay should not be drowned out by a continuously monitored vital. However, channels with a handful of observations can dominate a group mean. One HiRID medication channel with twelve observed cells supplies 41 % of its group’s mean. We therefore apply a minimum-support filter of 100 observations to group-level tables (Appendix § 14).

Model pool

The pool holds 14 models per database (full listing, Appendix B): two trivial baselines (NULL, all-zeros in normalized space, and LOCF, last-observation-carried-forward); a linear model, DLinear (Zeng et al., [38]); a direct multi-step dense model, Dense DMS ([33]); two recurrent models (GRU ([7]), LSTM ([17])); a state-space model, Mamba ([13]); a patch transformer, PatchTST ([27]); two long-horizon transformers, Informer ([39]) and Autoformer ([37]); an irregular-sampling clinical transformer, STraTS ([34]); an autoregressive dense decoder, DenseAR ([33]); and two zero-shot foundation models, Moirai ([36]) and TiRex-2 ([29]). Every learned member is tuned per database rather than sharing one configuration, and pool membership is decided on validation error under a single rule applied to every family; the selected configurations are listed in Appendix B.

Because the foundation models’ pretraining corpora are partially undisclosed, we cannot rule out prior exposure to our four public databases. We therefore do not interpret their absolute performance, including them solely for ensemble diversity as the pool’s least correlated learned members (Appendix § 15).

Training uses one fixed protocol for every model, schedule and database: early stopping on validation with patience 6, and learning-rate reduction on plateau with patience 3.

Even LOCF, one of the trivial baselines, has its place because it wins 2.5–3.4 % of well-supported test cells outright (Appendix § 14).

Base predictions are cached to disk once, which reduces the entire combiner ladder over five databases to a seven-minute CPU job, and no combiner ever reruns a network.

Method

Per-variable stacking

Let y^nhf(m)\hat{y}^{(m)}_{nhf} be model mm’s prediction for window nn, horizon step hh, channel ff, with observation mask mnhfm_{nhf}. We seek weights wfmw_{fm} — tied across windows and (in our main variant) across horizon steps, but free across channels — minimizing masked squared error.

The objective is a masked sum of squared errors, SSE, and because it is quadratic in ww it does not depend on the predictions individually, only on two summaries of them. Accumulating those summaries once lets every combiner below be fitted and scored without ever revisiting a prediction. The summaries are a per-channel Gram matrix GfG_f, holding how strongly each pair of models (m,m′)(m,m') agrees on channel ff, and a cross-term bfb_f, holding how strongly each model agrees with the truth:

Gfmm′=∑n,hmnhfy^nhf(m)y^nhf(m′),bfm=∑n,hmnhfy^nhf(m)ynhf,\begin{aligned} G_{fmm'} &= \sum_{n,h} m_{nhf}\hat{y}^{(m)}_{nhf}\hat{y}^{(m')}_{nhf},\\ b_{fm} &= \sum_{n,h}m_{nhf}\hat{y}^{(m)}_{nhf}y_{nhf}, \end{aligned}

GfG_f is the piece that heuristics lack: it records how strongly each pair of members agrees, so the learner can discount two models that make the same mistake and reward one that captures variance nobody else does — which is why the zero-shot foundation models earn weight despite middling accuracy. Ranking-based rules such as HARM-kk use each member’s error on its own and never the cross-model products. Then the error of any weight vector follows in closed form, SSE⁡f(w)=wTGfw−2wTbf+∑n,hmnhfynhf2\operatorname{SSE}_f(w)=w^\mathsf{T}G_fw-2w^\mathsf{T}b_f+\sum_{n,h}m_{nhf}y_{nhf}^{2}, so a combiner is fitted by minimizing a quadratic in M=14M=14 unknowns per channel, one weight per pool member, rather than by any other pass over data. The last term does not depend on ww and only fixes the offset. Both statistics are plain sums over windows, which also makes them additive: partial sums over disjoint groups of stays can be recombined by addition, and that is what makes the bootstrap and the randomization test in Appendix E affordable.

Combiners

We evaluate a ladder of increasing flexibility (formal definitions, Appendix § 11): BEST-SINGLE, the validation-best member, which every percentage below is measured against; per-variable SELECT, which uses only the best member on each channel; unweighted MEAN and G-EQUAL-3; inverse-MSE weighting; rank-based harmonic weighting over the top kk members (w∝1,12,…,1kw \propto1,\frac{1}{2},\ldots,\frac{1}{k}) HARM-kk following Cormack et al. [9] unsmoothed and on models rather than ranked lists on retrieved items; per-variable RIDGE, the unconstrained least-squares solution wf=(Gf+λI)−1bfw_f=(G_f+\lambda I)^{-1}b_f, free to assign negative weights; simplex-constrained least squares, the same objective restricted to w≥0w \ge0, ∑mwm=1\sum_m w_m=1, so the result is a weighted average whose error on any single channel is bounded by that channel’s worst member (projection following Duchi et al., [10]); GREEDY forward selection with replacement (Caruana et al., [6]); and a shrunk simplex variant that interpolates each channel’s fitted solution toward the harmonic ranking prior with strength nf/(nf+κ)n_f/(n_f+\kappa): a dense channel keeps almost all of its fitted weights, while a channel with a handful of observations falls back to HARM-5 on that channel, and a channel with no support at all falls back to a single hour-pooled weight vector estimated from every channel at once. The shrunk simplex variant, at κ=500\kappa= 500, is the combiner we fixed on validation before opening test and is what ENSEMBLE denotes throughout. Shrinking toward the ranking prior rather than toward uniform weights is deliberate: the prior is already near-optimal (§ 5.2), so the fallback costs little where it engages. SELECT and RIDGE also have hour-resolved counterparts (SEL-hh, RIDGE-hh), in which ww varies with hh.

How each combiner is fitted. None of the combiners needs backpropagation or a training loop. RIDGE has the closed form wf=(Gf+λI)−1bfw_f = (G_f + \lambda I)^{-1}b_f with λ=1\lambda= 1 on the unnormalized statistics. The regularizer is not optional: GfG_f is a Gram matrix of highly collinear predictors (mean pairwise residual correlation 0.82–0.87), and on a thin channel it is rank-deficient outright, with fewer observed cells than members. Adding λI\lambda I makes it positive definite and the inverse well posed (Hoerl and Kennard, [18]). The simplex-constrained variants, ENSEMBLE among them, are convex programs, so projected gradient descent on the quadratic above reaches a global optimum; collinear members can leave it nonunique, which is a reason not to over-read an individual weight. All of them read only GfG_f and bfb_f, so the entire ladder costs one pass over the cached statistics.

What a fitted combination looks like. The weights are interpretable, and they move with the character of the channel. On eICU, heart rate — observed in 18.7 of 24 hours — spreads across neural members and gives no weight to copying the last value; its five largest weights are 0.27 DenseAR, 0.18 Informer, 0.13 STraTS, 0.12 LSTM, 0.12 GRU. Lactate, 150 times sparser, puts LOCF first outright (0.29 LOCF, 0.24 STraTS, 0.16 DenseAR), and noradrenaline — a treatment decision rather than a physiological signal — splits between copying forward and predicting nothing (0.26 LOCF, 0.25 STraTS, 0.21 Dense DMS, 0.09 null). The combiner is not only reweighting networks, but deciding, per channel, how much of the forecast should be a forecast at all.

Protocol. Combiner weights are never fitted on data used to score them: during development on a stay-grouped half of validation, scored on the disjoint half; for the final confirmation, run once, on all of validation and scored on all of test. Test was never used to select a combiner, a pool member or a hyperparameter (Appendix B).

What “per variable” does and does not mean. The base models are multivariate in the input and joint in the output. Each receives the full 94-channel window with its observation mask and emits all 94 channels, so channels inform one another inside a model. “Per variable” refers only to the combination layer, which fits one weight vector per channel over the pooled member forecasts. The loss is masked, so a channel contributes only where a value was recorded, and within a (variable, hour) cell the target is the last recorded value in the bin. Feeding the iterative decoders the combined forecast during generation is no deployable alternative: it helps only when injected at the cells where the target will be observed, which leaks the future observation mask (Appendix § 19).

Results

Main result

As detailed in Table 2, per-variable combination beats the best single model on all five databases, by 6.1–11.0% (8.7% mean), on held-out data, with the combiner fixed in advance. Committing to the best model per channel also beats the best single model, but recovers under half as much (−3.8% mean), and a rank-based harmonic blend with no fitted weights recovers most of the rest (−7.2%). So the gain comes from blending several members rather than committing to one, and most of it is available without fitting anything. Every number in the last two columns is significant at p<10−4p < 10^{-4} under a paired randomization test over stays (Appendix E); the only combiner anywhere that fails significance is the unweighted mean, on MIMIC-IV. The full ladder is in Appendix § 12.

BESTSELECTHARM-5ENS.
UMM0.60360.57400.55050.5375
—−4.9%−8.8%−11.0%
HiRID0.48650.45090.44500.4378
—−7.3%−8.5%−10.0%
MIMIC-III0.46380.44600.42440.4184
—−3.8%−8.5%−9.8%
MIMIC-IV0.44310.43660.42110.4143
—−1.5%−5.0%−6.5%
eICU0.41570.40940.39490.3904
—−1.5%−5.0%−6.1%
mean0.48250.46340.44720.4397
—−3.8%−7.2%−8.7%

Table 2. Held-out test error (MSE-per-var, lower is better). BEST-SINGLE is the validation-chosen best member; SELECT commits to the best member per channel; HARM-5 is the rank-based harmonic blend of the best five per channel; ENSEMBLE is the combiner chosen on validation. Weights fitted on all of validation, scored on all of test, spent once. Percentages are against BEST-SINGLE, which is DenseAR on all five databases. All charted channels, without the support filter of the group-level tables (Appendix § 16).

Further findings

  1. Per-channel selection does the work; the weight profile adds little. Two things happen when the best five members per channel are blended harmonically: the members are chosen per channel, and they are weighted unequally. Crossing the two isolates them. Taking the five best members pool-wide and weighting them equally recovers −4.89% (G-EQUAL-5); choosing the five best per channel and still weighting them equally recovers −6.97% (EQUAL-5). Membership is therefore worth about

2.1 pp. Applying the harmonic profile to the per-channel members adds a further 0.18 pp (−7.15 %, HARM-5), while applying it to the pool-wide members costs 0.56 pp (G-HARM-5). The profile pays only when rank 1 is that channel’s best model, not merely the pool’s, and k=3k = 3 behaves the same way (Table D1). The gain comes from ranking members per channel, and the decaying weights are close to free either way. A freely fitted rank profile confirms this: it reproduces the harmonic one (correlation 0.99, maximum deviation 0.045) and leaves accuracy unchanged (Appendix J).

combinerUMM devUMM testHiRID devHiRID testMIMIC-III devMIMIC-III testMIMIC-IV devMIMIC-IV testeICU deveICU testmean devmean test
GREEDY-8.85-10.69-3.21-12.40-5.39-9.87-5.57-6.57-4.89-6.34-5.58-9.17
simplex-10.04-10.69-6.80-12.42-5.33-9.83-5.49-6.54-4.96-6.30-6.52-9.16
shrunk100-10.79-10.94-7.97-10.29-5.47-9.83-5.52-6.54-4.97-6.10-6.94-8.74
ENS.-10.67-10.95-8.00-10.01-5.61-9.79-5.57-6.51-5.00-6.09-6.97-8.67
HARM-5-8.02-8.79-4.01-8.52-5.12-8.48-4.70-4.97-4.12-5.01-5.19-7.15
g-simplex-10.83-9.95-6.78-7.91-6.24-7.51-5.18-5.06-4.94-5.02-6.79-7.09
equal-5-8.67-8.98-3.74-7.36-5.05-8.85-4.08-5.05-3.75-4.62-5.06-6.97
HARM-3-7.13-8.10-3.17-8.08-4.53-7.57-4.41-4.14-3.87-4.46-4.62-6.47
equal-3-7.93-8.58-3.05-6.96-4.88-7.95-4.37-3.99-3.79-4.21-4.80-6.34
inv-MSE-9.74-9.18-5.63-7.80-4.69-7.34-2.81-3.39-3.72-3.67-5.32-6.28
ridge-3.57-7.77-3.591.7714.49-7.86-4.83-6.39-3.35-4.30-0.17-4.91
g-equal-5-7.30-8.14-5.57-6.36-4.14-4.35-2.97-2.72-1.92-2.87-4.38-4.89
mean-8.10-8.01-4.50-6.26-3.63-5.86-1.20-1.07-3.14-2.67-4.11-4.77
g-harm-5-7.11-7.44-4.68-4.57-3.95-3.81-3.21-3.04-1.91-2.80-4.17-4.33
BMAₛₜₐᵧₛ-4.72-5.28-4.65-7.91-0.73-3.93-1.17-1.57-1.32-1.57-2.52-4.05
SEL-h-2.32-4.640.33-7.26-0.81-5.34-1.06-0.80-1.61-1.81-1.09-3.97
Hedge-5.37-5.51-4.71-5.38-1.62-4.25-1.62-1.80-1.32-2.11-2.93-3.81
SELECT-2.45-4.89-0.64-7.30-0.56-3.84-1.04-1.48-1.45-1.52-1.23-3.81
g-equal-3-6.04-8.05-4.02-2.34-3.72-3.08-2.68-2.68-1.66-2.47-3.62-3.72
ridge-h-2.73-6.81-2.693.3315.66-7.27-2.80-4.47-2.23-3.381.04-3.72
g-harm-3-6.61-6.68-3.49-2.15-3.36-2.84-2.72-2.71-1.58-2.39-3.55-3.35
BMA-4.93-4.98-4.56-4.74-1.33-3.95-1.07-1.49-1.38-1.54-2.65-3.34
BEST0.000.000.000.000.000.000.000.000.000.000.000.00

Table D1. Combiner ladder, as percentage change against the best single model (negative is better). Dev fits on a stay-grouped half of validation and scores on the disjoint half; test fits on all of validation and scores on test, so the pair differs both in how much data the weights are fitted on and in the population they are scored on. Rows are ordered by the test mean. Ensemble is simplex-shrunk500, chosen on the dev columns before test was opened.

For practitioners without a tuning budget, a harmonic rank blend is the recommendation: HARM-5 recovers 76–87 % of the ensemble’s gain with one ranking pass per channel and nothing fitted, equal weights on the same five cost only 0.2 pp, and extending the blend to the whole pool (HARM-14) does slightly better in development and on test (−6.30 and −7.89 % against −5.19 and −7.15 %, Appendix J).

2. Per-variable weights beat shared weights. An ensemble can beat the best single model even

when one weight vector would serve the whole panel, so we fit that control: a single simplex-constrained vector on the same pool, the same fitting data, and the same macro objective, with each channel’s statistics normalized by its own support so that dense channels do not dominate the loss. On test, per-variable weighting wins on every database; in development the shared vector is ahead on UMM and MIMIC-III. The unshrunk per-variable fit, SIMPLEX, reaches −9.16 % against −7.09 % for G-SIMPLEX, the single shared vector; and ENSEMBLE, the shrunk estimator we report throughout, reaches −8.67 %. The margin runs from 0.74 pp on UMM to 4.51 pp on HiRID (Table D1). The advantage is smallest where the fit is most data-starved, which is finding 6 seen from the other side. A shared vector is worth about as much as a fixed rank rule with per-channel members (−7.09 against −7.15 % for HARM-5): fitting weights globally buys no more than ranking members locally.

3. The forecast combination puzzle ([25, 26]) does not reproduce here. The puzzle holds that simple combinations beat estimated ones; our fitted combiner beats every fixed rule (−8.67 against −7.89 % for HARM-14, the best fixed rule, Appendix J). Bosch et al. [3] report the same direction at benchmark scale and explicitly describe their findings as challenging the puzzle. Two independent settings therefore agree that learned combination wins once there is enough data to fit it, and Smith and Wallis [32] and Claeskens et al. [8] trace the puzzle to the error of estimating the weights, which makes it a question of sample size per parameter: per-channel fitting has far more effective observations per weight than the series-level setting in which the puzzle was established. What our data adds is the boundary condition — how the weights are estimated decides whether that advantage survives. Unconstrained ridge is the clearest case: its test mean is −4.91 % against −8.67 % for ENSEMBLE, and it fails outright on a different database in each protocol, +14.49 % on MIMIC-III during development and +1.77 % on HiRID at test, both worse than a single model. The instability is a property of unconstrained estimation on thin channels, not of one cohort. Here the constraint and not the learning keeps the fit reliable.

4. Hard selection under-performs the soft combiners, and it is weak because it is over-fitted. Committing to the single best model on each channel beats the best single model on all five databases, but by less than any soft combiner (−1.48 to −7.30% against −6.09 to −10.95% for ENSEMBLE), and it is the only combiner whose advantage is ever marginal: on eICU and UMM its interval nearly touches zero (p=0.04p = 0.04 on both, Appendix E). The weakness is a small-sample effect. Selecting on half of validation and scoring on the disjoint half recovers only −0.56 to −2.45%, against −1.48 to −7.30% when the selection sees all of validation — HiRID moves from −0.64 to −7.30 — though the two are scored on different populations, so this is suggestive rather than an isolated measurement of fitting size. An argmin over 14 noisy per-channel estimates is itself high-variance, and the modal winner holds only 0.58–0.67 of a channel’s 24 hours, with three to four distinct winners typical (top panel of Appendix Figure F1).

5. Performance gains in sparse channels. Averaged over the five databases and measured against the best member within each system, the sparsest third of channels gains −11.9% against −3.1% for the densest third; laboratory values gain −7.5% against −3.7% for vital signs; and organ-failure channels gain −7.6%. Per-system tables for every database are in Appendix § 16. Sparse channels are where individual models are least reliable and therefore most mutually complementary. All per-variable means require at least 100 observations in both the fitting and the scoring half, which removes one channel each from UMM (total protein), HiRID (vasopressin), MIMIC-III (absolute lymphocytes) and eICU (mechanical ventilation), and none from MIMIC-IV.

6. Shrinkage is a small-sample correction, and the final protocol removes the small sample. Repeating the development protocol over five stay-grouped partitions, the gain is tight on the three large databases (sd 0.38–0.73 pp) and not on the two small ones: HiRID sd 2.48 pp, UMM sd 4.18 pp, with UMM ranging from −11.15% to −1.17% on the choice of half alone. That spread is the direct evidence that the per-channel fits are unstable at this size, and interpolating toward the ranking prior is the correction: on validation it wins on all five, tracking how many thin channels a database has (HiRID +1.20 pp down to eICU +0.04 pp). The final fit uses the whole validation split, twice the stays, which removes the instability and with it the reason for the correction. On test the effect is confined to one database. Shrinkage still helps on UMM (−0.26 pp)

and costs almost nothing on MIMIC-III, MIMIC-IV and eICU (+0.04, +0.03, +0.21 pp), but on HiRID it costs 2.41 pp — the whole of the 0.49 pp mean difference, since excluding HiRID the shrunk and unshrunk fits are within 0.01 pp. HiRID is the one database with both a poor prior (HARM-5 is 3.90 pp worse than its fitted simplex, against 1.29–1.90 pp elsewhere) and enough validation stays (1398) to fit without help. GREEDY selection, a quantized approximation to the same simplex objective, tracks the unshrunk fit to within 0.04 pp on test (Table D1). We report ENSEMBLE because it was fixed before test was opened.

A stronger single model

We extend the pool of § 3.3 by a per-channel gradient-boosted tree, LightGBM (Ke et al., [23]), on hand-built window summaries, together with ARIMA (Box and Jenkins, [4]) and ETS (Gardner and McKenzie, [12]; Hyndman and Khandakar, [20]), and score the extended pools once. These runs are post-hoc: the original evaluation was fixed before the test split was used for the first time, and fixing a pool in advance does not make it independent of the earlier test-set feedback, so we report the two evaluations separately.

The tree is fitted with a fixed 200 trees per channel, on seven summaries of that channel’s own window, one of which is the number of hours since the last real observation. The neural members see the observation mask but never that recency as an explicit number, which is one reason the tree does comparatively well on rarely recorded channels. It has the lowest validation error of any member on all five databases, beats DenseAR, the strongest neural member of the original pool, by 1.7–11.1%, and wins 57–76% of well-supported test cells (Table L2). Table L3 therefore measures every combiner against LightGBM.

Table L2: Share of well-supported (variable, hour) test cells won by each member of the extended 15-member pool (%). This pool excludes Moirai and TiRex-2, so the shares are not comparable with Table 4, which covers the original 14-member pool. Columns and the mean each sum to 100%.

memberUMMHiRIDM-IIIM-IVeICUmean
LightGBM75.774.561.956.858.565.5
DenseAR1.61.713.66.016.47.9
STraTS1.46.45.413.36.56.6
PatchTST3.22.15.44.95.24.2
DLinear8.18.50.81.20.73.8
GRU1.30.90.87.12.32.5
Informer3.11.21.83.61.12.2
Dense DMS2.20.52.90.94.12.1
locf0.91.93.11.82.82.1
null0.10.02.81.80.11.0
LSTM1.40.40.10.61.20.8
ARIMA0.30.70.61.20.10.6
Autoformer0.30.40.30.10.80.4
ETS0.10.70.40.50.10.4
Mamba0.30.00.00.10.20.1

Table L2.

UMMHiRIDMIMIC-IIIMIMIC-IVeICU
random convex weights−6.8 [-7.6, −5.9]−4.7 [-6.3, −2.8]−4.4 [-5.3, −3.5]+0.5 [-0.7, +2.1]−1.1 [-1.8, −0.4]
4 DenseAR seeds vs. best seed−3.9−4.9−3.4−2.6−1.1
4 STraTS seeds vs. best seed−5.1−2.0−1.9−8.2−6.1
diverse pool vs. DenseAR seeds−12.8−13.6−4.7−4.1−4.2
diverse pool vs. STraTS seeds−17.1−21.8−8.3−20.6−14.6

Table L3. Controls behind the Discussion (§ 6), in percent (negative is better). First row: 1 000 random convex weight vectors per database over the original 14-member pool, on test, against best-single; brackets give the 5th and 95th percentile. Remaining rows, development protocol: ensemble fitted on four seeds of one architecture, against that architecture’s best seed; and the error of the stacked diverse 14-member pool relative to each seed stack.

Per-variable stacking stays 2.9–5.9% ahead of the tree, significant on four databases. On HiRID the randomization test does not reach significance (p=0.11p = 0.11), so we treat HiRID as inconclusive. Equal weighting of the extended pool no longer beats the tree significantly anywhere, so the fitted per-variable weights carry the gain. Repeating the original evaluation without Moirai and TiRex-2 changes the test gain by at most 0.15 points, so the result does not rest on their undisclosed pretraining.

Table L1 gives every member and combiner. Equal weighting of the extended pool is worse than LightGBM alone, while the stacker is 5.3% better than it: averaging a pool with one strong and several weaker members is not enough, the weights have to be fitted per variable. Informer and DenseAR are effectively tied over the five databases (+5.7 against +6.1%), DenseAR leading on HiRID, MIMIC-IV and eICU and Informer on UMM and MIMIC-III. The tree leads on laboratory and rarely recorded channels and is behind the best neural member on medication by 2.4–9.6%, where DLinear wins on UMM and HiRID and PatchTST, GRU and Informer on the larger databases.

Table L1: Test MSE per variable for every member and combiner of the extended 15-member pool (the original pool without Moirai and TiRex-2, plus LightGBM, ARIMA and ETS). The last column is the mean over databases of the relative change against LightGBM, not a mean of the absolute values: absolute MSE differs in scale between databases, so averaging it would weight UMM most. harm-15 is the harmonic blend of all 15 members. Only channels with at least 100 observations, as in Appendix § 16; Table 3 uses all charted channels.

UMMHiRIDM-IIIM-IVeICUmean Δ
stacker (shrunk simplex)0.5150.4180.3970.4100.390−5.3
harm-150.5240.4270.4030.4170.394−3.8
harm-50.5230.4230.4020.4170.395−4.0
per-variable select0.5390.4310.4150.4290.408−1.2
equal weights0.5440.4540.4220.4360.407+0.6
LightGBM0.5300.4500.4200.4360.4110.0
Informer0.5770.4880.4440.4490.422+5.7
DenseAR0.5960.4870.4470.4430.418+6.1
GRU0.5910.4910.4500.4480.425+6.8
Dense DMS0.5800.4970.4550.4530.435+7.6
STraTS0.6010.5400.4710.4700.447+12.4
LSTM0.6490.5470.4730.4570.427+13.0
PatchTST0.6170.5130.4870.5300.462+16.1
DLinear0.6050.4950.5100.5330.483+17.1
Mamba0.6640.5680.4850.4970.460+18.6
Autoformer0.6610.5590.5360.5540.506+25.4
locf0.7020.5910.5620.6240.526+33.8
ARIMA0.7420.6650.6090.6280.564+42.8
ETS0.7770.6750.6330.7000.605+51.1
null1.0601.0100.9330.9860.995+123.0

Table L1.

Which model wins where

Counting well-supported (variable, hour) test cells won by each family (Table 4) shows that the leader differs by database. Figure 1 shows each member’s rank on every variable.

Figure 1. The 14 pool members ×\times the 40 hardest variables and, to the right of the rule, the 10 easiest, ordered by mean MSE across the pool on validation, averaged over the databases that chart each channel. Cells are colored by each model’s rank within that variable; we color by rank because absolute MSE is dominated by how noisy each variable is, which redraws the column structure and hides which model is good where. Top marginal: per-variable difficulty. Right marginal: each model’s mean MSE over all 94 variables, which sets the row order.

familyUMMHiRIDM-IIIM-IVeICU
2.9k5.5k18k30k50k
STraTS12.224.430.040.035.5
DenseAR22.821.134.220.635.8
Informer21.29.37.17.52.9
GRU6.211.12.614.13.9
Dense DMS9.18.77.93.07.3
DLinear13.916.51.71.10.8
PatchTST5.72.97.15.25.6
LOCF + null3.42.56.24.53.1
Moirai + TiRex-23.22.42.52.31.5
LSTM1.70.50.40.92.7
Autoformer0.30.60.30.10.7
Mamba0.30.10.00.70.2

Table 4. Share of well-supported (variable, hour) test cells won by each family (%). Column subheadings give training stays. Shares sum to 100 % per column; every member is listed in Appendix F.

STraTS wins 12% of cells on UMM and 40% on MIMIC-IV; DLinear moves the opposite way, 13.9% against 0.8%; DenseAR is the most consistent member at 20.6–35.8%. These patterns are associated with database size, but size co-varies with case mix, measurement practice and tuning outcome, so they

do not show that an architecture needs more data. On channels that behave like random walks over 24 hours, LOCF and the training mean are the right model and win 2.5–6.2% of cells. Members differ mainly by training regime rather than by architecture family: hyperparameter variants of one decoder correlate at ρ≥0.965\rho\ge0.965, transformer variants span the whole cross-family range, and the two zero-shot models are the least correlated learned members (Appendix G).

Discussion

Most of the improvement comes from choosing members per variable rather than from fitting their weights precisely. A fixed rank-based blend of all fourteen members recovers about 90 % of the stacker’s gain, and fitted weights add a further 0.3–1.3 points. Fitting matters more as data grow. Random convex weights still beat the best single model on four databases, by 1.1–6.8 % on average, but not on MIMIC-IV (+0.5 %), where the fitted stacker gains 6.5 % (Table L3).

The gain needs different models. Stacking four seeds of a single architecture gains 1–8 % over its best seed, while the stacked diverse pool has a 4–14 % lower error than the stacked DenseAR seeds (Table L3). Random subsets of the pool recover about nine-tenths of the gain with four to six members (Figure L1); the only subsets that lose to the best single model are individual two-member draws, so pool spread matters more than pool size.

Random subsets of the 14-member pool

Figure L1. Random subsets of the 14-member pool, five draws per size, scored with the shrunk-simplex combiner against the full pool’s best single model (negative is better). Each point is the mean over the five draws; the rightmost point at 14 is the full pool, not a subset. Development protocol: fitted on one stay-grouped half of validation and scored on the other, so these are not test gains. Four to six members recover about 90 % of the full pool’s gain and eight recover 97 %. Only 4 of the 150 subsets (six sizes from 2 to 12, five draws each, five databases) are worse than the best single model, and all four are individual draws at k=2k = 2, which the means hide. The subsets are random rather than chosen on validation, so a guided choice would likely need fewer.

Adding a gradient-boosted tree raises the single-model baseline substantially and the combination still improves on it. Which model is best stays a per-variable question, and it is not a contest between the tree and the neural members: DLinear wins about 8 % of cells on the two small databases and the medication channels there, LOCF wins 1–3 %, and the all-zeros baseline wins 2.8 % on MIMIC-III in the extended pool (Table L2).

The target is the value a clinician recorded, under the measurement practice of each hospital. Sampling is informative, since a laboratory test is often ordered because deterioration is suspected, so gains on sparse channels are gains on a benchmark and not on unobserved physiology. How forecast error propagates into SOFA and SAPS-II is beyond the scope of this paper. Each database is evaluated within itself, so the five sets are replications of one procedure. Validation is used twice, to select base checkpoints and to fit the combiner, which makes development numbers mildly optimistic; the held-out result is unaffected because the combiner was fixed before test was used.

Conclusion

Forecast combination is the more durable finding of this study. Which architecture leads changed completely when we added a gradient-boosted tree, and it changes from variable to variable within every database, but combining several models per variable helped on all five databases, against both the original pool’s best member and the much stronger tree. A ranking of architectures is therefore the less transferable result to report, and per-variable combination is cheap enough to be a default rather than a contribution: member predictions are cached once, after which the entire combiner ladder over five databases is a seven-minute CPU job, and random subsets of four to six members recover about nine-tenths of the gain (Figure L1). We would report per-variable combination as a baseline in clinical forecasting work, in the same way a no-change predictor is reported.

Acknowledgments

The authors acknowledge support by the state of Baden-Württemberg through bwHPC and the German Research Foundation (DFG) through grant INST 35/1597-1 FUGG.

Marius Fracarolli is funded by the Helmholtz Information & Data Science School for Health (HIDSS4Health), Karlsruhe/Heidelberg, Germany.

We thank the clinical staff of the intensive care unit at University Medicine Mannheim for the records analyzed here, and the PhysioNet maintainers for curating the four public databases.

Databases

Sizes are in Table 1. Charted counts the channels a database records at least once inside a 24-hour target window, and is the denominator ∣C∣|C| of the metric; never-recorded channels are excluded from every per-variable mean. It differs by at most one from the non-dash entries of Table A2, which counts a channel recorded anywhere in a stay — a channel seen only outside the first seven days, or only in the input half of a window, appears in one and not the other. Density is the fraction of (variable, hour) target cells carrying an observation, over the full training split.

channelUMMHiRIDM-IIIM-IVeICUchannelUMMHiRIDM-IIIM-IVeICU
hr99.9100.099.499.998.7peep74.266.344.739.831.3
spo299.8100.099.399.997.7alp82.636.936.846.245.1
glucose99.799.399.098.696.5tidal_volume70.966.140.740.126.2
potassium99.898.599.198.695.3albumin83.014.832.631.750.7
sodium99.898.499.098.694.9neutrophils_pct38.159.127.025.154.1
hemoglobin99.897.598.698.493.4pt——78.781.634.5
wbc98.397.598.898.492.9amv71.9—44.440.015.6
creatinine98.094.999.098.694.7svo277.333.730.818.6—
platelets98.297.198.898.492.3lymphocytes_pct38.1—27.125.160.0
mcv98.397.598.698.490.6monocytes_pct38.1—27.025.159.6
mchc98.397.298.698.490.6eosinophils_pct38.1—27.025.157.5
hco397.191.198.998.693.8ldh82.9—28.433.9—
chloride97.189.598.998.694.8basophils_pct38.1—27.025.154.4
mch98.397.598.698.486.3weight—5.959.175.7—
bun98.185.199.098.694.7mechanical_vent.76.1—17.836.6—
rr71.2100.099.499.892.9fibrinogen69.513.713.715.85.0
temp_core99.365.299.299.395.9propofol18.557.410.125.44.7
urine_output99.598.693.996.667.9insulin50.347.24.58.54.4
gcs_eye65.2100.099.499.964.9norepinephrine42.437.96.120.06.1
gcs_motor64.8100.099.499.864.9lipase70.79.317.19.8—
gcs_verbal64.6100.099.499.964.9crp97.9—1.03.1—
magnesium73.085.098.098.267.7etco29.367.4—15.14.6
phosphate71.384.694.397.649.4crr——2.288.7—
hematocrit98.3—98.998.493.5midazolam34.439.16.78.41.9
inr97.866.878.781.635.4cardiac_output55.425.0—2.11.9
ph_arterial97.182.063.957.034.7lymphocytes_abs—59.00.313.7—
sbp_noninvasive23.845.791.495.176.3mpv————63.9
dbp_noninvasive23.845.791.495.176.3pct58.24.1———
mbp_noninvasive23.745.791.395.276.3stroke_volume55.5——1.6—
base_excess97.191.158.150.929.8protein_total1.6—2.22.145.7
mbp_invasive96.598.759.143.127.0fentanyl1.241.32.71.73.8
dbp_invasive96.598.759.143.026.9heparin22.8—5.414.52.3
sbp_invasive96.598.759.143.026.9troponin_i31.9—1.6—9.3
pao297.481.958.150.935.3troponin_t—30.6—11.69.3
paco297.481.858.150.935.0bilirubin_direct5.09.86.16.311.5
calcium_total18.214.193.697.592.4creatinine_urine——14.311.83.0
fio271.866.158.251.953.4dobutamine6.418.50.41.71.0
ptt97.622.078.280.822.1amiodarone11.5—3.26.12.1
lactate99.891.839.545.817.9dexmedetomidine6.7—1.911.22.2
rbc——98.698.492.7calcium_corrected16.7————
rdw——98.698.487.1epinephrine0.98.40.92.90.5
calcium_free99.890.649.042.6—morphine3.06.01.32.0—
alt81.169.937.446.345.4bilirubin_indirect——5.86.0—
sao297.390.836.626.228.8furosemide0.5—3.13.50.5
ast82.364.337.447.045.9dopamine0.1—1.21.80.9
anion_gap——98.498.675.0milrinone——1.32.20.3
bilirubin_total88.253.738.446.644.7vasopressin—0.1——1.6

Table A2. Percentage of ICU stays in which each channel is recorded at least once. A dash marks a channel the database never records. Channels are ordered by mean coverage. Coverage spans the full range within every database, which is why all per-variable means here carry a ≥100\ge100-observation filter; the same data is plotted in Figure A1.

Figure A1. Per-channel stay coverage. The block of channels every database records is small; most of the panel is local practice.

Windows are drawn from the first seven ICU days at a 4-hour stride, each with a 24-hour input and a 24-hour horizon, giving up to 31 windows per stay. Splits are 64 train /16 validation/20 test by patient at one fixed seed; Table A1 gives the resulting sizes.

train staystrain windowsval staysval windowstest staystest windowsobs./ window
MIMIC-III18 242274 1954 56869 3515 70487 565249
MIMIC-IV29 604444 8707 382111 5679 234137 111257
eICU49 533685 13412 197168 64315 385211 251200
HiRID5 54388 4071 39822 1431 68026 133304
UMM2 88962 46671015 05489119 541425

Table A1. Database sizes by split, and the mean number of observed (variable, hour) cells per 24-hour target window out of 24×94=225624 \times94 = 2256 possible.

The last column is the mean number of observed (variable, hour) cells per 24-hour target window, out of 24×94=225624 \times94 = 2256 possible. Even UMM, the densest, observes under a fifth.

Variable systems. Results below are reported over the catalog grouping (lab, vital, medication, ventilation), over the organ-failure channels (sofa-relevant), and over observation-frequency tertiles computed within each database (freq:low/mid/high). Group means use only channels with at least 100 observations: one HiRID medication channel with twelve observed cells otherwise supplied 41% of that group’s mean.

Per-channel coverage. Table A2 gives the percentage of stays in which each channel is recorded at least once. Coverage varies with local practice, not data volume: albumin appears in 83% of UMM stays and 14.8% of HiRID stays. Hence the ≥100\ge100-observation filter on the group-level tables.

Base pool

14 members per database, each tuned per database, selected on validation error under one rule applied to every family, and trained on the training split alone. The two foundation models are used zero-shot: public checkpoints, no fitting to these databases. Table B1 lists the selected configurations by family. Table B2 gives the same selection in full: for every (database, member) pair, the checkpoint whose predictions were cached, its configuration, its seed, and the number of stored checkpoints in that family on that database. The last column is why we report a selected configuration rather than the lowest error found during search: the number of configurations differs between families by more than an order of magnitude, so a minimum over more attempts is optimistically biased. On UMM, DenseAR has 290 stored configurations against Autoformer’s 17. LOCF and NULL have no fitted hyperparameters, and the two zero-shot foundation models have no fitted configuration at all.

familyUMMHiRIDMIMIC-IIIMIMIC-IVeICU
Autoformerms2hp_d512_L2hp_d128_L2hp_d512_L1hp_d512_L3
DLinearmsmshp_k21_individualms2ms2
Dense DMShp_d256_L2msmshp_d256_L1ms
DenseARdefaultdefaultdefaultpteacherctrldefault
GRUgrugrumsmsms
Informerhp_d256_L2hp_d512_L3hp_d256_L2hp_d128_L1hp_d256_L1
LOCFpersistencepersistencepersistencepersistencepersistence
LSTMlstmmsmsmsms
Mambamsmmsmsm
Moirai—————
Nullnullnullnullnullnull
PatchTST4h.d256_L2_lr3e44h.d256_L2_lr1e34h.d128_L3_lr1e34h.d128_L3_lr3e44h.d128_L2_lr3e4
STraTSd128_l1_lr1e-3_d0005d64_l1_lr5e-4_d0005msd128_l1_lr1e-3d128_l2_lr1e-3_d0005
TiRex-2—————

Table B1. Selected configuration per architecture family and database. Names are stripped of the database, the family prefix and the run hash; default means the family’s standard configuration was selected.

memberrunddlayersdropoutlrseedckpts
UMM
Autoformerms_s45612810.10.000345617
DLinearms_s12312820.20.000312316
Dense DMShp_d256_L2_s12325620.10.000312317
DenseARhpsf_6a82f5b05121/10.00.000142290
GRUhps_b7eec8ac51210.10.00034245
Informerhp_d256_L2_s12325620.10.000312358
LOCFpersistence1
LSTMhps_01f6488225610.10.00034245
Mambams_s45638410.10.000345644
PatchTST4h_d256_L2_dr00_lr3e4_pl4st2_s4225620.00.000342100
STraTSd128_l1_lr1e-3_do00512810.050.00142128
null—1
HiRID
Autoformerhp_d512_L2_s12351220.10.000312317
DLinearms_s12312820.10.000312316
Dense DMSms_s133712820.10.0003133717
DenseARhpsf_f4e3a6c72561/20.00.000342280
GRUhps_3f4077c051210.10.00034245
Informerhp_d512_L3_s12351230.10.000312358
LOCFpersistence1
LSTMms_s133725610.10.0003133745
Mambahpsm_2c6df0cb51210.10.00014244
PatchTST4h_d256_L2_dr00_lr1e3_pl4st2_s4225620.00.00142100
STraTSd64_l1_lr5e-4_do0056410.050.000542128
null—1
MIMIC-III
Autoformerhp_d128_L2_s12312820.10.000312317
DLinearhp_k21_individual_s12312820.10.000312316
Dense DMSms_s12312820.10.000312317
DenseARs4565123/20.00.0001456273
GRUms_s133738410.20.0003133745
Informerhp_d256_L2_s12325620.10.000312358
LOCFpersistence1
LSTMms_s78925610.10.000378945
Mambams_s78912810.00.000378944
PatchTST4h_d128_L3_dr00_lr1e3_pl4st2_s4212830.00.0014296
STraTSms_s78925620.10.0003789126
null—1
MIMIC-IV
Autoformerhp_d512_L1_s12351210.10.000312317
DLinearms2_s45612820.10.000345616
Dense DMShp_d256_L1_s12325610.10.000312317
DenseARpteacherctrl51220.00.000142273
GRUms_s12351210.10.000312345
Informerhp_d128_L1_s12312810.10.000312358
LOCFpersistence1
LSTMms_s133738410.20.0003133745
Mambams_s12312810.00.000312343
PatchTST4h_d128_L3_dr00_lr3e4_pl4st2_s4212830.00.00034271
STraTSd128_l1_lr1e-312810.10.00142122
null—1
eICU
Autoformerhp_d512_L3_s12351230.10.000312317
DLinearms2_s45612820.10.000345617
Dense DMSms_s45612820.10.000345617
DenseARhpsf_0cf6127a51220.00.000142258
GRUms_s12351210.10.000312345
Informerhp_d256_L1_s12325610.10.000312358
LOCFpersistence1
LSTMms_s45651210.10.000345645
Mambahpsm_f602f35f12810.00.00034235
PatchTST4h_d128_L2_dr00_lr3e4_pl4st2_s4212820.00.00034259
STraTSd128_l2_lr1e-3_do00512820.050.00142115
null—1

Table B2. Selected run per database and member: the checkpoint whose cached predictions enter the pool, its width dd, its depth (encoder/decoder where the family has both), dropout, learning rate and seed, and ckpts, the number of checkpoints stored for that family on that database. Run names are stripped of the database and the family prefix; default means the family’s standard configuration was selected.

The three members of the extended pool (§ 5.3) use one fixed configuration on every database. LightGBM fits one regressor per channel with the horizon step as a feature, on seven summaries of that channel’s input window (last value, mean, minimum, maximum, count, hours since the last observation, last minus first value) plus the static features: 200 trees, 63 leaves, learning rate 0.05, at least 50 samples per leaf. ARIMA is a per-channel ARIMA(2,1,0)(2,1,0) fitted by conditional least squares on the forward-filled input window, falling back to last value plus linear drift where the fit fails. ETS is damped Holt smoothing with fixed α=0.5\alpha= 0.5, β=0.1\beta= 0.1 and ϕ=0.9\phi= 0.9. Neither ARIMA nor ETS is fitted on the training split.

Combination algorithms

Every combiner produces a weight tensor wfhmw_{fhm} (channel, horizon step, model) and is scored by one shared code path, so differences lie in the estimator alone. Let MSE⁡fm\operatorname{MSE}_{fm} be model mm’s masked error on channel ff over the fitting split, nfn_f its observation count, and GfG_f, bfb_f the Gram matrix and cross-term of § 4.1.

BEST-SINGLE one-hot on arg⁡min⁡mMSE⁡m\arg\min_m \operatorname{MSE}_m. The reference.

MEAN w=1/Mw=1/M everywhere. No fitting.

G-EQUAL-kk 1/k1/k on the kk best members pool-wide, the same members on every channel.

G-HARM-kk the harmonic profile 1,12,…,1k1,\frac{1}{2},\ldots,\frac{1}{k} on those same pool-wide members. With EQUAL-kk and HARM-kk below it completes a 2×22 \times2 over membership and weight profile.

EQUAL-kk 1/k1/k on the kk best members of that channel — the matched control for HARM-kk, differing from it only in the weight profile.

INV-MSE wfm∝1/MSE⁡fmw_{fm} \propto1/\operatorname{MSE}_{fm}, per channel.

HARM-kk rank the members on each channel, weight the best kk by 1,12,…,1k1,\frac{1}{2},\ldots,\frac{1}{k} normalized. Only the ranking is estimated; the weights are fixed in advance, so this cannot overfit the weight values.

SELECT one-hot on arg⁡min⁡mMSE⁡fm\arg\min_m \operatorname{MSE}_{fm}, per channel, the k=1k=1 end of HARM-kk. Fewest parameters, highest variance: a discrete arg min over MM noisy estimates.

RIDGE wf=(Gf+λI)−1bfw_f=(G_f+\lambda I)^{-1}b_f, unconstrained; may be negative or >1>1.

SIMPLEX min⁡ww⊤Gfw−2w⊤bf\min_w w^\top G_f w-2w^\top b_f subject to w≥0w\ge0, ∑mwm=1\sum_m w_m=1, by projected gradient. The constraint bounds the combination’s error on that channel by that channel’s worst member.

G-SIMPLEX the same program with one weight vector for the whole panel: min⁡w∑f(w⊤Gfw−2w⊤bf+cf)/nf\min_w \sum_f (w^\top G_f w-2w^\top b_f+c_f)/n_f over the simplex, where cfc_f is the sum of squared observed targets on channel ff and channels with nf=0n_f=0 are excluded. Dividing by nfn_f matches the macro metric, so a densely sampled channel does not dominate the loss. This is the control for whether weights need to vary across channels at all.

BMA Bayesian model averaging, wfm∝exp⁡(−NfMSE⁡fm/2σf2)w_{fm}\propto\exp(-N_f\operatorname{MSE}_{fm}/2\sigma_f^2) with σf2\sigma_f^2 the best member’s error on the channel and flat priors. NfN_f is the observed-cell count; BMAstays_{\mathrm{stays}} uses the stay count instead, since cells within a stay are not independent.

HEDGE exponentiated-gradient weights wfm∝exp⁡(−ηfTfMSE⁡fm/MSE⁡‾f)w_{fm}\propto\exp(-\eta_f T_f\operatorname{MSE}_{fm}/\overline{\operatorname{MSE}}_f) with the standard ηf=8ln⁡M/Tf\eta_f=\sqrt{8\ln M/T_f}, so the exponent grows as Tf\sqrt{T_f} rather than TfT_f.

GREEDY forward selection with replacement: repeatedly add the member that most reduces error, then average the selections.

SIMPLEX-SHRUNKκ\kappa wf=αfwfsimplex+(1−αf)wfpriorw_f=\alpha_f w_f^{\mathrm{simplex}}+(1-\alpha_f)w_f^{\mathrm{prior}} with αf=nf/(nf+κ)\alpha_f=n_f/(n_f+\kappa) and wfpriorw_f^{\mathrm{prior}} the HARM-5 ranking weights for that channel, so a dense channel keeps its fitted solution and a thin one falls back to the ranking prior; a channel with no support at all falls back to one hour-pooled weight vector estimated across every channel. κ∈[100,500]\kappa\in[100,500]; κ=500\kappa=500 is ENSEMBLE.

SELECT and RIDGE additionally have hour-resolved counterparts, SEL-hh and ridge-hh, in which ww varies with hh.

Combiner ladder: validation and test

Ensemble (simplex-shrunk500) was chosen on the development column of Table D1, where it has the best mean; test was opened afterwards. Most fitted combiners improve from development to test, where the final fit sees twice the stays, and the ordering changes with it: the shrunk and unshrunk simplex swap places (§ 5.2). Unconstrained ridge is the exception on HiRID, moving from −3.59 % to +1.77 %. The two protocols also differ in scoring population, not only in fitting size.

Uncertainty

Paired approximate randomization test over stays (10 000 draws), each combiner against the validation-chosen best single model. Because MSE-per-var is a ratio of sums per channel and then a mean over channels—not a mean over units—the full statistic is recomputed on every draw rather than approximated by resampling per-stay scores. p=10−4p = 10^{-4} is the floor, 1/(R+1)1/(R + 1). Results are in Table E1.

combinerUMMHiRIDMIMIC-IIIMIMIC-IVeICU
ENS.-10.9 [-13.3, -8.2]-10.0 [-15.5, -6.9]-9.8 [-11.5, -7.9]-6.5 [-8.6, -4.9]-6.1 [-6.7, -5.2]
simplex-10.7 [-13.1, -8.0]-12.4 [-16.7, -8.9]-9.8 [-11.8, -7.6]-6.5 [-8.6, -4.9]-6.3 [-6.9, -5.3]
GREEDY-10.7 [-13.1, -7.9]-12.4 [-16.7, -8.8]-9.9 [-11.8, -7.6]-6.6 [-8.6, -5.0]-6.3 [-6.9, -5.3]
HARM-5-8.8 [-11.2, -6.0]-8.5 [-14.2, -5.3]-8.5 [-10.1, -6.8]-5.0 [-6.3, -3.8]-5.0 [-5.8, -3.8]
HARM-3-8.1 [-10.4, -5.4]-8.1 [-13.7, -4.8]-7.6 [-9.2, -6.0]-4.1 [-5.2, -3.2]-4.5 [-5.2, -3.2]
SELECT-4.9 [-9.3, -0.6] *-7.3 [-12.3, -3.3]-3.8 [-5.5, -2.2]-1.5 [-2.3, -0.7] *-1.5 [-2.5, -0.0] *
mean-8.0 [-10.7, -4.7]-6.3 [-8.7, -4.0]-5.9 [-8.0, -3.6]-1.1 [-3.9, 1.5] †-2.7 [-3.2, -1.6]

Table E1. Change against the best single model on test, with 95% bootstrap confidence intervals over stays. † marks p≥0.05p \ge0.05 and * marks 10−4<p<0.0510^{-4} < p < 0.05 under a paired randomization test.

† not significant at p<0.05p < 0.05. Every soft combiner, fitted or rank-weighted, is significant at the floor on every database. Per-channel SELECT is significant everywhere, but above the floor on UMM (p=0.04p = 0.04), MIMIC-IV (p=4×10−4p = 4 \times10^{-4}) and eICU (p=0.04p = 0.04); the only failure is the unweighted mean on MIMIC-IV, i.e. naive averaging is the one combiner that can fail to beat a single model. Brackets give a 95% percentile confidence interval from 2 000 bootstrap resamples of stays. Table 3 resamples patients instead.

stackerharm-15equal
UMM−2.85*−1.42+2.32
HiRID−5.27−4.38+0.77
MIMIC-III−4.97***−3.82***+0.22
MIMIC-IV−5.90***−4.36***+0.04
eICU−4.89***−4.02***−0.81

Table 3. Extended 15-member pool on test: change in macro-MSE relative to LightGBM, the validation-selected best single model, in percent. Negative is better than LightGBM. Each entry is tested against LightGBM, pp from approximate randomization over patients, *p<0.05p < 0.05, ***p<0.001p < 0.001. Re-running with the 17-member pool, which adds the two foundation models, changes the stacker by at most 0.11 points.

Which model wins which cells

Share of well-supported (variable, hour) test cells on which each family has the lowest error (≥200\ge200 observations on the scoring split), in Table F1. Shares sum to 100% per column. Table F1 aggregates over the horizon; Figure F1 shows the same quantity resolved by (variable, hour) on one database.

familyUMMHiRIDMIMIC-IIIMIMIC-IVeICUmean
STraTS12.224.430.040.035.528.4
DenseAR22.821.134.220.635.826.9
Informer21.29.37.17.52.99.6
GRU6.211.12.614.13.97.6
Dense DMS9.18.77.93.07.37.2
DLinear13.916.51.71.10.86.8
PatchTST5.72.97.15.25.65.3
LOCF3.32.53.42.73.03.0
LSTM1.70.50.40.92.71.3
Moirai1.31.41.81.30.51.2
TiRex-21.90.90.71.01.01.1
Null0.10.02.81.80.11.0
Autoformer0.30.60.30.10.70.4
Mamba0.30.10.00.70.20.3

Table F1. Share of well-supported (variable, hour) test cells won by each family (%).

Pool diversity

Mean pairwise residual correlation between pool members, averaged over the five databases and weighted by observation count, in Table G1. What a combiner can exploit is exactly what this matrix is missing from one: two members correlating at 0.95 leave almost nothing to arbitrate. Three structures are visible. The seven in-domain sequence models form a block at 0.91–0.95, so they are close to interchangeable as ensemble members. The trivial baselines are the most distant members of all — null at 0.42–0.68 and locf at 0.77–0.90 — which is why they still earn weight. And the two zero-shot foundation models correlate most with PatchTST (0.92–0.93) and with each other (0.91), and least with the in-domain sequence models (0.81–0.84), which makes them the least correlated learned members.

DenseARInformerDenseDMSGRULSTMSTraTSMambaDLinearPatchTSTAutoformerMoiraiTiRex-2LOCFnull
DenseAR1.000.950.940.950.940.920.940.900.900.860.830.840.790.67
Informer0.951.000.940.950.940.920.940.900.900.870.830.840.790.66
DenseDMS0.940.941.000.940.940.910.940.890.890.860.820.840.780.67
GRU0.950.950.941.000.950.920.950.900.900.860.830.840.800.67
LSTM0.940.940.940.951.000.920.940.890.890.850.820.830.780.68
STraTS0.920.920.910.920.921.000.920.870.870.850.810.820.770.64
Mamba0.940.940.940.950.940.921.000.900.890.860.830.840.780.67
DLinear0.900.900.890.900.890.870.901.000.930.890.870.870.810.65
PatchTST0.900.900.890.900.890.870.890.931.000.920.920.930.880.57
Autoformer0.860.870.860.860.850.850.860.890.921.000.830.870.770.55
Moirai0.830.830.820.830.820.810.830.870.920.831.000.910.900.48
TiRex-20.840.840.840.840.830.820.840.870.930.870.911.000.850.48
LOCF0.790.790.780.800.780.770.780.810.880.770.900.851.000.42
null0.670.660.670.670.680.640.670.650.570.550.480.480.421.00

Table G1. Mean pairwise residual correlation between pool members, averaged over the five databases; 1.00 on the diagonal by construction, lower is more complementary.

Results by channel group

MSE-per-var on test for each pool family and for the combiner chosen on validation (ENSEMBLE), within each channel group. Channels carrying fewer than 100 observations are excluded, as in every group-level table here. One table per database follows (Table H1–Table H5). The combination beats every pool member in all 50 group × database cells, ten groups of which the tables show seven (freq:mid, type:ventilation and the channels outside sofa_relevant are omitted). Without the support filter it loses two of them — eICU ventilation and HiRID medications — but both losses are produced entirely by channels too thin to estimate a per-channel weight on, which is the artifact the filter exists to remove.

familyall_chartedtype:labtype:vitaltype:medicationsofa_relevantfreq:lowfreq:high
ENSEMBLE0.5300.4690.4440.8870.8440.8310.407
GREEDY0.5320.4690.4440.9010.8560.8380.407
harm-50.5430.4840.4570.8970.8560.8510.412
mean0.5490.4980.4550.8820.8700.8680.414
select0.5670.5050.4750.9450.8830.8980.420
Informer0.5770.5010.4711.0210.8980.9230.425
Dense DMS0.5800.5070.4811.0080.8900.9230.428
GRU0.5910.5510.4810.9150.8760.9380.429
DenseAR0.5960.5330.4771.0170.9560.9720.426
STraTS0.6010.5260.4961.0310.9650.9420.453
DLinear0.6050.5710.5040.9020.9180.9720.434
PatchTST0.6170.5800.5180.9190.9280.9720.457
LSTM0.6490.6100.4941.0531.0241.0640.443
Autoformer0.6610.6190.5521.0030.9811.0320.502
Moirai0.6620.6220.5880.9420.9541.0110.526
Mamba0.6640.6330.5061.0501.0301.0680.442
TiRex-20.6670.6280.5780.9750.9571.0210.515
LOCF0.7020.6380.6461.0001.0401.0220.603
Null1.0601.0480.9811.2561.3241.2600.974

Table H1. Table H1: UMM: MSE-per-var by channel group (test). ENSEMBLE is the combiner chosen on validation.

familyall_chartedtype:labtype:vitaltype:medicationsofa_relevantfreq:lowfreq:high
greedy0.3940.3650.4610.4210.3360.4330.385
ENSEMBLE0.3940.3650.4610.4210.3350.4330.386
harm-50.4000.3710.4640.4300.3390.4440.388
mean0.4100.3850.4710.4260.3460.4510.400
select0.4140.3810.4730.4670.3570.4710.395
DenseAR0.4180.3880.4730.4690.3670.4800.396
Informer0.4220.3930.4790.4650.3670.4760.403
GRU0.4250.3970.4760.4680.3740.4840.402
LSTM0.4270.4010.4770.4660.3680.4830.405
Dense DMS0.4350.4070.4910.4690.3890.4820.416
STraTS0.4470.3880.4880.6080.4420.5440.401
Mamba0.4600.4330.4890.5340.3970.5570.411
PatchTST0.4620.4420.5200.4720.3880.5200.444
DLinear0.4830.4680.5380.4810.4100.5340.467
Autoformer0.5060.4740.5470.5760.4360.5900.472
Moirai0.5060.4750.6040.5060.4190.5600.508
LOCF0.5260.4800.6610.5220.4300.5660.550
TiRex-20.5310.5020.5750.5880.4640.6080.504
Null0.9950.9961.0360.9550.9921.0070.992

Table H5. eICU: MSE-per-var by channel group (test). ENSEMBLE is the combiner chosen on validation.

familyall_chartedtype:labtype:vitaltype:medicationsofa_relevantfreq:lowfreq:high
ENSEMBLE0.4290.3460.3930.8650.5450.5140.425
GREEDY0.4300.3470.3930.8660.5460.5150.425
harm-50.4360.3550.3970.8710.5500.5260.428
select0.4550.3730.4070.9160.5620.5530.441
mean0.4570.3660.4220.9370.5950.5700.437
DenseAR0.4870.3950.4171.0500.6570.6370.445
Informer0.4880.3800.4421.0790.6700.6270.453
GRU0.4910.3970.4281.0490.6700.6440.447
DLinear0.4950.4100.4620.9450.5970.6310.459
Dense DMS0.4970.3910.4281.1250.7030.6340.465
PatchTST0.5130.4240.4830.9650.6070.6280.492
STraTS0.5400.4560.4731.0820.6900.7860.446
LSTM0.5470.4630.4641.1230.7410.7790.456
Autoformer0.5590.4690.5221.0360.6470.6750.537
Moirai0.5610.4570.5531.0320.6460.6510.570
Mamba0.5680.4860.4881.1400.7470.8260.459
TiRex-20.5810.4810.5541.0890.6830.6660.566
LOCF0.5910.4680.5851.1130.6840.6490.635
Null1.0100.9510.9891.3281.2991.1950.930

Table H2. Table H2: HiRID: MSE-per-var by channel group (test). ENSEMBLE is the combiner chosen on validation.

familyall_chartedtype:labtype:vitaltype:medicationsofa_relevantfreq:lowfreq:high
GREEDY0.4030.3940.4510.3910.3520.4590.449
ENSEMBLE0.4040.3940.4520.3900.3520.4590.449
harm-50.4100.4000.4530.4020.3630.4690.452
mean0.4240.4210.4680.3970.3650.4920.463
select0.4290.4230.4590.4330.3850.5040.458
Informer0.4440.4330.4660.4740.4060.5340.465
DenseAR0.4470.4400.4590.4830.4080.5600.458
GRU0.4500.4470.4660.4650.3970.5500.466
Dense DMS0.4550.4460.4840.4680.4300.5340.484
STraTS0.4710.4460.4710.5810.4390.5980.468
LSTM0.4730.4800.4730.4730.4240.5930.474
Mamba0.4850.4860.4730.5260.4510.6300.472
PatchTST0.4870.4960.5270.4310.4020.5780.515
DLinear0.5100.5190.5490.4550.4280.5970.530
Moirai0.5340.5350.6150.4550.4340.6040.596
Autoformer0.5360.5430.5520.5240.4640.6480.555
LOCF0.5620.5400.7070.4750.4650.5980.666
TiRex-20.5690.5610.5780.6080.5610.6910.595
Null0.9330.9810.9670.7741.0450.9460.938

Table H3. MIMIC-III: MSE-per-var by channel group (test). ENSEMBLE is the combiner chosen on validation.

familyall_chartedtype:labtype:vitaltype:medicationsofa_relevantfreq:lowfreq:high
GREEDY0.4140.3890.5070.3970.2920.5050.429
ENSEMBLE0.4140.3890.5070.3960.2920.5050.429
harm-50.4210.3990.5110.4000.2920.5210.430
select0.4370.4150.5230.4180.3050.5510.438
mean0.4380.4170.5190.4260.3010.5340.443
DenseAR0.4430.4220.5300.4250.3150.5530.446
GRU0.4480.4340.5200.4220.3090.5710.441
Informer0.4490.4310.5290.4310.3110.5730.443
Dense DMS0.4530.4370.5310.4300.3190.5810.444
LSTM0.4570.4440.5250.4370.3200.5830.448
STraTS0.4700.4470.5320.5020.3370.6400.442
Mamba0.4970.5030.5380.4590.3350.6990.448
PatchTST0.5300.4980.5830.6150.3440.6320.497
DLinear0.5330.5300.6000.4840.3690.6620.521
Autoformer0.5540.5260.6000.6350.3790.6780.511
Moirai0.5890.5280.6770.7260.3820.6570.570
TiRex-20.6080.5600.6380.7850.3960.6740.555
LOCF0.6240.5340.7720.7750.4130.6600.667
Null0.9861.0421.0100.8210.8441.0910.943

Table H4. MIMIC-IV: MSE-per-var by channel group (test). ENSEMBLE is the combiner chosen on validation.

Gain by horizon

Combination gain of the combiner chosen on validation against the best single model, by horizon step. The gain decays with horizon on all five databases, from h=1h = 1 to h=24h = 24, which is the complement of the residual correlation rising with horizon: members converge on the same drift, so there is progressively less disagreement to arbitrate. Table I1 gives the values and Figure I1 plots them.

horizonUMMHiRIDMIMIC-IIIMIMIC-IVeICUmean
1-20.99-14.51-15.35-9.80-11.58-14.45
2-15.48-12.40-16.29-8.15-11.08-12.68
3-9.19-25.65-14.63-8.07-8.20-13.15
4-8.95-10.62-11.52-10.23-9.18-10.10
5-11.08-10.68-9.01-8.55-8.08-9.48
6-11.13-10.42-14.31-6.47-8.14-10.09
7-9.10-18.84-11.76-7.29-6.74-10.75
8-7.23-8.72-10.17-8.29-6.85-8.25
9-12.14-11.77-4.45-6.82-6.09-8.25
10-10.53-10.20-11.41-5.49-6.12-8.75
11-9.74-18.61-10.19-6.38-5.63-10.11
12-11.63-8.00-8.97-7.12-5.42-8.23
13-12.05-9.02-4.37-7.69-4.94-7.61
14-8.97-9.28-10.63-4.87-5.06-7.76
15-6.98-16.19-9.10-5.79-4.82-8.58
16-10.05-7.73-7.62-6.22-4.55-7.23
17-11.21-9.67-5.61-7.34-4.47-7.66
18-8.13-8.13-9.89-4.92-4.09-7.03
19-7.06-13.20-8.29-5.21-4.11-7.57
20-12.63-8.08-6.98-5.46-3.93-7.42
21-11.53-9.28-4.44-5.48-3.71-6.89
22-8.76-8.26-8.68-3.99-3.96-6.73
23-7.06-8.86-7.61-4.47-3.80-6.36
24-10.31-5.48-6.30-5.27-3.27-6.13

Table I1. Combination gain against the best single model by horizon step (%).

Figure I1. Combination gain by horizon. The gain decays as members converge on the same drift. The two smallest databases are visibly noisier hour to hour, the per-hour form of the split-seed spread in § 5.2. HiRID’s excursion at h=3h = 3 is a period-4 oscillation present in the underlying data rather than a plotting artifact; it recurs at h=7h = 7, 11 and 15. Its period equals the 4-hour window stride, which is the likely but unverified cause.

Does the weight profile matter?

Section 5.2 shows that choosing members per channel is worth about 2 pp while the harmonic profile on those members is worth about 0.2. This appendix adds the third rung: instead of fixing the profile at 1, 12,…,1k\frac{1}{2}, \ldots, \frac{1}{k}, fit one weight vector over rank positions, shared by every channel. It is cheap — permuting the model axis of GG and bb into rank order gives the rank-space normal equations with no pass over predictions — and it changes little, tracking normalized 1/k1/k to within 0.045 at a correlation of 0.99.

Table J1 weights identical per-channel members three ways, and how much the profile matters turns out to depend on how many members it spans. At k=3k = 3 and k=5k = 5 the three columns agree on average to within about 0.2 pp, and fitting beats the fixed 1/k1/k decay only marginally (by at most 0.21 pp on the development means, and not on the test means). At k=14k = 14 they separate: equal-14, the unweighted mean, is 3.1 pp worse than harm-14 on test and worse even than equal-5, because flat weights over the whole pool give the weakest members the same say as the best. It is the decay rather than the fitting that carries the full pool — harm-14 estimates nothing and still edges out fit-14, −7.89 against −7.85.

databaseEQUAL-3HARM-3fit-3EQUAL-5HARM-5fit-5EQUAL-14HARM-14fit-14ENS.
development (half of val → held-out half)
UMM-7.93-7.13-7.76-8.67-8.02-8.56-8.10-10.02-10.35-10.67
HiRID-3.05-3.17-3.18-3.74-4.01-4.07-4.50-5.62-5.14-8.00
MIMIC-III-4.88-4.53-4.73-5.05-5.12-5.13-3.63-5.91-5.67-5.61
MIMIC-IV-4.37-4.41-4.53-4.08-4.70-4.72-1.20-4.87-5.14-5.57
eICU-3.79-3.87-3.95-3.75-4.12-3.92-3.14-5.06-4.64-5.00
mean-4.80-4.62-4.83-5.06-5.19-5.28-4.11-6.30-6.19-6.97
test (all of val → test)
UMM-8.58-8.10-8.44-8.98-8.79-9.04-8.01-9.95-10.13-10.95
HiRID-6.96-8.08-7.48-7.36-8.52-8.07-6.26-9.73-9.13-10.01
MIMIC-III-7.95-7.57-7.75-8.85-8.48-8.54-5.86-8.98-9.09-9.79
MIMIC-IV-3.99-4.14-4.18-5.05-4.97-4.83-1.07-5.30-5.31-6.51
eICU-4.21-4.46-4.49-4.62-5.01-5.00-2.67-5.48-5.61-6.09
mean-6.34-6.47-6.47-6.97-7.15-7.10-4.77-7.89-7.85-8.67

Table J1. Weight profiles at fixed membership, as percentage change against the best single model (negative is better). EQUAL, HARM and FIT use the same per-channel top-kk members and differ only in how those members are weighted. FIT-kk fits one weight vector over rank positions, shared by every channel. EQUAL-14 is the unweighted mean over the whole pool. ENS. is the per-channel shrunk simplex, shown for scale. Referenced from Section 5.2.

Inference-time injection and fine-tuning

Five variants of feeding a decoder better values during generation, on two databases and two iterative decoders, as validation MSE-per-var relative to free-running (positive is worse, Table K1):

DenseAR UMMSTraTS UMMDenseAR HiRIDSTraTS HiRID
free-running (absolute)0.48530.52530.43440.4715
ensemble, masked−0.3−9.4+0.3−3.4
ground truth, masked+1.1−29.4+1.7−23.5
ensemble, unmasked+1.9+13.7+0.3+29.1
per-channel best, unmasked+3.5+20.5+0.6+38.6
live hybrid+3.6+29.1+0.6+178.3

Table K1. Inference-time injection, as percentage change against free-running generation on validation. The masked rows require the future observation mask and are therefore not deployable.

The masked rows inject only where the target is observed, so which channels receive an injection encodes the future observation mask; since ICU sampling is informative, those rows are not deployable however large the improvement looks. The unmasked rows need no such information and are uniformly worse. A decoder trained on its own predictions is already adapted to free-running and even ground truth costs it 1–2%; a decoder trained under a teacher-forcing curriculum carries exposure bias instead, which is why injection appears to help it so much and why that help cannot be used.

Fine-tuning DenseAR on UMM to consume the combined trajectory improved validation error from 0.4245 to 0.4166, but a matched control — identical fine-tuning, no injection — reached 0.4187. Of the apparent −1.9%, therefore, −1.37 pp comes from training longer with a fresh learning-rate schedule, which the matched control gets as well, and only −0.49 pp is attributable to injection, while the resulting model requires the whole pool at inference exactly as post-hoc combination does.

Pool size and the extended pool

Table L1 gives the test error of every member and combiner of the extended 15-member pool, and Table L2 the share of cells each member wins.

References

  1. [1]Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda Arango, Shubham Kapoor, et al. Chronos: Learning the language of time series. Transactions on Machine Learning Research, 2024. URL https://arxiv.org/abs/2403.07815.
  2. [2]J. M. Bates and C. W. J. Granger. The combination of forecasts. Journal of the Operational Research Society, 20(4):451–468, 1969. URL https://link.springer.com/article/10.1057/jors.1969.103.
  3. [3]Nathanael Bosch, Oleksandr Shchur, Nick Erickson, Michael Bohlke-Schneider, and Ali Caner Turkmen. Multi-layer stack ensembles for time series forecasting. In Leman Akoglu, Carola Doerr, Jan N. van Rijn, Roman Garnett, and Jacob R. Gardner, editors, Proceedings of the Fourth International Conference on Automated Machine Learning, volume 293 of Proceedings of Machine Learning Research, pages 15/1–30. PMLR, 08–11 Sep 2025. URL https://proceedings.mlr.press/v293/bosch25a.html.arxiv.org/abs/2511.15350
  4. [4]George E. P. Box and Gwilym M. Jenkins. Time Series Analysis: Forecasting and Control. Holden-Day, San Francisco, 1970.
  5. [5]Leo Breiman. Stacked regressions. Machine Learning, 24(1):49–64, 1996. URL https://link.springer.com/article/10.1007/BF00117832.DOI
  6. [6]Rich Caruana, Alexandru Niculescu-Mizil, Geoff Crew, and Alex Ksikes. Ensemble selection from libraries of models. In Proceedings of the Twenty-first International Conference on Machine Learning (ICML), 2004. URL https://dl.acm.org/doi/10.1145/1015330.1015432.DOI
  7. [7]Kyunghyun Cho, Bart van Merriënboer, Çağlar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Alessandro Moschitti, Bo Pang, and Walter Daelemans, editors, Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1724–1734, Doha, Qatar, October 2014. Association for Computational Linguistics. doi: 10.3115/v1/D14-1179. URL https://aclanthology.org/D14-1179/.DOI
  8. [8]Gerda Claeskens, Jan R. Magnus, Andrey L. Vasnev, and Wendum Wang. The forecast combination puzzle: A simple theoretical explanation. International Journal of Forecasting, 32(3):754–762, 2016. ISSN 0169-2070. doi: https://doi.org/10.1016/j.ijforecast.2015.12.005. URL https://www.sciencedirect.com/science/article/pii/S0169207016000327.
  9. [9]Gordon V. Cormack, Charles L A Clarke, and Stefan Buettcher. Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’09, page 758–759, New York, NY, USA, 2009. Association for Computing Machinery. ISBN 9781605584836. URL https://doi.org/10.1145/1571941.1572114.
  10. [10]John Duchi, Shai Shalev-Shwartz, Yoram Singer, and Tushar Chandra. Efficient projections onto the ℓ₁-ball for learning in high dimensions. In Proceedings of the 25th International Conference on Machine Learning (ICML), 2008. URL https://dl.acm.org/doi/10.1145/1390156.1390191.DOI
  11. [11]M.A. Ganaie, Minghui Hu, A.K. Malik, M. Tanveer, and P.N. Suganthan. Ensemble deep learning: A review. Engineering Applications of Artificial Intelligence, 115:105151, 2022. ISSN 0952-1976. URL https://www.sciencedirect.com/science/article/pii/S095219762200269X.DOI
  12. [12]Everette S. Gardner, Jr. and Ed. McKenzie. Forecasting trends in time series. Management Science, 31(10):1237–1246, October 1985. doi: 10.1287/mnsc.31.10.1237. URL https://www.bauer.uh.edu/egardner/3301H%20Operations%20Management/ESG%20Publications/1985%20Forecasting%20trends.pdf.DOI
  13. [13]Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. In First Conference on Language Modeling (COLM), 2024. URL https://arxiv.org/abs/2312.00752.
  14. [14]L.K. Hansen and P. Salamon. Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(10):993–1001, 1990. URL https://dl.acm.org/doi/10.1109/34.58871.DOI
  15. [15]Espen Haugsdal, Erlend Aune, and Massimiliano Ruocco. Persistence initialization: a novel adaptation of the transformer architecture for time series forecasting. Applied Intelligence, 53(22):26781–26796, 2023. URL https://doi.org/10.1007/s10489–023–04927–4.
  16. [16]Hansika Hewamalage, Klaus Ackermann, and Christoph Bergmeir. Forecast evaluation for data scientists: common pitfalls and best practices. Data Mining and Knowledge Discovery, 37:788–832, 2023. URL https://doi.org/10.1007/s10618–022–00894–5.
  17. [17]Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory (LSTM). Neural Computation, 9(8):1735–1780, 1997. doi: 10.1162/neco.1997.9.8.1735. URL http://dx.doi.org/10.1162/neco.1997.9.8.1735.
  18. [18]Arthur E. Hoerl and Robert W. Kennard. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1):55–67, 1970. ISSN 00401706. URL http://www.jstor.org/stable/1271436.
  19. [19]Stephanie L. Hyland, Martin Faltys, Matthias Hüser, Xinrui Lyu, Thomas Gumbsch, Cristóbal Esteban, Christian Bock, Max Horn, Michael Moor, Bastian Rieck, Marc Zimmermann, Dean Bodenham, Karsten Borgwardt, Gunnar Rätsch, and Tobias M. Merz. Early prediction of circulatory failure in the intensive care unit using machine learning. Nature Medicine, 26:364–373, 2020. URL https://doi.org/10.1038/s41591–020–0789–4.
  20. [20]Rob J Hyndman and Yासmin Khandakar. Automatic time series forecasting: the forecast package for r. Journal of statistical software, 27:1–22, 2008.
  21. [21]Alistair E. W. Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom J. Pollard, Sicheng Hao, Benjamin Moody, Brian Gow, Li-wei H. Lehman, Leo A. Celi, and Roger G. Mark. MIMIC-IV, a freely accessible electronic health record dataset. Scientific Data, 10(1), 2023. URL https://doi.org/10.1038/s41597-022-01899-x.
  22. [22]Alistair E.W. Johnson, Tom J. Pollard, Lu Shen, Li wei H. Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G. Mark. MIMIC-III, a freely accessible critical care database. Scientific Data, 3(1):160035, 2016. URL https://doi.org/10.1038/sdata.2016.35.
  23. [23]Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. Lightgbm: A highly efficient gradient boosting decision tree. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https://proceedings.neurips.cc/paper_files/paper/2017/file/6449f44a102fde846869bdd9ebf676fa-Paper.pdf.
  24. [24]Anders Krogh and Jesper Vedelsby. Neural network ensembles, cross validation, and active learning. In G. Tesauro, D. Touretzky, and T. Leen, editors, Advances in Neural Information Processing Systems, volume 7. MIT Press, 1994. URL https://proceedings.neurips.cc/paper_files/paper/1994/file/b8c37e33defde51cf91e1e03e51657da-Paper.pdf.
  25. [25]Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. The M4 competition: 100,000 time series and 61 forecasting methods. International Journal of Forecasting, 36(1):54–74, 2020. ISSN 0169-2070. URL https://www.sciencedirect.com/science/article/pii/S0169207019301128. M4 Competition.DOI
  26. [26]Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. M5 accuracy competition: Results, findings, and conclusions. International Journal of Forecasting, 38(4):1346–1364, 2022. ISSN 0169-2070. URL https://www.sciencedirect.com/science/article/pii/S0169207021001874. Special Issue: M5 competition.DOI
  27. [27]Yuqi Nie, Nam H Nguyen, Phanwadее Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In Proceedings of the 11th International Conference on Learning Representations (ICLR), Kigali, Rwanda, 2023. URL https://openreview.net/forum?id=Jbdc0vT0col.
  28. [28]Michael P. Perrone and Leon N. Cooper. When networks disagree: Ensemble methods for hybrid neural networks. In R. J. Mammone, editor, Artificial Neural Networks for Speech and Image Processing, pages 126–142. Chapman-Hall, New York, 1992. URL https://www.worldscientific.com/doi/10.1142/9789812795885_0025.
  29. [29]Patrick Podest, Marco Pichler, Elias Bürger, Levente Zólyomi, Bernhard Voggenberger, Wilhelm Berghammer, Daniel Klotz, Sebastian Böck, Günter Klambauer, and Sepp Hochreiter. TiRex-2: Generalizing TiRex to multivariate data and streaming, 2026. URL https://arxiv.org/abs/2607.01204.
  30. [30]Tom J. Pollard, Alistair E. W. Johnson, Jesse D. Raffa, Leo A. Celi, Roger G. Mark, and Omar Badawi. The eICU collaborative research database, a freely available multi-center database for critical care research. Scientific Data, 5(180178), 2018. URL https://doi.org/10.1038/sdata.2018.178.
  31. [31]Shigehiko Schamoni, Michael Hagmann, and Stefan Riezler. Ensembling neural networks for improved prediction and privacy in early diagnosis of sepsis. Proceedings of Machine Learning Research: Machine Learning for Healthcare, 182: 1–23, 2022. URL https://proceedings.mlr.press/v182/schamoni22a/schamoni22a.pdf.
  32. [32]Jeremy Smith and Kenneth F. Wallis. A simple explanation of the forecast combination puzzle. Oxford Bulletin of Economics and Statistics, 71(3):331–355, 2009. doi: https://doi.org/10.1111/j.1468-0084.2008.00541.x. URL https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1468-0084.2008.00541.x.
  33. [33]Michael Staniek, Marius Fracarolli, Michael Hagmann, and Stefan Riezler. Early prediction of causes (not effects) in healthcare by long-term clinical time series forecasting. Proceedings of Machine Learning Research: Machine Learning for Healthcare, 252:1–29, 2024. URL https://proceedings.mlr.press/v252/staniek24a.html.
  34. [34]Sindhu Tipirneni and Chandan K. Reddy. Self-supervised transformer for sparse and irregularly sampled multivariate clinical time-series. ACM Transactions on Knowledge Discovery from Data, 16(6), 2022. URL https://doi.org/10.1145/3516367.
  35. [35]David H. Wolpert. Stacked generalization. Neural Networks, 5(2):241–259, 1992. ISSN 0893-6080. URL https://www.sciencedirect.com/science/article/pii/S0893608005800231.DOI
  36. [36]Gerald Woo, Chenghao Liu, Akshat Kumar, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. Unified training of universal time series forecasting transformers. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org, 2024. URL https://dl.acm.org/doi/10.5555/3692070.3694248.
  37. [37]Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. In Advances in Neural Information Processing Systems (NeurIPS), virtual, 2021. URL https://proceedings.neurips.cc/paper_files/paper/2021/file/bcc0d400288793e8bdcd7c19a8ac0c2b-Paper.pdf.
  38. [38]Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the 37th AAAI Conference on Artificial Intelligence, Washington DC, USA, 2023. URL https://ojs.aaai.org/index.php/AAAI/article/view/26317/26089.
  39. [39]Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In The Thirty-Fifth AAAI Conference on Artificial Intelligence, virtual, 2021. URL https://doi.org/10.1609/aaai.v35i12.17325.

Paper details

vlnc:W_gvu5m4wt
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY-SA 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents