WorksComputer Science
What Survival Benchmarks Don't Tell You: Impact of Model Selection and Dataset Regimes
Abstract
Despite the proliferation of survival analysis methods, current benchmarks often operate under narrow conditions: they heavily rely on the concordance index (C-index) for evaluation and model selection, and frequently either exclude deep learning (DL) methods or underrepresent challenging dataset regimes. To address these limitations, we conduct a large-scale, neutral benchmark comparing 10 classical, tree-based, and DL methods across 49 diverse datasets. Crucially, we treat the model selection criterion (optimizing for C-index, Integrated Brier Score (IBS), or Mean Absolute Error (MAE-PO)) as an independent experimental variable. Using Bayesian mixed-effects modeling, we demonstrate that evaluation design heavily dictates conclusions. For complex DL and tree-based architectures, selecting hyperparameters to optimize for IBS instead of C-index yields substantial improvements in probabilistic accuracy and calibration at a negligible cost to discriminative ranking. Furthermore, method performance (C-index, IBS, MAE-PO, and X-CAL) is fundamentally governed by dataset characteristics. While DL models scale exceptionally well with larger sample sizes, they suffer notable performance drops in high-dimensional feature spaces. Additionally, we expose a structural vulnerability where heavy censoring artificially inflates absolute calibration metrics, creating overly conservative tests. Ultimately, our findings challenge the utility of global single-number performance summaries and advocate for survival analysis results that transparently report the evaluation setup. Our code is available at: https://github.com/sdening/surv-bench-study
Introduction
The Cox proportional hazards model [12] has been a cornerstone method of survival analysis (time-to-event modeling) for over fifty years. While regularized extensions of it such as CoxNet [45] remain competitive today, the methodological landscape has rapidly expanded with, for instance, tree-based ensembles (e.g., [24, 2]) and a recent wave of deep survival models (e.g., [27, 33, 32, 50, 51] (Knott et al. 2025)). Yet, the evaluation practices used to compare these methods have not kept pace. Results are typically summarized through average ranks, winner counts on Harrell’s concordance index [22] (or variants of it such as those by Antolini et al. [ [3] and Uno et al. [ [48]); for ease of exposition, throughout the paper we abbreviate these collectively simply as the “C-index”, specifying a particular variant as needed) or mean metric values across datasets—a compression that obscures crucial method-level variation. This paper investigates what this standard evaluation process misses, demonstrating that the field has underestimated the nuances of model performance across different dataset regimes and especially across different model selection criteria.
Current neutral benchmarks for survival analysis generally converge on a stable consensus: no modern method consistently outperforms classical Cox-based methods [11, 7, 23, 5, 52]. However, this conclusion is drawn under evaluation conditions that leave large areas of the methodological space unexamined. As highlighted by Lillelund et al. [ [34], survival analysis model evaluations overwhelmingly rely on the C-index, neglecting probabilistic accuracy, time-to-event accuracy, and calibration. Crucially, the model selection strategy (e.g., the choice of evaluation metric driving hyperparameter and checkpoint selection) is almost never varied. Papers introducing new models frequently select hyperparameters based on the C-index without exploring how other hyperparameter selection strategies might have performed (e.g., [27, 33, 28]). Taken together, the conclusion that Cox remains competitive may hold under these narrow conditions, including datasets of small sample and feature size, but it has not been robustly tested in regimes where alternatives could plausibly excel.
To address this, we conduct a large-scale benchmark treating evaluation conventions as experimental variables. We compare 10 methods (across linear Cox, tree-based, and deep learning method families) across 49 diverse datasets, each independently optimized under three model selection criteria on a validation set: time-dependent Antolini C-index () [3], Integrated Brier Score (IBS) [19], and a Pseudo-Observation-based Mean Absolute Error (MAE-PO) [42]. On the held-out test set, all three metrics plus calibration (measured via X-CAL [18]) are computed for every experiment, yielding a comprehensive set of results across methods, datasets, and selection strategies. Using Bayesian Generalized Linear Mixed-effects Models (GLMMs) [17], we estimate effect strengths of different components in our benchmark on target metrics, thereby accounting for interaction and confounding effects. We further complement all these findings with descriptive group-wise mean and median trends in the Appendix (E). Our core findings fall into two main categories:
The Evaluation vs. Optimization Gap: Moving Beyond C-index Selection
A.1 What Drives Performance Depends on the Metric: C-index typically varies little between methods within a dataset, but much more across datasets. Other metrics tell a different story: Probabilistic accuracy (IBS) and calibration (X-CAL), in contrary to C-index, show significant sensitivity to optimization and algorithmic choice.
A.2 The Benefits of Proper Scoring Rules as Model Selection Criteria: Relying solely on the C-index for hyperparameter selection fails to regularize the predictive distributions of complex deep learning and tree-based methods. Shifting model selection to optimize a proper scoring rule like the Integrated Brier Score (IBS) yields significant gains in probabilistic accuracy and calibration, without significantly altering the model’s discriminative ranking.
A.3 Time-to-event Accuracy Requires Direct Optimization: When models are tuned by optimizing C-index or IBS, their ability to improve Mean Absolute Error (MAE-PO) over a simple Kaplan-Meier (KM) baseline [26] is essentially a coin flip (50%). By instead tuning models to optimize MAE-PO, the models we benchmark outperform the KM baseline 72% of the time while reducing extreme temporal prediction outliers.
Contextualizing Performance: The Role of Dataset Regimes
B.1 Dataset Scale and Dimensionality Govern Method Performance: Tree-based and deep learning methods consistently scale to larger sample sizes. However, deep learning models struggle with high-dimensional (feature count () > observation count ()) data, suffering notable IBS and MAE performance drops as feature counts increase. This hints that deep learning may be overfitting and may benefit from regularization strategies when many features are available.
B.2 The Calibration Fallacy: Relative vs Absolute Utility. We observe artificial improvement in X-CAL scores as the dataset censoring rate increases. While X-CAL remains a robust and highly informative metric for the relative ranking of models (i.e., comparing one architecture against another on the same data), its absolute value becomes highly unreliable under heavy censoring. Consequently, standard D-Calibration tests and X-CAL objectives become overly conservative, masking underlying miscalibration and falsely labeling models as well-calibrated simply because they are evaluated on highly censored datasets.
Related Work
Survival analysis benchmarking has matured considerably in recent years, yet several structural gaps continue to limit the conclusions these studies can draw. We summarize eight dedicated benchmarks in Tab. A.1; these are studies that compare existing methods without proposing new ones. Despite the progress in scale and evaluation diversity, four interrelated gaps persist: systematic underrepresentation of deep learning methods, limited dataset diversity, inconsistent evaluation methodology, and the treatment of model selection criteria as a fixed convention rather than a design variable.
Deep Learning is Systematically Underrepresented. Most benchmarks in Tab. A.1 are R-based, structurally excluding recent, Python-native DL methods despite Wiegrebe et al. [ [51] cataloging over 60 DL survival methods when they wrote their review. Herrmann et al. [ [23] and Burk et al. [ [7] include no DL methods; Burk et al. [ [7] explicitly cite unreliable integration of DL methods into their R-based benchmark suite as the reason for their exclusion. Zhang et al. [ [52] include three DL methods but without tuning. Birolo et al. [ [5] evaluate tuned DL methods on six mid-sized biomedical datasets and their study is the only benchmark to report Antolini’s time-dependent C-index [Antolini et al., [3]], which they show is critical for fair evaluation of non-proportional-hazards methods, yet no other benchmark in Tab. A.1 adopts it for evaluation.
Dataset Diversity is Limited in Both Domain and Scale. Burk et al. [ [7] provide perhaps the most comprehensive method comparison to date with 32 datasets across multiple domains and two model selection criteria, but restrict their study to low-dimensional settings () and exclude DL. Herrmann et al. [ [23] cover high-dimensional omics data but include no DL. Birolo et al. [ [5] include tuned DL methods but are limited to six small-to-medium () biomedical datasets. Green et al. [ [20] are the first to leave the biomedical domain entirely and provide by far the largest datasets in any survival benchmark, with 16 tasks containing up to millions of observations. While a valuable contribution to large-scale evaluation, all tasks share the same data-generating process and censoring mechanism, limiting generalizability, and DL methods are evaluated without a systematic tuning protocol and inconsistently on different variants of the C-index – conditions under which DL exhibited particularly poor performance that the authors themselves attribute to tuning difficulties. No existing benchmark simultaneously spans multiple domains, covers both low- and high-dimensional regimes, and includes systematically tuned DL methods.
Evaluation Methodology is Inconsistent and Incomplete. Harrell’s C-index [Harrell et al., [22]] — known to be inappropriate for non-proportional-hazards models [Birolo et al., [5]] — remains the primary evaluation metric in most studies. Moreover, benchmarks disagree on which C-index variant to use: Harrell’s, Uno’s [Uno et al., [48]], Antolini’s [Antolini et al., [3]], or other modified versions. Those C-index variants are known to yield different rankings on identical data [Sierra et al., [44]]. As Lillelund et al. [ [34] observe, the research community remains focused on optimizing the C-index, and Zhou et al. [ [53] find that it serves as the primary performance measure for over 80% of survival analysis papers. This focus continues even though the metric has well-known limitations for non-proportional-hazards models, cannot assess predictive accuracy and offers limited practical usefulness – and despite Herrmann et al. [ [23] recommending already in 2020 that IBS “should be considered more important” given that C-index is not a strictly proper scoring rule. Calibration is absent from several benchmarks entirely. Some benchmarks claim to assess calibration via IBS, but as Chen [ [9] states and we show, calibration needs a distinct focus. No prior benchmark includes a time-accuracy metric, despite Qi et al. [ [42] establishing MAE-PO as a proper scoring rule for this purpose. To our knowledge, however, no benchmark has incorporated it at scale, even though accurate event-time estimation is highly relevant in many real-world applications.
Model Selection Criteria are Rarely Varied. Finally, model selection criteria have been treated as a convention rather than a variable. Only Burk et al. [ [7] experiment with two selection targets, but do not analyze how the choice affects rankings or calibration. No benchmark investigates the interaction between the model selection criteria and the target evaluation metric, which we show is a consequential evaluation design decision.
A consistent substantive finding across all eight benchmarks is that no single method dominates. Notably, Herrmann et al. [ [23] observed already that method rankings change between C-index and IBS — yet no subsequent benchmark has systematically investigated this metric dependence. More consequentially, most studies conclude that the classical (regularized) Cox Proportional Hazards model remains competitive with modern alternatives and recommend it as the practical default – a conclusion we show holds only under specific evaluation and dataset characteristic regimes that prior benchmarks have often implicitly assumed.
Benchmark and Experimental Setup
Datasets
In total, our benchmark comprises 49 survival analysis datasets (Tab. B.1). These were deliberately selected to span a wide range of application domains (clinical, omics, engineering, socio-economics, etc.), sample sizes (26 to 100,000), feature sizes (3 to 15,500), censoring rates (0% to >95%), and time-to-event distributions. The objective is not to evaluate models on a small set of commonly used datasets, but to assess their performance across a sufficiently diverse landscape that reveals strengths and weaknesses under varying data regimes.
We group the datasets into two tiers. Tier 1 comprises 43 datasets from the SurvSet repository [14]. Tier 2 adds six datasets from external sources to cover domains and scales underrepresented in SurvSet and to include datasets widely used as reference points in the survival analysis literature: SUPPORT2 [29], METABRIC [13, 38], KKBox (subsampled to ) [32, 31], MIMIC-IV [25], ADNI [39], and a custom-processed financial dataset from the FinSurvival suite [20].
Selection Criteria. Datasets are included if they contain a continuous time variable with a binary right-censoring indicator, at least two predictive features, static tabular data, and are publicly available (with or without an application process). Unlike Burk et al. [ [7], we impose no minimum event count or dimensionality restriction, deliberately retaining small sample (), high-censoring (up to 95%), and high-dimensional () regimes. Datasets with a sample size exceeding are subsampled with stratification on the event indicator as very large datasets are impractical to run on our standardized and unified training pipeline. The 49 datasets satisfy the minimum of 43 recommended by Boulesteix et al. [ [6] for sufficient statistical power.
Preprocessing. In the preprocessing, observations with non-positive follow-up time are removed. All datasets pass through a unified pipeline: (1) category summarization (collapsing levels with more than 10 categories into "other"), (2) one-hot-encoding with reference-category dropping, (3) -score standardization of continuous features, (4) listwise deletion of remaining missing values, (5) near-zero variance removal (numeric: standard deviation ; one-hot: prevalence ), and (6) high-correlation removal (Spearman ). Tier 2 datasets are not available through SurvSet and require dataset-specific loading and cleaning before entering the shared pipeline. Where an established reference preprocessing exists, we follow it to preserve comparability with prior work. For more details on Tier-2 preprocessing see Appendix B.
Methods
We benchmark 10 time-to-event prediction methods spanning three method families. Comprehensive implementation details, including software libraries, model classes, and exhaustive hyperparameter search spaces, are provided in Appendix § 11 and corresponding Tabs. C.1, C.2.
Linear Cox (2 methods). We evaluate the original unregularized Cox proportional hazards baseline without hyperparameter tuning (abbreviated CoxPH) [12], and CoxNet [45], an elastic net regularized Cox model.
Tree-Based (3 methods). We evaluate Random Survival Forest (RSF) [24] and Gradient Boosting Survival Analysis (GRBoost) [16]. We additionally benchmark XGBoost [10], training it with both Cox and Accelerated Failure Time (AFT) objectives. Survival functions for XGBoost are obtained via the Breslow estimator or a closed-form AFT construction (Appendix § 11.4).
Deep Learning (5 methods). DeepSurv uses a single-hidden-layer MLP trained with the Cox partial log-likelihood and constructs survival curves via the Breslow estimator [27]. DeepHit [33] is a discrete-time model whose loss combines a discrete-time log-likelihood with a ranking loss term, trained across varying temporal discretization schemes. CoxTime [32] extends the Cox framework by making the network a function of both covariates and time. SurvTRACE [50] applies a BERT-style transformer over tabular features, utilizing quantile-based time discretization; it is trained from scratch using a multi-task learning framework that incorporates auxiliary mortality and “length-of-stay” (i.e., survival time) prediction tasks to learn robust shared representations. Finally, CoxKAN [30] replaces the standard MLP in DeepSurv with a Kolmogorov-Arnold Network to produce interpretable symbolic expressions for the log-partial hazard.
Evaluation
Training Protocol. Figure 3.1 shows the setup: each dataset is evaluated using 5-fold cross-validation, stratifying on the binary event indicator. Within each fold, the training set is further split into 80/20 train and validation subsets. Hyperparameters are selected via exhaustive grid search on the validation set to ensure exact reproducibility given the same random seed. Linear Cox and tree-based methods without an internal notion of training progression are refit on the full train+validation set before test evaluation after hyperparameter selection. In contrast, deep learning models and XGBoost include an additional selection step over epochs or boosting rounds, performed via validation-based early stopping (with a patience of 10 epochs/rounds). The resulting checkpoint is used directly for test evaluation without refitting.
Figure 3.1. Benchmark setup overview.
A compute budget of 15 hours per experiment is enforced to ensure tractability of the large-scale exhaustive grid search across all experiments. This threshold was manually chosen empirically from initial runs across datasets of varying sizes. All experiments are run on CPUs (see Appendix § 11.§ 1 for details), as GPU acceleration yielded negligible gains due to evaluation metric computations (C-index, IBS, MAE-PO) being dominated by NumPy/pandas CPU operations.
Triple Model Selection Criteria. Every experiment is evaluated under three distinct validation-based model selection criteria: , IBS, and MAE-PO. Using these three different targets is central to the evaluation design analysis in Section 4: it enables direct comparison of how the same method behaves when its hyperparameters are tuned to optimize different evaluation criteria, isolating the effect of the evaluation design choice from the effect of the method itself.
Evaluation Metrics. On the held-out test set, four metrics are computed for every method, fold, and model selection criterion. To ensure comparability, all metrics are evaluated on a shared time grid, constructed from training follow-up times and trimmed to avoid IPCW boundary instability. All native survival curves – which differ substantially across method families – are mapped to this grid via linear interpolation following Archetti et al. [2024] (full construction details in Appendix § 11.5). The four metrics are:
Discrimination: Antolini (higher is better, abbreviated “↑”) [3] as a time-dependent concordance index that does not assume proportional hazards and – as shown by Birolo et al. [2025] – is better suited for non-proportional-hazards methods. We compute scores via pycox’s EvalSurv Python class [32] with the
adj_antolinitie-handling.
Probabilistic Accuracy: Integrated Brier Score (IBS) (lower is better, ↓) with IPCW weighting [19], computed via SurvivalEVAL [43]. As IBS is dataset dependent, we also report (where refers to the IBS score
for a method being evaluated, and refers to the IBS score achieved by the Kaplan-Meier baseline estimator), measuring percentage improvement over the KM baseline.
Time Accuracy: MAE-PO () – mean absolute error using pseudo-observations to handle censoring [42], computed via SurvivalEVAL [43]; shown by Qi et al. [ [42] to best approximate the true MAE across a variety of censoring distributions. We report , analogously to , and .
Calibration: X-CAL () [18] – a continuous, differentiable relaxation of distributional calibration (D-Calibration) [21] that measures calibration via the probability integral transform. Calibration is also computed using SurvivalEVAL [43]. While D-Calibration yields a binary hypothesis test (calibrated or not), X-CAL provides a continuous score suitable for comparison across methods.
Benchmarking Results
Statistical Evaluation: Bayesian Mixed-Effects Modeling
To rigorously disentangle the performance drivers in our benchmark, we evaluate the results using a Bayesian Generalized Linear Mixed-effects Modeling (GLMM) framework. Simple aggregate rankings often confound an algorithm’s true capability with the baseline difficulty of the datasets it was evaluated on. To correct for this, we model the predictive performance as a function of fixed effects: the learning algorithm, the model selection strategy, dataset meta-features (e.g., dataset observation and feature count, and censoring rate), and their interactions, while utilizing a random intercept for each dataset to absorb intrinsic, unobserved dataset heterogeneity. We have one model for each of IBS, C-index, MAE-PO (ratio), and X-CAL. The governing formula for the linear predictor across all our GLMMs reads:
where:
represents the linear predictor for method , model selection strategy , and dataset (averaged across the 5 cross-validation folds), mapped to the conditional mean via an identity link (Gaussian for and IBS) or a log link (Gamma for X-CAL and MAE-PO ratio).
is the global intercept.
and define the fixed main effects for the learning method and the model selection strategy, respectively, with capturing their two-way interaction.
is a vector of standardized dataset meta-features: log-transformed sample size (), log-transformed feature count (), log-transformed time range (), and the censoring rate.
is a vector representing the fixed main effects of these meta-features.
captures the interaction between method and the dataset meta-features, quantifying how specific architectures scale with sample size, dimensionality, or censoring.
is a random intercept for dataset , absorbing unobserved dataset-specific baseline difficulty.
Models were fitted using Hamiltonian Monte Carlo via the bambi [8] and PyMC [1] libraries. This fully probabilistic approach allows us to report the Probability of Direction () [35] for specific performance gains and to perform variance partitioning: estimating the relative contributions of combined fixed effects, dataset-level random effects, and residual variation to modeled performance variance. Probability of Direction (), posterior predictive checks, and the variance decomposition underlying Figure 4.1 are detailed in Appendices D.1.2, D.1.4, and D.1.3, respectively.
Figure 4.1. Bayesian Variance Partitioning Across Evaluation Metrics with 94% highest-density interval. † denotes log scale.
Run Completion and Failure Analysis
At the scale of this benchmark, incomplete runs are expected: not all 1,470 experiments completed successfully. Our main analyses are performed on the unbalanced union of successful runs.
We mark a dataset-method pair as failed if any fold did not produce results. The failures are not random but reflect interpretable scalability limits of specific architectures: mathematical errors (e.g. unregularised CoxPH on ill-conditioned design matrices), compute budget timeouts (e.g. RSF, GrBoost, CoxKAN), and memory exhaustion (e.g. CoxTime, SurvTRACE). Failure counts also differ marginally across selection criteria (: 48, IBS: 50, MAE-PO: 56), as some validation metrics are more expensive to compute. Appendix § 13.§ 5 provides a detailed breakdown of runtime, memory, and numerical failures, distinguishing timeouts shared across all model-selection criteria from those specific to individual criteria. Figure E.9 illustrates these systematic patterns for -based model selection, showing how failures concentrate in specific method–regime combinations.
To verify that the systematic failures do not harm our analysis methods through survivorship bias, especially the Bayesian GLMM, we also conducted the same analysis on the common intersection of successful runs, restricting the analysis to datasets on which all methods completed successfully. The overarching trends and main findings remain completely unchanged and we conclude that our results are robust in these terms. Detailed results are provided in Appendix § 13.4, Fig. E.10.
Key Findings
The following findings A and B should not be interpreted independently. Instead, they form an interdependent evaluation protocol. Practitioners should first determine what they aim to optimize through an application-specific evaluation objective, as this influences the different performance metrics on the test set.
A The Evaluation vs. Optimization Gap: Moving Beyond C-index Selection
Our Bayesian GLMM analysis reveals a profound dichotomy in how performance variance is explained across different evaluation metrics, demonstrating that standard C-index selection systematically masks the true capabilities of complex architectures (Figure 4.1).
A.1 What Drives Performance Depends on the Metric. For discriminative performance (C-index), a model’s performance is largely predetermined by the dataset heterogeneity. It acts as an overpowering constraint, explaining 67.4% of the variance, while the combined fixed effects of model choice, selection strategy, and dataset meta-features account for only 23% (Figure 4.1). Consequently, for practitioners primarily concerned with ranking, the choice of algorithm seems less important, gaining importance only in large and large regimes (see Fig. E.3, E.1, and D.3).
Conversely, probabilistic accuracy (IBS) and calibration (X-CAL) are highly susceptible to algorithmic intervention. For IBS, our controlled choices explain the majority of the variance (52.6%), and for X-CAL, this rises dramatically to 70.9%. This indicates that while discrimination is largely a property of the data, proper calibration and probabilistic scoring are fundamentally properties of the model. Therefore, adopting a targeted, dataset-regime-specific model selection strategy is not merely an optimization detail, but a critical requirement for unlocking substantial performance gains.
A.2 The Benefits of Proper Scoring Rules as Model Selection Criteria. Our analysis reveals a profound asymmetry in model selection: optimizing for strictly proper scoring rules unlocks significant performance gains for highly flexible models without sacrificing their discriminative ranking.
The C-index evaluates concordance but ignores the scale of predicted probabilities, allowing models to generate wildly uncalibrated distributions. In contrast, optimizing hyperparameters for IBS forces models to learn well-calibrated probabilistic distributions, while largely preserving their ranking capabilities in our benchmark.
This phenomenon is strikingly evident in complex methods. As shown in Table 4.1, when shifting the optimization metric from C-index to IBS, XGBoost exhibits a substantial median IBS improvement of -0.083 () and SurvTRACE improves by -0.068 (). Crucially, this massive gain in probabilistic accuracy comes at no cost to discrimination, as their corresponding C-index shifts remain statistically negligible (). In contrast, standard C-index selection does not adequately regularize flexible models, leading to highly uncalibrated predictions.
| Method | C-index → IBS: C-index ↑ | C-index → IBS: IBS ↓ | C-index → IBS: MAE† ↓ | C-index → IBS: X-CAL† ↓ | C-index → MAE: C-index ↑ | C-index → MAE: IBS ↓ | C-index → MAE: MAE† ↓ | C-index → MAE: X-CAL† ↓ | IBS → MAE: C-index ↑ | IBS → MAE: IBS ↓ | IBS → MAE: MAE† ↓ | IBS → MAE: X-CAL† ↓ |
| coxnet | −0.009* | −0.002 | 0.163 | −0.157 | −0.002 | −0.002 | −0.420 | 0.052 | 0.007 | −0.001 | −0.584* | 0.208* |
| rsf | −0.000 | −0.001 | −0.007 | −0.039 | −0.001 | −0.001 | −0.014 | −0.057 | −0.000 | 0.000 | −0.005 | −0.018 |
| grboost | 0.000 | −0.001 | 0.004 | −0.034 | −0.006 | 0.000 | 0.008 | 0.073 | −0.006 | 0.001 | 0.002 | 0.107 |
| xgboost | 0.001 | −0.083*** | 0.386 | −2.521*** | 0.000 | −0.068*** | −1.307*** | −1.725*** | −0.000 | 0.015*** | −1.690*** | 0.796*** |
| deepsurv | 0.009* | −0.020*** | −0.867** | −0.311** | 0.011* | −0.026*** | −4.474*** | −0.570*** | 0.001 | −0.007 | −3.608*** | −0.259** |
| deephit | −0.001 | −0.032*** | −0.358 | −0.751*** | −0.007 | −0.007* | −0.000 | 0.352*** | −0.006 | 0.024*** | 0.366 | 1.102*** |
| survtrace | 0.002 | −0.068*** | 0.180 | −1.948*** | −0.003 | −0.049*** | −0.475 | −1.092*** | −0.004 | 0.019*** | −0.650* | 0.854*** |
| coxkan | −0.006 | −0.005 | −2.652*** | −0.080 | −0.010* | −0.003 | −2.651*** | −0.010 | −0.004 | 0.002 | 0.002 | 0.071 |
| coxtime | 0.002 | −0.020*** | −0.987** | −0.548*** | 0.011* | −0.027*** | −3.984*** | −0.846*** | 0.009 | −0.007 | −2.986*** | −0.299** |
Table 4.1. Effects of model selection strategy on target metrics, when switching selection strategy From → To. Insignificant results () are kept colorless. Significant results are color-coded (light green for beneficial, light red for detrimental) and marked with asterisks (* , ** , *** ). A dagger (†) denotes metrics that are reported on a log-multiplier scale.
Time-to-Event Accuracy Requires its Own Model Selection Approach
The necessity of target-specific selection extends to absolute time-to-event accuracy (MAE-PO). Unlike and IBS, which are inherently bounded, MAE-PO is expressed in native time units and is therefore unbounded, meaning a single strongly mispredicted survival time can inflate the error substantially. Due to heavy outliers and noise, MAE-PO variance is harder to explain (42.7% residual variance), but the choice of selection criterion remains highly consequential. If model selection relies on C-index or IBS, whether a model’s MAE-PO improves over the Kaplan–Meier (KM) baseline [26] is essentially a coin flip (50%) (see Tab. E.4). Selecting models explicitly to optimize MAE-PO raises this success rate (i.e., rate of outperforming the KM baseline) to 72% while simultaneously curbing severe prediction outliers (see Tab. E.4). Furthermore, Table 4.1 demonstrates that transitioning from C-index to MAE-PO selection drastically reduces temporal errors for DeepSurv by 99% and CoxTime by 98% (calculated from the log-multiplier scale). We conclude that researchers must move beyond C-index-only optimization to accurately assess a modern architecture’s full capabilities beyond ranking.
Once the evaluation objective and corresponding model selection criterion have been established, the remaining question is which methods are expected to perform best under the characteristics of the target dataset. We therefore turn to regime-conditioned analyses that explain how benchmark conclusions change across different dataset regimes. Practitioners can use these findings to narrow down a suitable method for their specific setup.
Contextualizing Performance: The Role of Dataset Regimes
In practice, a model’s performance is deeply tied to dataset properties, including scale, dimensionality, and censoring rates. Through our Bayesian GLMM, we isolate these meta-features to show how model performance interacts with distinct data regimes. We highlight the danger of relying on aggregate summaries by visualizing the severe rank shifts that occur between small and large datasets. Furthermore, we expose a structural vulnerability in absolute metric evaluation, demonstrating that heavy censoring renders X-CAL scores artificially conservative.
Dataset Scale and Dimensionality Govern Method Performance
The structural properties of a dataset fundamentally dictate which architectural family will succeed. Regarding sample scale (log ), highly flexible architectures exhibit strong, consistent positive scaling for discriminative performance (Fig. D.4). As sample sizes increase, models like XGBoost, DeepHit, and SurvTRACE show substantial IBS improvements with median posterior effects ranging from +0.041 to +0.049 (all ). While all models seem to benefit from larger sample sizes, we have to emphasize that their effect strengths are vastly different. This scaling effect is so severe that it entirely upends model rankings: as demonstrated in Figure 4.2, DeepHit leaps from near last place on small datasets to first place on large-scale datasets. Please note that methods that did fail on large datasets were automatically assigned the worst rank for that specific dataset.
Figure 4.2. Average ranks (lower is better) for small () vs large () datasets.
However, this architectural advantage reverses in high-dimensional feature spaces (). Our posterior projections reveal a severe vulnerability of deep learning models to high dimensionality concerning probabilistic accuracy (Figure 4.3). DeepHit (+0.026), CoxKAN (+0.027), and SurvTRACE (+0.02) all exhibit significant IBS deterioration with certainty (). This suggests that while deep survival models effectively leverage large sample sizes to improve ranking, they lack the regularization required to control their predictive distributions for high-dimensional settings. We further investigate the effect of regularization through an additional dropout ablation study on a representative high-dimensional dataset, presented Appendix § 13.§ 16.1, Table E.5, where the ablation study indicates that regularization in general can be beneficial.
Figure 4.3. Effects of changing the Log-Feature size by one standard deviation for each method on target metrics. Significant () results are colored. Red: detrimental effect, green: beneficial effect. A dagger (†) denotes metrics that are reported on a log-multiplier scale.
The Calibration Fallacy: Relative vs Absolute Utility
Our analysis exposes a critical structural vulnerability in standard survival evaluation practices: raw absolute calibration metrics are heavily biased by the underlying censoring regime. Our Bayesian GLMM reveals a systematic negative effect of the censoring rate on X-CAL (Fig. D.5). As censoring increases, a model’s apparent calibration artificially improves. This phenomenon can be explained by a mathematical “smearing effect”: survival probabilities are distributed over unobserved time horizons, artificially smoothening the distribution towards good calibration.
While X-CAL remains a robust and highly informative metric for the relative ranking of models on the exact same dataset, its absolute value becomes unreliable under heavy censoring. Consequently, standard D-Calibration tests become overly conservative and susceptible to false negatives. As shown in Table 4.2, the rejection rate for miscalibration in DeepHit drops from 33.8% on low-censoring datasets () to just 13.3% on high-censoring datasets (). Practitioners must be wary: a model is not necessarily well-calibrated simply because a test fails to reject it on a highly censored dataset.
| Method | Cens. | Cens. |
| CoxNet | 24.6% | 0.0% |
| DeepSurv | 27.7% | 0.0% |
| DeepHit | 33.8% | 13.3% |
Table 4.2. D-Calibration rejection rates at low versus high censoring levels.
This finding matches the identified flaws of D-Calibration in a recently published paper by Simonsen and Waagepetersen [2025].
Conclusion
The prevailing consensus that classical Cox models remain competitive is an artifact of narrow evaluation conventions, not an absolute methodological truth. By benchmarking 10 methods across 49 diverse datasets under varying model selection criteria, we demonstrate that survival model performance is fundamentally regime-dependent and that the selection metric strongly shapes model behavior.
Beyond Single-Number Summaries. We challenge the convention of summarizing survival method performance reporting through global average summaries, particularly when models are uniformly optimized on the C-index but evaluated across distinct metrics. While average summary statistics provide a convenient overview (we provide standard heatmaps and average ranks in Appendix E.1 for reference), they are inherently shaped by the structural composition of the benchmark, including the distribution of dataset scales, dimensionalities, and censoring rates. As our Bayesian GLMMs show, highly flexible methods benefit strongly from IBS-based selection, a dynamic completely hidden under standard C-index optimization. Rankings also shift dramatically with dataset scale: DeepHit, for instance, moves from second last place on average-sized datasets to first on large-scale ones when using IBS as the model selection criterion (see Figure 4.2). Global rankings should therefore be treated as a description of the benchmark corpus, not as intrinsic properties of the algorithms. We advocate for regime-conditioned reporting: stratifying performance by dataset characteristics and evaluation target, and treating model selection criteria as an independent experimental variable.
Practical Recommendations. Based on our findings, we offer the following guidance for practitioners: CoxNet remains a robust default choice, especially on small to mid-sized datasets, and offers interpretability. On large datasets, DeepHit, DeepSurv, and XGBoost consistently outperform Linear Cox models B.1. When accurate event-time prediction matters, MAE-PO-based model selection is essential A.3; SurvTRACE performs particularly well under this regime, possibly due to its auxiliary “length-of-stay” objective (essentially a squared error term for predicted survival times). RSF provides the strongest trade-off between discrimination, calibration, and robustness, with stable MAE-PO and no observed outliers, but it does not scale well to datasets with very large sample size . In high-dimensional regimes, RSF, CoxNet, and GrBoost remain reliable, while DL methods often degrade or even fail due to compute constraints B.1. IBS-based selection consistently improves calibration A.2. Under high censoring, practitioners should be aware of the Calibration Fallacy B.2. For interpretable yet flexible modeling, CoxKAN provides symbolic log-hazard expressions, while being a competitive and stable neural method, especially for ranking assessment.
Limitations. The study is subject to several limitations. The benchmark, despite containing 17 more datasets than the largest previous benchmark, particularly expanding in high-dimensional and large- regimes, remains overall skewed toward small and mid-sized datasets. A uniform CPU-only environment was fixed for all methods, and a fixed per-fold compute budget was enforced due to the model selection metrics being CPU-bound. Some aspects are especially detrimental to deep learning methods. Hyperparameter grids were informed by published papers, reference implementations, and prior benchmarks to the best of our knowledge, but may nonetheless miss optimal configurations and systematically underestimate individual methods’ potential. Lastly, the benchmark is restricted to right-censored, single-event, static tabular data.
Outlook. Our findings suggest that time-to-event prediction needs to be rethought end-to-end, not only in terms of which metrics are reported, but also how model selection is performed and how metric implementations are engineered to make proper scoring rules practical at scale. Currently, probabilistic accuracy and time-to-event accuracy still diverge under optimization: improving one can degrade the other, forcing target-specific selection rather than yielding jointly accurate predictions. Closing this gap, whether through joint losses, unified metrics, or architectures that structurally couple calibration with temporal precision, remains an open challenge. We showed that aligning the selection criterion with the target application can unlock larger gains than switching to a more complex architecture. The importance of these findings will only grow as foundation models enter survival analysis and datasets scale by orders of magnitude.
Related Work
| Study | #Meth. | #DL | #Data | HP Tuning | Selection Criteria | Prob. Acc | Time Acc | Cal | Impl. | |||
| Herrmann et al. [23] | 13 | 0 | 18 | (35, 735) | (90154, 103369) | Yes | Internal Objective | No | Yes | No | No | R |
| Zhang et al. [52] | 22 | 3 | 16 | (41, 3323) | (7, 19818) | No | - | No | Yes | No | No | R |
| Abubakari et al. (2024) | 9 | 1 | 4 | (58, 228) | (8, 19818) | No | - | No | No | No | No | R |
| Chien et al. [11] | 7 | 3 | 6 | (985, 10536) | (25, 100) | No | - | No | No | No | No | - |
| Báskay et al. (2025) | 5 | 1 | 4 | (137, 9105) | (6, 40) | Yes | Modified C-index | No | No | No | No | Python |
| Birolo et al. [5] | 8 | 3 | 6 | (1903, 6524) | (8, 9) | Yes | Yes | Yes | No | No | Python | |
| Green et al. (2025) | 5 | 2 | 16 | (208939, 644296) | 128 | Yes | only DL-HP selection | No | No | No | No | R |
| Burk et al. [7] | 19 | 0 | 32 | (137, 52422) | (2, 26) | Yes | C-index, IBS | No | Yes | No | Yes | R |
| Ours (2026) | 10 | 5 | 49 | (26, 100000) | (3, 15500) | Yes | , IBS, MAE-PO | Yes | Yes | Yes | Yes | Python |
Table A.1. Summary of existing survival analysis benchmarking studies.
Datasets
Table B.1: Overview of all 49 datasets included in the benchmark. denotes the number of observations, the number of features after preprocessing, and Cens. the percentage of right-censored observations. Tier 2 datasets are marked with †.
| Dataset | Source | Cens. (%) | Domain | ||
| acath | SurvSet | 2,230 | 3 | 34 | Clinical |
| ADNI† | ADNI | 760 | 29 | 63 | Clinical |
| Aids2 | SurvSet | 2,810 | 11 | 38 | Clinical |
| AML_Bull | SurvSet | 115 | 6,266 | 43 | Omics |
| borrow_repay† | FinSurvival | 54,147 | 70 | 40 | Financial |
| breast | SurvSet | 100 | 4 | 74 | Clinical |
| burn | SurvSet | 154 | 13 | 69 | Clinical |
| cancer | SurvSet | 169 | 18 | 28 | Clinical |
| cgd | SurvSet | 128 | 17 | 66 | Clinical |
| chop | SurvSet | 412 | 3,833 | 60 | Omics |
| cost | SurvSet | 518 | 13 | 22 | Clinical |
| dataDIVAT1 | SurvSet | 5,943 | 14 | 84 | Clinical |
| dataDIVAT3 | SurvSet | 4,267 | 16 | 94 | Clinical |
| DBCD | SurvSet | 295 | 4,912 | 73 | Omics |
| Dialysis | SurvSet | 6,805 | 16 | 76 | Clinical |
| DLBCL | SurvSet | 240 | 7,343 | 43 | Omics |
| divorce | SurvSet | 3,371 | 4 | 69 | Socioeconomic |
| e1684 | SurvSet | 284 | 3 | 31 | Clinical |
| flchain | SurvSet | 6,521 | 25 | 70 | Clinical |
| Framingham | SurvSet | 4,658 | 16 | 69 | Clinical |
| gse1992 | SurvSet | 123 | 15,500 | 72 | Omics |
| gse3143 | SurvSet | 158 | 8,491 | 68 | Omics |
| gse4335 | SurvSet | 113 | 12,769 | 66 | Omics |
| hdfail | SurvSet | 52,422 | 15 | 95 | Engineering |
| hepatoCellular | SurvSet | 101 | 42 | 55 | Clinical |
| kkbox_100k† | pycox | 100,000 | 27 | 35 | Commercial |
| LeukSurv | SurvSet | 1,043 | 16 | 16 | Clinical |
| MCLcleaned | SurvSet | 92 | 574 | 30 | Omics |
| metabric† | cBioPortal | 1,979 | 13 | 42 | Clinical |
| micro.censure | SurvSet | 117 | 80 | 78 | Omics |
| mimic_iv† | PhysioNet | 54,724 | 66 | 91 | Clinical |
| nki70 | SurvSet | 144 | 76 | 67 | Omics |
| NSBCD | SurvSet | 115 | 549 | 67 | Omics |
| nwtco | SurvSet | 4,028 | 9 | 86 | Clinical |
| ovarian | SurvSet | 26 | 4 | 54 | Clinical |
| pbc | SurvSet | 312 | 6 | 60 | Clinical |
| pharmacoSmoking | SurvSet | 113 | 15 | 32 | Socioeconomic |
| prostateSurvival | SurvSet | 14,065 | 6 | 94 | Clinical |
| rhc | SurvSet | 634 | 64 | 58 | Clinical |
| scania | SurvSet | 1,931 | 8 | 44 | Engineering |
| smarto | SurvSet | 1,021 | 31 | 89 | Omics |
| support2† | SUPPORT | 8,873 | 19 | 32 | Clinical |
| UnempDur | SurvSet | 3,241 | 6 | 39 | Socioeconomic |
| Unemployment | SurvSet | 450 | 7 | 44 | Socioeconomic |
| vdv | SurvSet | 78 | 4,672 | 56 | Omics |
| veteran | SurvSet | 137 | 8 | 7 | Clinical |
| vlbw | SurvSet | 257 | 30 | 86 | Clinical |
| Z243 | SurvSet | 100 | 15 | 0 | Clinical |
| zinc | SurvSet | 431 | 15 | 81 | Clinical |
Table B.1. Overview of all 49 datasets included in the benchmark. denotes the number of observations, the number of features after preprocessing, and Cens. the percentage of right-censored observations. Tier 2 datasets are marked with †.
Preprocessing Details - Tier 2 Datasets. Tier 2 datasets undergo dataset-specific preprocessing before entering the shared pipeline. For METABRIC, this includes binary encoding of receptor status, grade normalization, and median imputation of missing clinical values. For KKBox, missing gender and city values are filled with placeholder categories, and remaining numeric missingness is median-imputed. SUPPORT2 follows the established preprocessing from Katzman et al. [27], which manually selects 14 features and applies its own scaling and encoding - it bypasses the shared pipeline but still passes through the NZV and high-correlation filters. MIMIC IV preprocessing follows Tang et al. [47], extracting 26 features from the first 24 hours of ICU data. In all cases, the dataset-specific steps produce a dataframe with num_/fac_ prefixed columns, which then enters the shared pipeline.
ADNI. The Alzheimer’s Disease Neuroimaging Initiative (ADNI) is a longitudinal, multi-site study launched in 2003 with the objective of determining whether serial MRI, PET, CSF biomarkers, and clinical assessments can be combined to measure the progression of mild cognitive impairment (MCI) and early Alzheimer’s disease [39]. The raw data requires an application process but can be obtained for free after passing it. Brain volumetric features were obtained through FreeSurfer [15]. The event of interest is conversion from MCI to Alzheimer’s disease, with a censoring rate of 62.9%. The mean age is 73.5 years, 60.4% are male, and the median follow-up time is 2 years.
Borrow-Repay. To include a financial survival analysis setting, we include a dataset from the FinSurvival suite by Green et al. [ [20], which models among other things lending and borrowing events in decentralized finance. The original FinSurvival benchmark uses predefined train/test splits that are incompatible with our cross-validation protocol and carries a risk of entity-level leakage (multiple observations per user). We therefore merge the original splits into a single dataset, restrict to the first transaction per user, and drop user-history features that become degenerate under this restriction and that are uninformative at a user’s first interaction (e.g., time since first transaction). Time is converted from seconds to days. This substantially changes the dataset characteristics: the censoring rate shifts from approximately 14% to approximately 40%, and the task shifts from predicting repayment from behavioral history to predicting it from loan characteristics (amount or interest rate), market-level activity, and temporal context (e.g., time of the day).
KKBox. The KKBox dataset models customer churn for the KKBox music streaming service, where the event of interest is subscription cancellation. It originates from the WSDM KKBox Churn Prediction Challenge and was adapted for survival analysis by Kvamme et al. [Kvamme et al., [32]; Kvamme and Borgan [31]. The data is accessed via pycox.datasets.kkbox
With subjects and features, KKBox is by far the largest dataset in the benchmark; a subsampled version with is used for the standardized evaluation, following Alberge et al. [ [2]. Before the shared pipeline, missing categorical values are filled with placeholder categories (gender → "n/a", city and registration channel → "0"), and remaining numeric missingness is median-imputed.
METABRIC. The Molecular Taxonomy of Breast Cancer International Consortium (METABRIC) dataset was obtained from cBioPortal2 [Curtis et al., 2012], [Pereira et al., 2016]. Clinical data and PAM50 gene expression measurements (50 genes; see below) were downloaded separately and merged on sample identifier. The cohort comprises 1,980 breast cancer patients after outcome filtering, with overall survival as the endpoint (event rate 57.8%, median follow-up 116.5 months). The raw clinical features are age at diagnosis, tumor size, number of positive lymph nodes, neoplasm histologic grade, and receptor status for ER, PR, and HER2. Before the shared pipeline, receptor status variables are encoded as binary numerics, histologic grade is normalized to via and excluded from z-score scaling, and missing clinical values are median-imputed. Rows with missing gene expression are dropped.
The PAM50 gene signature consists of 50 genes commonly used for breast cancer subtype classification. The following list corresponds to the genes used in the METABRIC dataset variants.
UBE2T
BIRC5
NUF2
CDC6
CCNB1
TYMS
MYBL2
CEP55
MELK
NDC80
RRM2
UBE2C
CENPF
PTTG1
EXO1
ORC6
ANLN
CCNE1
CDC20
MKI67
KIF2C
ACTR3B
MYC
EGFR
KRT5
PHGDH
CDH3
MIA
KRT17
FOXC1
SFRP1
KRT14
ESR1
SLC39A6
BAG1
MAPT
PGR
CXXC5
MLPH
BCL2
MDM2
NAT1
FOXA1
BLVRA
MMP11
GPR160
FGFR4
GRB7
TMEM45B
ERBB2
MIMIC IV. The MIMIC dataset is derived from the publicly available Medical Information Mart for Intensive Care database and provides anonymized clinical data of patients admitted to an intensive care unit (ICU) [47]. Access requires completion of a CITI Program training course4, after which the data can be obtained from PhysioNet5. MIMIC-IV requires credentialed access, therefore we solely provide the preprocessing steps and no raw MIMIC data. Preprocessing follows Tang et al. [ [47], originally described for MIMIC-III, and Purushotham et al. [ [41] for MIMIC-IV. The cohort comprises adult patients () at their first ICU stay who are alive 24 hours after admission. The event of interest is in-hospital mortality, with the event time observed if a death record exists in the database and the censored time defined as the last hospital discharge time otherwise. Time is measured in hours from ICU admission, with administrative censoring applied at 30 days (720,h). 28 features are extracted from the first 24 hours of clinical data: 7 vital signs from chart events and 12 laboratory values - each mean-aggregated over the observation window - plus 8 demographic categoricals and age. Numerical features with >50% missingness are dropped; missing categoricals are filled with a placeholder category; remaining numeric missingness is handled via listwise deletion. The resulting dataset contains subjects, encoded features, and a censoring rate of 91.2%, providing a large-scale ICU mortality prediction setting.
SUPPORT2. The SUPPORT2 dataset originates from the Study to Understand Prognoses Preferences Outcomes and Risks of Treatments and models time to death for seriously ill hospitalized patients [29, 27]. Although a version of this dataset exists in SurvSet, we use the established preprocessing from Katzman et al. [ [27] which is implemented in the survival analysis introduction repository by Chen [ [9]. It selects 14 clinically motivated features (age, sex, race, number of comorbidities, diabetes, dementia, cancer status and seven physiological measure). This preprocessing applies its own encoding and scaling: sex is binarized, cancer status mapped to an ordinal scale and normalized by its maximum, number of comorbidities divided by 9, eight continuous features z-scored, and race one-hot encoded without dropping a reference category. Rows with any missing values are dropped. After encoding, the dataset contains subjects with features and an event rate of 68.0%. Time is measured in days.
Methods, Implementations and Evaluations
Reproducibility
All experiments were conducted using Python 3.10.19 on Linux x86_64 (glibc 2.31) on a CPU-based machine (32 CPU Cores, 64GB RAM). Key dependencies include PyTorch 2.5.1, NumPy 1.26.4, pandas 2.3.3, scikit-learn 1.7.2, pycox 0.2.3, and torchtuples 0.2.2.
Method Variants and (Hyper)parametrization
This section details the specific implementation and parameterization logic for the evaluated methods. Tab. C.1 links each method to its underlying Python library and model class, while Tab. C.2 explicitly defines the hyperparameter search spaces ().
For all deep learning architectures and XGBoost, checkpoint selection is performed dynamically via validation-based early stopping (patience = 10 epochs/rounds) to prevent overfitting.
Relevant libraries included in our experiments are survhive [5], scikit-survival [40], xgboost [10] and xgbse [49], pycox [32], survtrace [50] and coxkan [30].
CoxPH [12]: Unregularized, hyperparameter-free baseline accessed via
scikit-survivalandsurvhive.CoxNet [45]: Elastic net regularized Cox via
scikit-survivalandsurvhive. The regularization trade-off (l1_ratio) is tuned from Ridge to Lasso, utilizing a data-dependent penalty path for alpha.RSF [24] & GrBoost [16]: Random Survival Forest and Gradient Boosting via
scikit-survivalandsurvhive. Both tune structural tree-ensemble parameters (estimators, depth, split/leaf constraints).XGBoost [10]: Native
xgboostimplementation with survival functions extracted viaxgbse. Optimized under both Cox and Accelerated Failure Time (AFT) objectives. Survival functions are constructed via the Breslow estimator (Cox) or a closed-form distribution (AFT, detailed in Appendix § 11.4).DeepSurv [27]: Single-hidden-layer Cox MLP implemented via
pycox.DeepHit [33]: Discrete-time network via
pycox. The temporal discretization strategy (unique event times vs. 128 quantile-based bins) is treated as a structural hyperparameter, alongside the ranking loss weight alpha.CoxTime [32]: Time-dependent neural network via
pycox, utilizing case-control sampling to dynamically approximate the partial likelihood.SurvTRACE [50]: Tabular transformer via
survtrace. Treats numerical features as individual tokens mapped to multi-head self-attention and utilizes quantile-based time discretization.CoxKAN [30]: Kolmogorov-Arnold Network via
coxkan. Replaces standard linear layers with learnable univariate activations on spline grids, tuned via the grid andbase_funparameters.
Hyperparameter Search Spaces
Hyperparameter grids were designed from published method papers and are stored in per-method YAML configuration files. Experiments were executed exclusively on CPUs without GPU acceleration. Depending on the method and dataset, the average runtime per hyperparameter configuration ranged from below one second for classical methods to several seconds for deep learning approaches, with an overall mean runtime of approximately 2 seconds per hyperparameter combination across all evaluated methods and datasets.
Table C.1: Overview of all methods included in the benchmark. Family distinguishes linear Cox, tree-based, and DL. Library indicates the Python package used for model training; Model Class specifies the concrete class or API call. DeepHit is trained with two discretization variants (unique event times and 128 quantile-based bins). The libraries include survhive [5], scikit-survival [40], xgboost [10] and xgbse [49], pycox [32], survtrace [50] and coxkan [30].
| Method | Family | Library | Model Class | Reference |
| CoxPH | Cox Linear | survhive | CoxPH | scikit-survival |
| CoxNet | Cox Linear | survhive | CoxNet | scikit-survival |
| RSF | Tree | survhive | RSF | scikit-survival |
| GradBoost | Tree | survhive | GrBoostSA | scikit-survival |
| XGBoost | Tree | xgboost + xgbse | xgb.train (survival:cox, survival:aft) | xgboost |
| DeepSurv | DL | pycox | CoxPH | pycox |
| DeepHit | DL | pycox | DeepHitSingle | pycox |
| SurvTRACE | DL | survtrace | SurvTraceSingle | survtrace |
| CoxKAN | DL | coxkan | CoxKAN | coxkan |
| CoxTime | DL | pycox | CoxTime | pycox |
Table C.1.
XGBoost
Neither XGBoost objective produces survival functions directly; we construct them from raw model outputs.
Cox objective: The model outputs a linear predictor . We estimate the cumulative baseline hazard via the Breslow estimator at each unique training event time and obtain individual survival curves.
AFT objective: The tree ensemble predicts , the expected log survival time, under the model . With , the following formula shows how to derive . Starting from and substituting the AFT model:
where is the CDF of the chosen error distribution. This yields the following closed-form survival functions per distribution after plugging in the respective CDF:
where the extreme distribution corresponds to the standard Gumbel (minimum) distribution. The scale parameter the distribution choice, and their corresponding PDFs (Tab. C.3) are all treated as hyperparameters in the grid search.
| Method | Hyperparameter | Search space | |
| CoxPH | alpha | {0.0} | |
| CoxPH | ties | {efron} | 1 |
| CoxNet | l1_ratio | {0.05, 0.1, 0.25, 0.5, 0.75, 0.9, 0.99} | |
| CoxNet | alpha | data-dependent path | var. |
| RSF | n_estimators | {50, 100, 300} | |
| RSF | max_depth | {--, 3, 6, 10} | |
| RSF | min_samples_split | {0.01, 0.1} | |
| RSF | min_samples_leaf | {0.05, 0.2} | 48 |
| GrBoost | n_estimators | {50, 100, 300} | |
| GrBoost | max_depth | {--, 3, 6, 10} | |
| GrBoost | min_samples_split | {0.01, 0.1} | |
| GrBoost | min_samples_leaf | {0.05, 0.2} | 48 |
| XGBoost-Cox | learning_rate | {0.1, 0.3} | |
| XGBoost-Cox | max_depth | {--, 6, 10} | |
| XGBoost-Cox | num_boost_round | {500, 1000} | 12 |
| XGBoost-AFT | aft_loss_distribution | {normal, extreme} | |
| XGBoost-AFT | aft_loss_distribution_scale | {0.5, 1.0, 2.0} | |
| XGBoost-AFT | learning_rate | {0.1, 0.3} | |
| XGBoost-AFT | max_depth | {--, 6, 10} | |
| XGBoost-AFT | num_boost_round | {500, 1000} | 72 |
| DeepSurv | hidden | {16, 128, 1024} | |
| DeepSurv | learning_rate | {0.001, 0.01, 0.1} | |
| DeepSurv | batch_size | {256, 2048} | |
| DeepSurv | dropout | {0.0, 0.2} | 36 |
| DeepHit | hidden | {16, 128, 1024} | |
| DeepHit | learning_rate | {0.001, 0.01, 0.1} | |
| DeepHit | batch_size | {256, 2048} | |
| DeepHit | dropout | {0.0, 0.2} | |
| DeepHit | alpha | {0.2, 1.0} | 72 |
| SurvTRACE | hidden_factor | {4, 16} | |
| SurvTRACE | num_attention_heads | {2, 4} | |
| SurvTRACE | hidden_dropout | {0.0, 0.2} | 8 |
| CoxKAN | lr | {0.001, 0.01, 0.05} | |
| CoxKAN | base_fun | {silu, linear} | |
| CoxKAN | num_hidden | {0, 1} | |
| CoxKAN | grid | {3, 5} | |
| CoxKAN | batch_size | {256, 2048} | 48 |
| CoxTime | hidden | {16, 128, 1024} | |
| CoxTime | learning_rate | {0.001, 0.01, 0.1} | |
| CoxTime | batch_size | {256, 2048} | |
| CoxTime | dropout | {0.0, 0.2} | 36 |
Table C.2. Hyperparameter search spaces for all methods.
| AFT loss distribution | Probability Density Function |
| Normal | |
| Logistic | |
| Extreme |
Table C.3. AFT loss distributions and their probability density functions directly taken from the xgboost for Survival Analysis-Documentation.6
Evaluation
Time grid construction. The evaluation time grid is constructed from all unique training follow-up times (events and censored observations), trimmed to the 1st and 99th percentile of training times to avoid boundary instability of the IPCW weights, and constrained to the intersection of training and test time ranges. If the grid exceeds 500 points, it is subsampled to 500 via uniform spacing across unique times. All native survival curves – which differ substantially across method families (CoxPH: training event times; DeepHit: discrete bins; SurvTRACE: quantile cuts) – are mapped to this shared grid via linear interpolation with anchors at and , following the implementation recommendations of Archetti et al. [2024].
Two baselines are computed per fold: a random baseline for all patients and times) and a Kaplan-Meier baseline fitted on the training data. Both are evaluated on the same grid and serve as a reference points for and .
Analysis Methods
Bayesian GLMMs
Bayesian Generalized Mixed Effects Model (GLMM)
To rigorously isolate the effects of algorithmic choice, model selection strategy, and dataset characteristics on predictive performance, we employed a Bayesian mixed-effects modeling framework. This approach estimates the marginal effects of algorithms and optimization strategies while explicitly accounting for the varying baseline difficulties across our evaluation corpus.
Based on the geometric constraints of our evaluation metrics, we utilize the following likelihoods and link functions:
Discrimination and Brier Scores (Gaussian): For the C-index and IBS, we utilized a Gaussian likelihood with an identity link function. While these metrics are bounded on [0, 1], rigorous posterior predictive checks (D.1.4) demonstrated that a Gaussian model provides an excellent approximation with minor boundary violations. The model is formulated as , where . This preserves the direct, additive interpretability of the marginal effects.
Calibration and Absolute Error (Gamma): The X-CAL and MAE-PO metrics are strictly positive () and heavily right-skewed. To ensure stable inference and capture this geometry, we modeled these targets using a Gamma distribution parameterized by its mean and a shape parameter , paired with a log link function. The generative model is , where . Consequently, additive covariate effects in translate to multiplicative scaling effects on the expected error.
All Bayesian models were fitted using Hamiltonian Monte Carlo via the bambi [8] and PyMC [1] libraries in Python. Used versions: pymc 5.28.4, bambi 0.17.2.
Probability of Direction
The probability of direction () quantifies posterior certainty about the sign of an effect or contrast [35]:
where denotes the observed data. We estimate these probabilities from the proportions of posterior draws on either side of zero. Values near 0.5 indicate uncertainty about the direction, whereas values near 1 indicate high directional certainty, independent of effect magnitude.
Variance Decomposition
For the variance partitioning shown in Figure 4.1, we followed a Nakagawa-style decomposition [36, 37]. Therefore, we partition the variance of each fitted GLMM into fixed-effect, dataset-level random-effect, and residual components (matching the parametrization of our GLMM). For each posterior draw, fixed-effect variance is calculated as the empirical variance of the fixed-effects linear predictor across benchmark observations. Random-effect variance is given by the estimated dataset random-intercept variance, while residual variance is obtained from the Gaussian error variance or, for Gamma models, the trigamma function of the shape parameter. Furthermore, for each posterior draw, we normalize the three variance components by their sum. Figure 4.1 reports posterior mean proportions and 94% highest-density intervals (HDIs), evaluated on the original scale for Gaussian models and the log-link scale for Gamma models.
Posterior Analysis
The consensus in data analysis is that a simpler model should always be preferred. Yet simple can be defined along many axes. For an analysis paper in the field of benchmarking survival analysis methods, we argue that a Gaussian model should be prepared if it adequately (enough) describes the posterior distributions of the modeled variables. This is simply due to the fact, that the estimated coefficients are very easy to understand as they live in data space / scale and are additive.
That being said, in the following, we analyze the posterior probability curves of predicted models together with the actual data curves.
For C-index and IBS, the correct modeling choices would be a Beta distribution, as both metrics live on limited intervals. But for these two metrics we see in Fig. D.1 and D.2 that the Gaussian curves actually are very similar in total shape to their Beta variants. While they slightly transgress out of the valid ranges, we argue that this is still in an acceptable range. We especially confirmed that none of the findings and trends change when using Beta models for these two metrics in additional Bayesian pd and variance decomposition analyses.
For MAE-PO and X-CAL, however, we directly used Gamma distributions, as it was immediately apparent that simple Gaussian’s are highly inaccurate given the data distributions. In these cases, we therefore accept the conceptually harder to grasp and explain log-multiplier effects in order to better respect their true distributions.
Figure D.1. Posterior Predictive Check Used Assumptions in Main Body of Paper
Figure D.2. Posterior Predictive Check Beta for C-index and IBS
Probability Direction Analysis

Figure D.3. Probability of Direction () heatmap. Impact of dataset variables and model selection strategy on model performance for all metrics. Cells depict effect strengths. Higher is better for C-index. Lower is better for the rest.
Figure D.4. Effects of changing the Log-Sample size (Log N) by one standard deviation for each method on target metrics. Significant () results are colored. Red: detrimental effect, green: beneficial effect. A dagger (†) denotes metrics that are reported on a log-multiplier scale.
Figure D.5. Effects of changing the Censoring Rate (%) by one standard deviation for each method on target X-CAL. Significant () results are colored. Red: detrimental effect, green: beneficial effect. A dagger (†) denotes that X-CAL change is reported on a log-multiplier scale.
Supplementary Results Material
General Supplementary Material
Heatmaps
Figure E.1. 5-Fold cross-validated Concordance Index (C-index). Each cell reports the mean standard deviation across five folds for a given method–dataset combination. The figure shows datasets where (high-dimensional) sorted by ascending feature count . Cell colour intensity follows a sequential blue scale shared across both panels, ranging from 0.5 (random discrimination) to 0.9 (near-perfect discrimination). Gray cells labeled indicate that the experiment failed or ran but produced no valid result on all five folds. Cells marked with ’*’ are based on a subset of the five folds, because at least one fold failed. The C-index violin plots (right) show the cross-method distribution per dataset. A bold black border highlights the highest C-index per dataset and a red border highlights the best X-CAL score per dataset.
Figure E.2. 5-Fold cross-validated Integrated Brier Score (IBS). Each cell reports the mean standard deviation across five folds for a given method–dataset combination. The figure shows datasets where (high-dimensional) sorted by ascending feature count . Green shading indicates improvement over the Kaplan-Meier (KM) baseline IBS; intensity scales with the relative gain, with the darkest green corresponding to the best-performing method per dataset. White cells indicate performance at or below the KM baseline. Gray cells labeled indicate that the experiment failed or ran but produced no valid result on all five folds. Cells marked with ’*’ are based on a subset of the five folds, because at least one fold failed. The IBS violin plots (right) show the per-dataset distribution of relative improvement over KM across methods; the vertical dashed line marks zero improvement. The Baseline column reports the dataset-level KM IBS for reference.
Figure E.3. 5-Fold cross-validated Concordance Index (C-index). Each cell reports the mean ± standard deviation across five folds for a given method-dataset combination. The figure shows datasets where (low-dimensional) sorted by ascending sample size . Cell colour intensity follows a sequential blue scale shared across both panels, ranging from 0.5 (random discrimination) to 0.9 (near-perfect discrimination). Gray cells labeled NaN indicate that the experiment failed or ran but produced no valid result on all five folds. Cells marked with ‘*’ are based on a subset of the five folds, because at least one fold failed. The C-index violin plots (right) show the cross-method distribution per dataset. A bold black border highlights the highest C-index per dataset and a red border highlights the best X-CAL score per dataset.
Figure E.4. 5-Fold cross-validated Integrated Brier Score (IBS). Each cell reports the mean standard deviation across five folds for a given method-dataset combination. The figure shows datasets where (low-dimensional) sorted by ascending sample size . Green shading indicates improvement over the Kaplan-Meier (KM) baseline IBS; intensity scales with the relative gain, with the darkest green corresponding to the best-performing method per dataset. White cells indicate performance at or below the KM baseline. Gray cells labeled NaN indicate that the experiment failed or ran but produced no valid result on all five folds. Cells marked with “*” are based on a subset of the five folds, because at least one fold failed. The IBS violin plots (right) show the per-dataset distribution of relative improvement over KM across methods; the vertical dashed line marks zero improvement. The KM Baseline column reports the dataset-level KM IBS for reference.
Figure E.5. 5-Fold cross-validated Mean Absolute Error (MAE-PO). Each cell reports the mean standard deviation across five folds for a given method-dataset combination. The figure shows datasets where (low-dimensional) sorted by ascending sample size . Red shading indicates improvement over the Kaplan-Meier (KM) baseline MAE-PO across a dataset. Black cells mark outliers, defined per dataset row as values exceeding of the raw MAE distribution across methods. White cells indicate performance at or below the KM baseline. Gray cells labeled NaN indicate that the experiment failed or ran but produced no valid result on all five folds. Cells marked with ‘*’ are based on a subset of the five folds, because at least one fold failed. The violin plots show the relative-improvement distribution per dataset on a log-scale; outliers (black cells) are excluded to prevent extreme values from distorting the scale. The KM Baseline column reports the dataset-level KM MAE-PO for reference. A bold black border highlights the lowest MAE per dataset and a red border highlights the best X-CAL score per dataset.
Figure E.6. 5-Fold cross-validated Mean Absolute Error (MAE-PO). Each cell reports the mean standard deviation across five folds for a given method-dataset combination. The figure shows datasets where (high-dimensional) sorted by ascending feature count . Red shading indicates improvement over the Kaplan-Meier (KM) baseline MAE-PO across a dataset. Black cells mark outliers, defined per dataset row as values exceeding of the raw MAE distribution across methods. White cells indicate performance at or below the KM baseline. Gray cells labeled indicate that the experiment failed or ran but produced no valid result on all five folds. Cells marked with ’*’ are based on a subset of the five folds, because at least one fold failed. The violin plots show the relative-improvement distribution per dataset on a log-scale; outliers (black cells) are excluded to prevent extreme values from distorting the scale. The Baseline column reports the dataset-level KM MAE-PO for reference. A bold black border highlights the lowest MAE per dataset and a red border highlights the best X-CAL score per dataset.
Ranks
Figure E.7. Average Ranks Across Datasets for all three model selection metrics
Performance – Applicability trade-off
Figure E.8. Performance–applicability trade-off across model families for all three model selection criteria. X-axis shows the number of successful runs across all datasets and the y-axis denotes the fraction of how often a method lands within a fixed threshold of the best performing method on that dataset. The threshold for C-index (left) is set to absolute 0.02. As IBS and MAE-PO values are dataset dependent, the threshold is set to 5% – meaning that if a method achieved a score within 5% of the best performing dataset, it will be included in the fraction.
Runtime Analysis
Runtime in our benchmark is hard to disentangle, it is dependent on: (i) the model selection criterion, which affects the number of training epochs through early stopping, (ii) the size of each method’s hyperparameter grid, and (iii) the cost and implementation details of the individual evaluation metrics. Consequently, absolute runtime should not be interpreted as intrinsic algorithmic efficiency.
Our primary objective was to analyze how model selection criteria and dataset characteristics affect predictive performance, rather than benchmark computational efficiency. Within the successfully completed experiments, all methods were evaluated under identical hardware and runtime budgets, allowing controlled comparisons of model selection criteria.
To demonstrate the difficulty in disentangling runtime, we report runtime per hyperparameter (HP) configuration for the intersection of successfully completed method-dataset pairs. The results show substantial runtime differences both across methods and optimization criteria, illustrating that computational cost depends not only on the learning architecture but also on the selected evaluation objective.
| Method | Mean (s) | Median (s) |
| CoxNet | 0.540 | 0.041 |
| GrBoost | 2.338 | 0.186 |
| RSF | 0.414 | 0.245 |
| XGBoost | 0.356 | 0.269 |
| DeepSurv | 1.019 | 0.459 |
| DeepHit | 1.613 | 1.232 |
| CoxKAN | 2.131 | 1.370 |
| SurvTRACE | 11.119 | 4.154 |
| CoxTime | 11.868 | 5.556 |
Table E.1. Mean and median runtime (s) per HP configuration (C-index optimization).
We additionally compare the mean total runtime (s) for identical method-dataset pairs across different model selection criteria to demonstrate that runtime is influenced by the chosen optimization metric. Together, these show that deep models are slower in our setup both per evaluation and in total, while for flexible tree/boosting models, total cost additionally depends on which optimization metric is used.
| Method | C-index (s) | IBS (s) | MAE-PO (s) |
| CoxPH | 12.5 | 13.8 | 14.1 |
| CoxNet | 334.1 | 273.8 | 484.5 |
| RSF | 361.8 | 418.3 | 1,158.3 |
| XGBoost | 401.2 | 3,335.6 | 1,197.4 |
| SurvTRACE | 802.7 | 724.0 | 547.8 |
| GrBoost | 628.0 | 592.2 | 998.8 |
| DeepSurv | 953.6 | 1,488.1 | 1,055.3 |
| CoxTime | 1,942.4 | 1,786.0 | 2,058.7 |
| CoxKAN | 1,315.5 | 2,082.6 | 2,839.3 |
| DeepHit | 4,073.8 | 5,197.8 | 5,683.4 |
Table E.2. Mean runtime (s) per model-selection criterion.
Failure Analysis
Methods show inherent limitations. Fig. E.9 shows the failure rate for all experiments broken down by dataset regime and method, illustrated here for model selection; results under IBS and MAE-PO selection are qualitatively similar. We mark a dataset–method pair as failed if any fold did not produce results. The failures exhibit systematic structure. The failures are neither uniformly distributed across the matrix nor confined to individual methods. Instead, they concentrate in the large- and high- regimes, revealing where current survival analysis architectures reach their practical limits. The following subsections show the broken down analyses of the failure reasons runtime, memory and mathematical errors.
Runtime
We distinguish between symmetric runtime failures and asymmetric runtime failures. The symmetric runtime failure matrix highlights systematic scalability limitations. This is consistent with preliminary experiments before imposing the runtime budget: on hdfail (), GrBoost required 65h on a 32-core CPU with 128 GB RAM, compared to 7 minutes for CoxNet (+6.4% C-index) and 2.25 hours for DeepSurv (+7.4% C-index). This suggests that these failures reflect practical scalability limitations rather than the chosen budget.
| Method | Regime | Share |
| GrBoost | , low-dim | 67% (4/6) |
| RSF | , low-dim | 33% (2/6) |
Table 1.
In contrast, asymmetric runtime failures occur only for some model selection criteria. These are primarily explained by different optimization trajectories and metric evaluation costs. Consequently, increasing the runtime budget would most likely resolve many of these failures. A detailed analysis of the runtime in general is provided in E.2.
| Method | Regime | Share |
| CoxKAN | , high-dim | 59% (10/17) |
| CoxKAN | , low-dim | 23% (4/17) |
| CoxKAN | , high-dim | 6% (1/17) |
| DeepHit | , low-dim | 6% (1/17) |
| SurvTRACE | , low-dim | 6% (1/17) |
Table 2.
Memory
Memory failures, in contrast, cannot be resolved by extending the runtime budget. They occur systematically for the following method–regime combinations:
| Method | Regime | Share |
| SurvTRACE | high-dim () | 53% (8/15) |
| SurvTRACE | , low-dim | 13% (2/15) |
| CoxTime | , low-dim | 27% (4/15) |
| CoxTime | , low-dim | 7% (1/15) |
Table 3.
Memory failures are concentrated in CoxTime and SurvTRACE and occur consistently across all optimization targets within specific dataset regimes. They therefore likely reflect architecture-specific scalability limitations rather than the optimization objective. We observed increasing memory usage up to the available capacity, but did not collect detailed memory traces to identify the underlying causes.
Figure E.9. Failure rate matrix: fraction of datasets where a method fails under -optimized hyperparameters. Structured by observation count () and feature count ().
Mathematical Errors
Only unregularized CoxPH (baseline) exhibits mathematical failures (e.g., ill-conditioned matrices), indicating inherent numerical stability limitations rather than resource constraints, particularly in challenging high-dim and large- regimes.
Robustness Analysis on the Common Intersection of Successful Runs
In Fig. E.9, we see that some methods have systematical errors and could not run on certain datasets. This obviously might create a survivorship bias in our analysis. Here, we show that our Bayesian GLMM actually produced reliable results, by re-creating the Probability of Direction and median effect size analysis on the common intersection of successful runs. The median effect strengths with their probabilities of direction can be seen in Fig. E.10. All our main findings from the main body of the manuscript can be inferred from this balanced subset equally. Compare against Fig. D.3.
Figure E.10. Probability of Direction () heatmap on the common intersection of successful runs. Impact of dataset variables and model selection strategy on model performance for all metrics. Cells depict effect strengths. Higher is better for C-index. Lower is better for the rest.
Supplementary Material
Rank Shifts When Switching Target
| Method | ||||||||||||
| coxnet | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.008 | 0.000 | 0.000 | -0.001 | 0.005 | 0.000 |
| rsf | 0.000 | 0.003 | 0.002 | 0.000 | -0.001 | 0.000 | 0.003 | 0.000 | 0.000 | -0.002 | 0.000 | 0.000 |
| grboost | 0.000 | 0.001 | 0.002 | 0.000 | -0.001 | 0.000 | 0.000 | 0.000 | -0.001 | -0.001 | 0.000 | 0.000 |
| xgboost | -0.002 | 0.302 | 0.008 | 0.094 | -0.003 | 0.239 | 0.100 | 0.063 | 0.000 | -0.054 | 0.098 | -0.012 |
| deepsurv | -0.001 | 0.053 | 0.028 | 0.001 | 0.002 | 0.057 | 51.471 | 0.002 | 0.000 | 0.004 | 0.645 | 0.001 |
| deephit | -0.005 | 0.132 | -0.004 | 0.005 | -0.009 | 0.013 | 0.035 | 0.000 | -0.005 | -0.074 | 0.024 | -0.009 |
| coxtime | -0.002 | 0.051 | 0.447 | 0.002 | 0.005 | 0.087 | 5.389 | 0.007 | 0.003 | 0.002 | 0.300 | 0.001 |
| survtrace | -0.002 | 0.179 | 0.010 | 0.043 | -0.007 | 0.129 | 0.111 | 0.036 | -0.004 | -0.079 | 0.128 | -0.004 |
| coxkan | -0.004 | 0.006 | 0.004 | 0.000 | -0.004 | 0.000 | 0.048 | 0.000 | 0.000 | 0.000 | 0.028 | 0.000 |
Table E.3. Median paired effect of switching model selection targets. Each panel shows ( for X-CAL, as lower is better), where the superscript denotes the model selection criterion. All deltas are oriented such that indicates improvement. is reported on its standard scale in (higher is better). and denote relative improvement over the Kaplan–Meier baseline, unitless but interpretable as percentage gains. X-CAL (lower is better) is inverted here such that positive values indicate improved calibration.
Effects of Switching the Model Selection Criterion

Figure E.11. Effect of switching the hyperparameter optimization target from C-index to IBS. Deep learning and boosting methods gain substantially in IBS-rel with negligible C-index change (free lunch), while classical methods show little effect. CoxPH is excluded as no tuning is involved.

Figure E.12. Effect of switching the hyperparameter optimization target from C-index to MAE. MAE improves and becomes more stable across methods, with fewer extreme failures. The C-index remains largely unchanged, indicating that optimizing for time accuracy does not harm discrimination (free lunch).
Figure E.13. Effect of switching the hyperparameter optimization target from C-index to MAE, evaluated in terms of IBS-rel. Many methods show improved IBS relative to the Kaplan-Meier baseline, indicating better overall predictive accuracy. This demonstrates a free lunch effect where optimizing for MAE improves calibration-sensitive metrics without sacrificing discrimination.
Figure E.14. Effect of switching the hyperparameter optimization target from IBS to MAE. MAE improves consistently across most methods, confirming that direct optimization of time accuracy is effective. IBS shows mixed behavior: CoxNet, CoxKAN, DeepSurv and CoxTime benefit (free lunch), while boosting and transformer-based methods (e.g. XGBoost, SurvTRACE, DeepHit) on average perform better when directly optimized for IBS. RSF remains unaffected.
Figure E.15. Effect on X-CAL when switching the hyperparameter optimization target. Lower values indicate better calibration. Switching from C-index to IBS yields substantial improvements across all methods, confirming that IBS optimization directly improves calibration. Switching from C-index to MAE also improves calibration for most methods, though typically to a lesser extent. When comparing IBS to MAE optimization, two regimes emerge: CoxKAN, DeepSurv and CoxTime, MAE optimization further improves calibration, whereas for XGBoost, SurvTRACE, and DeepHit, calibration is best when directly optimized via IBS.
Mean MAE-PO Behaviour
How mean MAE-PO behaves when switching model selection criterion.
| Method | -optimized >KM (%)↑ | -optimized Median MAE (%)↑ | -optimized Out. (%)↓ | -optimized Median Degr. (%)↓ | IBS-optimized >KM (%)↑ | IBS-optimized Median MAE (%)↑ | IBS-optimized Out. (%)↓ | IBS-optimized Median Degr. (%)↓ | MAE-optimized >KM (%)↑ | MAE-optimized Median MAE (%)↑ | MAE-optimized Out. (%)↓ | MAE-optimized Median Degr. (%)↓ |
| coxnet | 69.4 | 2.63 | 0.0 | 0.00 | 71.4 | 2.64 | 2.0 | 892624.20 | 85.7 | 8.71 | 2.0 | 0.44 |
| rsf | 83.0 | 5.44 | 0.0 | 0.00 | 85.1 | 7.34 | 0.0 | 0.00 | 89.4 | 8.14 | 0.0 | 0.00 |
| grboost | 84.4 | 6.33 | 0.0 | 0.00 | 84.4 | 5.94 | 0.0 | 0.00 | 84.4 | 6.39 | 0.0 | 0.00 |
| xgboost | 49.0 | -4.25 | 6.1 | 86.15 | 46.9 | -2.56 | 6.1 | 434698.87 | 77.6 | 9.60 | 6.1 | 4.29 |
| deepsurv | 20.4 | -6560.74 | 55.1 | 1541743.88 | 16.3 | -391.81 | 57.1 | 2037.14 | 55.1 | 3.16 | 6.1 | 382.17 |
| deephit | 51.0 | 0.71 | 8.2 | 405221.69 | 46.9 | -0.44 | 12.2 | 232.46 | 59.2 | 4.37 | 20.4 | 934.85 |
| coxtime | 9.1 | -678.90 | 61.4 | 36704.92 | 18.2 | -79.37 | 36.4 | 3619.42 | 52.3 | 0.53 | 15.9 | 50.94 |
| survtrace | 42.1 | -2.12 | 2.6 | 5.91 | 43.6 | -1.87 | 7.7 | 8.60 | 87.2 | 11.47 | 2.6 | 3.76 |
| coxkan | 38.3 | -12.95 | 12.8 | 2272241.02 | 37.0 | -7.20 | 10.9 | 2097.30 | 61.0 | 4.29 | 9.8 | 6257.43 |
| ALL | 49.9 | -0.10 | 16.3 | 581276.38 | 50.1 | 0.04 | 14.9 | 2162.77 | 72.3 | 6.16 | 7.0 | 50.94 |
Table E.4. MAE Improvement and Outlier Analysis. Relative MAE improvement over the Kaplan-Meier (KM) baseline for different model selection criteria. Positive values indicate improvement over KM. Outliers are computed per dataset across methods using the lower-tail 1.5 · rule on relative MAE improvement values. The table reports the frequency of outperforming KM (>KM), median relative MAE improvement, the proportion of dataset-level failures (% outliers), and the median degradation among those failures. Lower outlier rates and degradation values indicate more stable behavior.
Supplementary Material
Regularization Ablation Study During the experiments, DL-specific regularization strategies were already included in the HP search: we optimized the dropout rate (0.0, 0.2) together with early stopping (patience = 10), see Appendix Table E.5. To better understand the effects of regularization on high-dimensional datasets, we additionally performed the an ablation study on a representative high-dimensional dataset (DBCD, , ). We evaluated DeepSurv, DeepHit, and CoxTime extending the dropout grid in total to 0.0, 0.2, 0.35, 0.5, 0.65, selecting the best candidate among a fixed dropout rate during the search while the remaining search grid was kept unchanged.
Table E.5 presents the results by model-selection criterion. Overall, incorporating dropout into the search space is generally beneficial. In most settings, models with dropout outperform their counterparts without dropout, while the optimal dropout rate depends on the architecture and model-selection criterion.
| Method | Dropout 0.0 | Dropout 0.2 | Dropout 0.35 | Dropout 0.5 | Dropout 0.65 |
| C-index (↑) | |||||
| DeepHit | |||||
| DeepSurv | |||||
| CoxTime | |||||
| IBS (↓) | |||||
| DeepHit | |||||
| DeepSurv | |||||
| CoxTime | |||||
| MAE-PO (↓) | |||||
| DeepHit | |||||
| DeepSurv | |||||
| CoxTime |
Table E.5. Dropout ablation study on DBCD (, ) for DeepHit, DeepSurv, and CoxTime, grouped by model-selection criterion. The remaining hyperparameter search grid and early stopping configuration were kept unchanged. Bold values indicate the best result within each row.

Figure E.16. Rank Shifts When Changing Regime. Effect of dataset size on method rankings in low-dimensional datasets. Average ranks for small vs large datasets. C-index; C-index-selected models. Average Rank (lower is better). (30); (9). coxnet (3.43), coxnet (5.00), coxkan (4.23), coxkan (5.11), rsf (4.67), rsf (6.33), survtrace (4.73), survtrace (6.00), coxtime (4.83), coxtime (7.89), grboost (4.90), grboost (6.78), deepsurv (5.43), deepsurv (4.00), xgboost (6.13), xgboost (4.33), deephit (6.63), deephit (3.89). MAE; MAE-selected models. Average Rank (lower is better). survtrace (3.17), survtrace (4.11), coxnet (3.90), coxnet (5.11), grboost (4.50), grboost (7.22), rsf (4.53), rsf (7.67), xgboost (5.17), xgboost (1.89), coxkan (5.77), coxkan (4.89), deephit (5.87), deephit (4.11), deepsurv (5.93), deepsurv (5.00), coxtime (6.17), coxtime (8.89). Figure E.16: Effect of dataset size on method rankings in low-dimensional settings. Average ranks for smaller () and larger () datasets reveal pronounced shifts, indicating that conclusions about the best-performing methods are not stable across dataset regimes.
Figure E.17. Effect of dimensionality on method rankings. Average ranks are shown for low-dimensional () and high-dimensional () datasets across three model selection settings. Substantial shifts in rankings are observed when moving to high-dimensional regimes, with tree-based methods often improving.
Supplementary Material
Calibration - Relative vs. Absolute Utility
Figure E.18. Dataset-level relationship between censoring rate and X-CAL. Points show the average X-CAL across all methods and model selection strategies per dataset (lower is better). A moderate negative correlation (Spearman ) indicates that calibration tends to improve as censoring increases, suggesting that heavily censored datasets may lead to lower X-CAL scores.
NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Answer: [Yes]
Justification: See sections Findings 4.3, and Conclusion 5
Guidelines:
The answer [N/A] means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A [No] or [N/A] answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: see Conclusion 5
Guidelines:
The answer [N/A] means that the paper has no limitation while the answer [No] means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate “Limitations” section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren’t acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [N/A] Justification: benchmarking and analysis paper; no theoretical proofs introduced or needed.
Guidelines:
The answer [N/A] means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: we describe data selection, preprocessings, used metrics, and all used libraries and reference implementations. See § 5 and the appendix at B, C, C.5. And for our GLMM, see D.1.1 for full details.
Guidelines:
The answer [N/A] means that the paper does not include experiments.
If the paper includes experiments, a [No] answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general, releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example (a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully. (c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: fully transparent and reproducible pipeline is provided: https://github.com/sdening/surv-bench-study.
Guidelines:
The answer [N/A] means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://neurips.cc/public/guides/CodeSubmissionPolicy) for more details.
While we encourage the release of code and data, we understand that this might not be possible, so [No] is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://neurips.cc/public/guides/CodeSubmissionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer) necessary to understand the results?
Answer: [Yes]
Justification: Overview in Section 3 and all related subsections and figures. Also see the appendix at B, C, C.5. And for our GLMM, see D.1.1 for full details. All of this can be found in the provided repo link equally.
Guidelines:
The answer [N/A] means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We have error bars at multiple levels. We perform K-fold Cross Val for main experiments. Error bars and details are all in the appendix. For our main analysis, we model a Bayesian Generalized Linear Mixed Effects Model. We do posterior plausibility checks, fully justify all our choices. And we avoid using frequentist p-tests, as we have multiple paired ones. We stick to a fully Bayesian analysis and report Bayesian Highest Density Intervals, Effect Strengths, and Probabilities of Directions.
Guidelines:
The answer [N/A] means that the paper does not include experiments.
The authors should answer [Yes] if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g., negative error rates).
If error bars are reported in tables or plots, the authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: The paper specifies the hardware and software environment used for all experiments, as well as the Python and library versions used throughout the benchmark C.1. All experiments were executed exclusively on CPUs without GPU acceleration. We additionally report runtime characteristics, such as the average runtime per hyperparameter configuration C.3 and the fixed compute budget used throughout the benchmark, to support reproducibility and provide transparency regarding computational requirements 3.3.
Guidelines:
The answer [N/A] means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn’t make it into the paper).
Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?
Answer: [Yes]
Justification: we fully adhere to the stated ethical principles.
Guidelines:
The answer [N/A] means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer [No], they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).
Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [N/A]
Justification: a deep benchmark analysis uncovering trends in model selection strategies and dataset regimes. Has no obvious or inherent / direct links to potential harms.
Guidelines:
The answer [N/A] means that there is no societal impact of the work performed.
If the authors answer [N/A] or [No], they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate Deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pre-trained language models, image generators, or scraped datasets)?
Answer: [N/A]
Justification: none of that applies to our paper.
Guidelines:
The answer [N/A] means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.
Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: any use of code libraries or datasets is properly cited.
Guidelines:
The answer [N/A] means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset’s creators.
New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We included our code base and documented it with useful comments and a sensible structure.
Guidelines:
The answer [N/A] means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [N/A]
Justification: [N/A]
Guidelines:
The answer [N/A] means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [N/A] Justification: [N/A]
Guidelines:
The answer [N/A] means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigor, or originality of the research, declaration is not required.
Answer: [N/A]
Justification: [N/A]
Guidelines:
The answer [N/A] means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy in the NeurIPS handbook for what should or should not be described.
References
- [1]Oriol Abril-Pla, Virgile Andreani, Colin Carroll, Larry Dong, Christopher J. Fonnesbeck, Maxim Kochurov, Ravin Kumar, Junpeng Lao, Christian C. Luhmann, Osvaldo A. Martin, Michael Osthegere, Ricardo Vieira, Thomas Wiecki, and Robert Zinkov. PyMC: A modern, and comprehensive probabilistic programming framework in Python. PeerJ Computer Science, 9:e1516, 2023.DOI
- [2]Julie Alberge, Vincent Maladière, Olivier Grisel, Judith Abécassis, and Gaël Varoquaux. Survival models: Proper scoring rule and stochastic optimization with competing risks. In International Conference on Artificial Intelligence and Statistics, 2025.
- [3]Laura Antolini, Patrizia Boracchi, and Elia Biganzoli. A time-dependent discrimination index for survival data. Statistics in Medicine, 24(24):3927–3944, 2005.
- [5]Giovanni Birolo, Ivan Rossi, Flavio Sartori, Cesare Rollo, Piero Fariselli, and Tiziana Sanavia. Beyond Cox models: Assessing the performance of machine-learning methods in non-proportional hazards and non-linear survival analysis. Computers in Biology and Medicine, 198, 2025.DOI
- [6]Anne-Laure Boulesteix, Rory Wilson, and Alexander Hapfelmeier. Towards evidence-based computational statistics: lessons from clinical research on the role and design of real-data benchmark studies. BMC Medical Research Methodology, 17(138), 2017.
- [7]Lukas Burk, John Zobolas, Bernd Bischl, Andreas Bender, Marvin N. Wright, and Raphael Sonabend. A large-scale neutral comparison study of survival models on low-dimensional data. Bioinformatics, 42(5), 2026.
- [8]Tomás Capretto, Carmen Piho, Ravin Kumar, Jacob Westfall, Tal Yarkoni, and Osvaldo A. Martin. Bambi: A simple interface for fitting bayesian linear models in python. Journal of Statistical Software, 103(15), 2022.
- [9]George H. Chen. An introduction to deep survival analysis models for predicting time-to-event outcomes. Foundations and Trends in Machine Learning, 17(6):921–1100, 2024.arxiv.org/abs/2410.01086
- [10]Tianqi Chen and Carlos Guestrin. XGBoost: A scalable tree boosting system. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 785–794, 2016.DOI
- [11]Isabel Chien, Zachary Izzo, Javier Gonzalez, and Richard E. Turner. Benchmarking survival models: Treatment effects, bias, and equity, 2024. ICLR 2025 withdrawn submission, available at https://openreview.net/forum?id=aoW5sm80p8.
- [12]David R. Cox. Regression models and life-tables. Journal of the Royal Statistical Society: Series B (Methodological), 34(2):187–202, 1972.
- [13]Christina Curtis, Sohrab P. Shah, Suet-Feung Chin, Gulisa Turashvili, Oscar M. Rueda, Mark J. Dunning, Doug Speed, Andy G. Lynch, Shamith Samarajiwa, Yinyin Yuan, Stefan Gräf, Gavin Ha, Gholamreza Haffari, Ali Bashashati, Roslin Russell, Steven McKinney, METABRIC Group, Anita Langerød, Andrew Green, Elena Provenzano, Gordon Wishart, Sarah Pinder, Peter Watson, Florian Markowetz, Leigh Murphy, Ian Ellis, Arnie Purushotham, Anne-Lise Børresen-Dale, James D. Brenton, Simon Tavaré, Carlos Caldas, and Samuel Aparicio. The genomic and transcriptomic architecture of 2,000 breast tumours reveals novel subgroups. Nature, 486(7403):346–352, 2012.DOI
- [14]Erik Drysdale. SurvSet: An open-source time-to-event dataset repository, 2022. arXiv preprint arXiv:2203.03094.arxiv.org/abs/2203.03094
- [15]Bruce Fischl. FreeSurfer. NeuroImage, 62(2):774–781, 2012.
- [16]Jerome H. Friedman. Greedy function approximation: A gradient boosting machine. The Annals of Statistics, 29(5):1189–1232, 2001.DOI
- [17]Andrew Gelman and Jennifer Hill. Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge University Press, 2007.
- [18]Mark Goldstein, Xintian Han, Aahad Puli, Adler Perotte, and Rajesh Ranganath. X-CAL: Explicit calibration for survival analysis. In Advances in Neural Information Processing Systems, 2020.
- [19]Érika Graf, Claudia Schmoor, Willi Sauerbrei, and Martin Schumacher. Assessment and comparison of prognostic classification schemes for survival data. Statistics in Medicine, 18(17-18):2529–2545, 1999.
- [20]Aaron Green, Zihan Nie, Hanzhen Qin, Oshani Seneviratne, and Kristin P. Bennett. FinSurvival: A suite of large scale survival modeling tasks from finance. Journal of Data-centric Machine Learning Research, 2026.arxiv.org/abs/2507.14160
- [21]Humza Haider, Bret Hoehn, Sarah Davis, and Russell Greiner. Effective ways to build and evaluate individual survival distributions. Journal of Machine Learning Research, 21(85):1–63, 2020.
- [22]Frank E. Harrell, Jr., Robert M. Califf, David B. Pryor, Kerry L. Lee, and Robert A. Rosati. Evaluating the yield of medical tests. Journal of the American Medical Association, 247(18):2543–2546, 1982.
- [23]Moritz Herrmann, Philipp Probst, Roman Hornung, Vindi Jurinovic, and Anne-Laure Boulesteix. Large-scale benchmark study of survival prediction methods using multi-omics data. Briefings in Bioinformatics, 22(3), 2020.DOI
- [24]Hemant Ishwaran, Udaya B. Kogalur, Eugene H. Blackstone, and Michael S. Lauer. Random survival forests. The Annals of Applied Statistics, 2(3):841–860, 2008.
- [25]Alistair E. W. Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shamout, Steven Horng, Tom J. Pollard, Shenghao Hao, Benjamin Moody, Brian Gow, Li-wei H. Lehman, Leo A. Celi, and Roger G. Mark. MIMIC-IV, a freely accessible electronic health record dataset. Scientific Data, 10(1), 2023.DOI
- [26]Edward L. Kaplan and Paul Meier. Nonparametric estimation from incomplete observations. Journal of the American Statistical Association, 53(282):457–481, 1958.
- [27]Jared L. Katzman, Uri Shaham, Alexander Cloninger, Jonathan Bates, Tingting Jiang, and Yuval Kluger. DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network. BMC Medical Research Methodology, 18(24), 2018.DOI
- [28]Da In Kim, Wei Siang Lai, and Kelly W. Zhang. Tabular foundation models can do survival analysis, 2026. arXiv preprint arXiv:2601.22259.arxiv.org/abs/2601.22259
- [29]William A. Knaus, Frank E. Harrell, Jr., Joanne Lynn, Lee Goldman, Russell S. Phillips, Alfred F. Connors, Jr., Neal V. Dawson, William J. Fulkerson, Jr., Robert M. Califf, Norman Desbiens, Peter Layde, Robert K. Oye, Paul E. Bellamy, Rosemarie B. Hakim, and Douglas P. Wagner. The SUPPORT prognostic model: Objective estimates of survival for seriously ill hospitalized adults. Annals of Internal Medicine, 122(3):191–203, 1995.
- [30]William Knottenbelt, William McGough, Rebecca Wray, Woody Zhidong Zhang, Jiashuai Liu, Ines Prata Machado, Zeyu Gao, and Mireia Crispin-Ortuzar. CoxKAN: Kolmogorov-Arnold networks for interpretable, high-performance survival analysis. Bioinformatics, 41(8), 2025.DOI
- [31]Håvard Kvamme and Ørnulf Borgan. The Brier score under administrative censoring: Problems and a solution. Journal of Machine Learning Research, 24(2):1–26, 2023.
- [32]Håvard Kvamme, Ørnulf Borgan, and Ida Scheel. Time-to-event prediction with neural networks and cox regression. Journal of Machine Learning Research, 20(129):1–30, 2019.
- [33]Changhee Lee, William R. Zame, Jinsung Yoon, and Mihaela Van Der Schaar. DeepHit: A deep learning approach to survival analysis with competing risks. In Proceedings of the AAAI Conference on Artificial Intelligence, 2018.
- [34]Christian Marius Lillelund, Shi-ang Qi, Russell Greiner, and Christian Fischer Pedersen. Position: Stop chasing the C-index when evaluating survival analysis models. In International Conference on Machine Learning, 2026.arxiv.org/abs/2506.02075
- [35]Dominique Makowski, Mattan S. Ben-Shachar, S. H. Annabel Chen, and Daniel Lüdecke. Indices of effect existence and significance in the bayesian framework. Frontiers in Psychology, 10:2767, 2019.DOI
- [36]Shinichi Nakagawa and Holger Schielzeth. A general and simple method for obtaining R² from generalized linear mixed-effects models. Methods in ecology and evolution, 4(2):133–142, 2013.
- [37]Shinichi Nakagawa, Paul C. D. Johnson, and Holger Schielzeth. The coefficient of determination R² and intra-class correlation coefficient from generalized linear mixed-effects models revisited and expanded. Journal of the Royal Society Interface, 14(134), 2017.
- [38]Bernard Pereira, Suet-Feung Chin, Oscar M. Rueda, Hans-Kristian Moen Vollan, Elena Provenzano, Helen A. Bardwell, Michelle Pugh, Linda Jones, Roslin Russell, Stephen-John Sammut, Dana W. Y. Tsui, Bin Liu, Sarah-Jane Dawson, Jean Abraham, Helen Northen, John F. Peden, Abhik Mukherjee, Gulisa Turashvili, Andrew R. Green, Steve McKinney, Arusha Oloumi, Sohrab Shah, Nitzan Rosenfeld, Leigh Murphy, David R. Bentley, Ian O. Ellis, Arnie Purushotham, Sarah E. Pinder, Anne-Lise Børresen-Dale, Helena M. Earl, Paul D. Pharoah, Mark T. Ross, Samuel Aparicio, and Carlos Caldas. The somatic mutation profiles of 2,433 breast cancers refine their genomic and transcriptomic landscapes. Nature Communications, 7(11479), 2016.DOI
- [39]Ronald C. Petersen, Paul S. Aisen, Laurel A. Beckett, Michael C. Donohue, Anthony C. Gamst, Danielle J. Harvey, Clifford R. Jack, Jr., William J. Jagust, Leslie M. Shaw, Arthur W. Toga, John Q. Trojanowski, and Michael W. Weiner. Alzheimer’s Disease Neuroimaging Initiative (ADNI): Clinical characterization. Neurology, 74(3):201–209, 2010.DOI
- [40]Sebastian Pölsterl. scikit-survival: A library for time-to-event analysis built on top of scikit-learn. Journal of Machine Learning Research, 21(212):1–6, 2020.
- [41]Sanjay Purushotham, Chuizheng Meng, Zhengping Che, and Yan Liu. Benchmarking deep learning models on large healthcare datasets. Journal of Biomedical Informatics, 83:112–134, 2018.DOI
- [42]Shi-Ang Qi, Neeraj Kumar, Mahtab Farrokh, Weijie Sun, Li-Hao Kuan, Rajesh Ranganath, Ricardo Henao, and Russell Greiner. An effective meaningful way to evaluate survival models. In Proceedings of the 40th International Conference on Machine Learning, 2023.
- [43]Shi-ang Qi, Weijie Sun, and Russell Greiner. SurvivalEVAL: A comprehensive open-source Python package for evaluating individual survival distributions. Proceedings of the AAAI Symposium Series, 2(1):453–457, 2024.DOI
- [44]Begoña B. Sierra, Colin McLean, Peter S. Hall, and Catalina A. Vallejos. The C-index multiverse, 2025. arXiv preprint arXiv:2508.14821.arxiv.org/abs/2508.14821
- [45]Noah Simon, Jerome H. Friedman, Trevor Hastie, and Robert Tibshirani. Regularization paths for Cox’s proportional hazards model via coordinate descent. Journal of Statistical Software, 39(5):1–13, 2011.
- [47]Weijing Tang, Jiaqi Ma, Qiaozhu Mei, and Ji Zhu. SODEN: A scalable continuous-time survival model through ordinary differential equation networks. Journal of Machine Learning Research, 23(34):1–29, 2022.
- [48]Hajime Uno, Tianxi Cai, Michael J. Pencina, Ralph B. D’Agostino, and Lee-Jen Wei. On the C-statistics for evaluating overall adequacy of risk prediction procedures with censored survival data. Statistics in Medicine, 30(10):1105–1117, 2011.
- [49]Davi Vieira, Gabriel Gimenez, Guilherme Mარმerola, and Vitor Estima. XGBoost survival embeddings: improving statistical properties of XGBoost survival analysis implementation, 2021. URL http://github.com/loft-br/xgboost-survival-embeddings.
- [50]Zifeng Wang and Jimeng Sun. SurvTRACE: Transformers for survival analysis with competing events. In ACM Conference on Bioinformatics, Computational Biology, and Health Informatics, 2022.
- [51]Simon Wiegrebe, Philipp Kopper, Raphael Sonabend, Bernd Bischl, and Andreas Bender. Deep learning for survival analysis: a review. Artificial Intelligence Review, 57(3), 2024.
- [52]Yunwei Zhang, Germaine Wong, Graham Mann, Samuel Muller, and Jean Y. H. Yang. SurvBenchmark: comprehensive benchmarking study of survival analysis methods using both omics data and clinical data. GigaScience, 11, 2022.
- [53]Hanpu Zhou, Hong Wang, Sizheng Wang, and Yi Zou. SurvMetrics: An R package for predictive evaluation metrics in survival analysis. The R Journal, 14(4):252–263, 2022.






















