In this paper
AbstractIntroductionMaterials and MethodsData CurationPDO Drug Screening StudiesIMPROVE drug screening study 1IMPROVE drug screening study 2IMPROVE drug screening study 3Pilot 1 drug screening studyCell Line Drug Screening DataOmics Data and Drug Feature DataDrug Response Prediction ModelsAnalysis SchemesModel Training on Cell Line Data OnlyModel Training on Cell Line and PDO Data JointlyTransfer Learning from Cell Lines to PDOsResultsResultsCorrelation Between PDO Drug Screening StudiesPrediction PerformanceModel Training Using CCL Data OnlyModel Training Using CCL and PDO Data JointlyModel Training Using CCL and PDO Data JointlyTransfer Learning From CCLs To PDOsDiscussionData and Code AvailabilityAcknowledgementSupplementary InformationDetails of Drug Response Prediction ModelsUNOIGTDGraphDRPHiDRALGBMDeepTTCTransfer Learning Implementation and Hyperparameter OptimizationLayer FreezingHyper-parameter OptimizationPrediction Performance Measured by PCCReferencesWorksAgricultural and Biological Sciences
PDO Drug Screening Data for Evaluating and Training AI Drug Response Prediction Models
Abstract
Anti-cancer drug response prediction (DRP) is critically important for precision oncology and therapeutic development. Currently, most DRP models are developed using drug screening data from immortalized cancer cell lines, which can differ substantially from patient tumors at the genomic and molecular levels, even though patient tumor response is the ultimate prediction target. To help bridge this gap and bring DRP models closer to their intended clinical application, we conducted four drug screening studies using patient-derived organoids (PDOs), which more faithfully recapitulate patient tumors than immortalized cell lines. These studies generated data for improving and evaluating DRP models, including multi-omics profiles of PDO lines, drug feature data, and treatment response measurements. Using these data, we retrained and evaluated multiple publicly available DRP models under three model-building scenarios: training models using cell line data only, training models using combined cell line and PDO data, and pretraining models on cell line data followed by fine-tuning on PDO data. We then evaluated and compared model prediction performance on PDOs across these analysis scenarios. In addition, we also assessed the consistency of drug screening viability data generated by different laboratories using different assay technologies and/or experimental protocols.
Introduction
Cancer remains a major global health challenge despite continued advances in diagnosis and treatment. Precision oncology seeks to improve therapeutic outcomes by using patient-specific molecular characteristics to guide treatment selection, and the increasing availability of genomic and other molecular profiling data has created new opportunities for computational approaches to support this goal [1]. In particular, artificial intelligence (AI) and machine learning methods are increasingly being developed for drug response prediction (DRP), in which molecular features of a cancer case and characteristics of a therapeutic compound are used to predict treatment sensitivity or resistance. A wide range of approaches, including graph neural networks [2], transformers [3], and gradient-boosted decision trees [4], have been developed for this task. However, the utility of these models depends fundamentally on the experimental data available for their development and evaluation.
Most existing DRP models have been developed using large-scale pharmacogenomic datasets generated using immortalized cancer cell lines (CCLs), including resources such as the Cancer Cell Line Encyclopedia (CCLE) [5] and Genomics of Drug Sensitivity in Cancer (GDSC) [6]. CCLs are relatively inexpensive to culture, experimentally reproducible, and capable of indefinite proliferation, making it possible to generate drug response measurements across large numbers of cancer models and compounds. These properties have made CCL datasets an essential resource for the development of computational DRP methods. However, prolonged in vitro culture and growth in two-dimensional monolayers can alter molecular characteristics [7] and fail to reproduce important features of patient tumors, including cellular heterogeneity, three-dimensional tissue architecture, and aspects of the tumor microenvironment. This creates an important gap between the experimental systems that provide most of the data used to train DRP models and the patient tumors to which such models are ultimately intended to be applied. Patient-derived organoids (PDOs) offer an experimental system that can help bridge the gap between conventional cancer cell lines and patient tumors. PDOs are three-dimensional cultures established from patient tumor tissues and can retain genomic, molecular, and phenotypic characteristics of the original tumors more faithfully than immortalized cell lines. These properties make PDOs a promising preclinical model for evaluating therapeutic response and for developing computational models that more closely reflect the intended application of precision oncology. However, PDO pharmacogenomic data remain limited compared with the large-scale drug screening data available for CCLs, posing a significant barrier to the development and evaluation of DRP models for PDOs.
In this work, we address this need by generating new PDO drug screening data specifically designed to support the development and evaluation of AI-based DRP models. We conducted four drug screening studies spanning colorectal cancer, head and neck cancer, and sarcoma PDOs, producing matched drug response measurements and molecular profiles across multiple experimental platforms and assay protocols. Three of these studies were designed with overlapping PDOs and compounds, enabling direct comparison of drug response measurements generated independently using different screening technologies and experimental procedures. Together, these studies provide new data for investigating both the reproducibility of PDO drug screening measurements and the ability of computational models to predict drug response in experimental systems that more closely represent patient tumors.
We further use these newly generated datasets to investigate how PDO data can support the development and evaluation of DRP models. Six publicly available DRP models representing diverse machine learning and deep learning architectures were evaluated under three learning strategies. The first strategy trains models exclusively on large-scale cancer cell line (CCL) data and directly applies them to predict drug responses in PDOs. The second strategy trains models on a combination of CCL data and a subset of PDO data, with predictive performance evaluated on the remaining PDO data. The third strategy pretrains models on CCL data and subsequently fine-tunes them using a subset of PDO data, with performance evaluated on the remaining PDO data. Model performance was assessed using two cross-validation schemes: random partitioning, in which data are randomly split into training, validation, and test sets, and cancer-blind partitioning, in which cancer cases are mutually exclusive across these sets. The cancer-blind scheme is designed to assess model generalizability to previously unseen cancer cases. By systematically evaluating multiple model architectures and learning strategies across independently generated PDO datasets, we characterize the extent to which knowledge learned from large-scale CCL datasets transfers to PDOs and whether limited PDO data can improve predictive performance in the target domain. Together, these analyses demonstrate the utility of the newly generated datasets for developing and evaluating DRP models for PDO drug response prediction.
Materials and Methods
Data Curation
PDO Drug Screening Studies
To bridge the gap between cell lines and patient tumors and to bring DRP models closer to their ultimate goal of predicting patient tumor response, we conducted four drug screening studies on PDOs of colorectal cancers, head and neck cancers, and sarcoma to collect matched treatment response and omics data with multiple drug screening technologies. Three of the drug screening studies were conducted for the Innovative Methodologies and New Data for Predictive Oncology Model Evaluation (IMPROVE) project [8], and their datasets are named ”IMPROVE dataset 1”, ”IMPROVE dataset 2”, and ”IMPROVE dataset 3”. The other drug screening study was conducted for our previous Pilot 1 project whose goal was to develop novel effective DRP approaches, and its dataset is named ”Pilot 1”. See Table 1 for the numbers of PDO/cell lines, compounds, and experiments (pairs of PDO/cell lines and compounds) included in the four datasets.
| Source | # Experiments | # Drugs | # Cell Lines | # Organoids |
| CCLE | 5,803 | 24 | 470 | 0 |
| CTRPv2 | 282,745 | 494 | 803 | 0 |
| GDSCv1 | 119,758 | 294 | 619 | 0 |
| GDSCv2 | 53,893 | 168 | 539 | 0 |
| gCSI | 3,130 | 16 | 355 | 0 |
| IMPROVE dataset 1 | 600 | 100 | 2 | 4 |
| IMPROVE dataset 2 | 280 | 40 | 2 | 5 |
| IMPROVE dataset 3 | 1,200 | 100 | 2 | 10 |
| Pilot 1 | 1,316 | 37 | 0 | 47 |
| Overall | 468,725 | 766 | 883 | 57 |
Table 1. Summary of datasets and experimental sources. The final row reports the total number of unique experiments, drugs, cell lines, and organoids across all sources.
IMPROVE drug screening studies 1-3 were designed jointly to share common experiments that enable cross-platform correlation analyses to evaluate the reproducibility of drug screening experiments. IMPROVE datasets 1-3 include 4 colorectal PDO lines, 5 colorectal PDO lines, and 9 colorectal PDO lines plus 1 sarcoma PDO line, respectively. All three IMPROVE drug screening studies also include two common colorectal cancer cell lines for benchmarking purpose. See Figure 1 for the numbers of overlapping PDO/cell lines, drugs, and experiments between studies. The Pilot 1 dataset includes 27 colorectal PDO lines and 20 head and neck cancer PDO lines, which are not included in IMPROVE datasets 1-3. Our drug screening studies focused on colorectal cancer and head and neck cancer, because these two cancer types have lower response prediction accuracies compared with other cancer types based on our previous analysis results. Accumulating more data for these two cancer types helps improve their prediction accuracies. The compounds used in the drug screening studies were selected to cover most of the major mechanisms of actions (MoAs) of current drugs and compounds used in cancer clinical practice and research. The following briefly introduces the experimental protocols that were used to generate the viability measurements in the four screening studies.

Figure 1. Overlap of samples, experiments, and drugs across the three IMPROVE datasets. Diagonal entries indicate the total number of elements in each set, while off-diagonal entries indicate pairwise intersections. The intersection and union across all three sets are shown below each matrix.
IMPROVE drug screening study 1
Drug responses were evaluated in two colorectal cancer cell lines (HT29 and SW620) and four PDO models (PDM-2, PDM-5, PDM-7, and PDM-9) using a 3D high-content imaging assay. PDO models were developed by the Human Cancer Models Initiative (HCMI) and purchased from the American Type Culture Collection (ATCC). Cell lines were expanded in 2D culture, harvested, mixed with hydrogel, and seeded at 1,250 cells/well into 384-well plates using an automated liquid handler. Organoids were expanded in 3D culture, size-filtered to 20–70 , mixed with hydrogel, and similarly seeded into 384-well plates. One day after seeding, compounds were added and cultures were exposed for 5 days. Compounds were evaluated at seven concentrations: 31.6, 3.16, 0.316, 0.0316, 0.00316, 0.000316, and 0.0000316 . Staurosporine (10 ) served as a cytotoxic positive control, while 0.3% DMSO served as the primary solvent control; 0.3% DMF or HO controls were included where appropriate. Conditions were generally tested in triplicate, distributed across replicate plates. After treatment, cultures were fixed with 2% paraformaldehyde and stained with Rhodamine-Phalloidin and Hoechst to visualize F-actin and nuclei, respectively.
Z-stack images of each well were acquired. Individual Z-stack planes were first analyzed separately and then the obtained data were integrated across planes to generate numeric features. Key features used in our analyses were organoid count and fraction of living cells. Measurements were normalized per model and plate relative to the appropriate solvent control. Before normalization, outliers were detected from organoid counts per well using the interquartile-range method and removed. Assay quality was evaluated using Z-prime and coefficient of variation (CV) calculated from readouts of control wells, with Z-prime and CV < 20% considered acceptable. We used the fraction of living cells to calculate the areas under the dose response curves (AUCs) for the dose range of [100 pM, 100 µM]. AUC values were then normalized by the dose range to obtain normalized AUC values whose typical range was [0, 1].
IMPROVE drug screening study 2
Drug screening was performed in seven colorectal cancer models, including two cancer cell lines (SW620 and HT-29) and five PDO lines (PDM-2, PDM-4, PDM-5, PDM-7, and PDM-9) developed by the HCMI. Both cell lines and PDO models were purchased from the ATCC. For the cell line screen, cells were embedded in hydrogel, plated, and pre-cultured for 1 day before compound treatment. PDO models were expanded for 1–5 passages, biobanked, embedded in hydrogel, and allowed to grow for 1 day; medium was refreshed on day 2 before treatment. Forty compounds were tested in each model at seven concentrations spanning 0.0000316–31.6 µM in 10-fold serial dilutions, with three replicate wells per concentration, followed by 5 days of drug exposure. Each screening plate included DMSO and/or PBS negative controls (8 wells per control type on each plate), a seven-point dose range of a standard-of-care reference compound, and 10 µM staurosporine as a positive cytotoxic control (8 wells per plate). Additional growth controls included serum-free conditions for the cell lines (4 wells per plate) and 0 ng/mL epithelial growth factor for PDOs (4 wells per plate). At the end of treatment, cell viability was measured using the CellTiter-Glo 3D ATP-release assay. ATP-release viability measurements of test conditions were normalized to those of negative controls on the same plate. The normalized viability values were used to calculate AUCs for the dose range of [100pM, 100µM]. AUC values were then normalized by the dose range to obtain normalized AUC values whose typical range was [0, 1].
IMPROVE drug screening study 3
Drug responses were evaluated in two colorectal cancer cell lines, nine colorectal cancer PDO lines (developed by the HCMI), and one sarcoma PDO line. The two cell lines, HT-29 and SW620, were seeded at 2,000 cells/well in 96-well plates using a 3:4 medium:Matrigel mixture and pre-cultured for 3 days in serum-free medium. The PDO screening was divided into two phases. In the first phase, five colorectal cancer PDO lines (PDM-2, PDM-5, PDM-7, PDM-8, and PDM-9) were expanded for approximately 3–4 passages and similarly seeded at 2,000 cells/well. In the second phase, four additional colorectal cancer PDO lines (PDM-4, PDM-95, PDM-103, and PDM-185) and one sarcoma PDO line (SARC0377TH) were expanded for approximately 1–2 passages. In drug screening experiments, colorectal cancer organoids were plated at 2,000 cells/well and sarcoma organoids at 5,000 cells/well. Both colorectal cancer cell lines and PDO lines were purchased from the ATCC.
For the cell line and the first PDO screens, 100 compounds were tested at seven concentrations from 100pM to 100µM in 10-fold serial dilutions, with three replicate wells per concentration. For the second organoid screen, compounds were tested at four concentrations from 0.01µM to 10µM, also in 10-fold dilutions and triplicate. Cells were treated twice on two consecutive days. Each plate contained 1% DMSO negative controls (8 wells per plate) and 10 µM staurosporine positive controls (4 wells per plate). Twenty-four hours after the final treatment, viability was measured using the CellTiter-Glo 3D ATP-release assay. ATP-release viability measurements of test conditions were normalized to those of negative controls on the same plate. The normalized viability values were used to calculate AUCs for the dose range of [100pM, 100µM]. AUC values were then normalized by the dose range. Assay quality was evaluated using Z factor and robust Z factor. A threshold of 0.2 was specified for both Z factor metrics; plates were rerun if neither Z factor reached 0.2.
Pilot 1 drug screening study
PDO lines were generated using an adapted epithelial cell-only submerged Matrigel culture method, as described in [9]. Tumor specimens were transported in chilled RPMI and dissociated to single-cell suspensions using a GentleMACs instrument with the Miltenyi Tumor Dissociation Kit. Cells were embedded in growth-factor-reduced (GFR) Matrigel at cells/L and plated as -L droplets in 24-well plates. After polymerization at , growth medium containing Y-27632 was added. PDOs were maintained in chemically defined, serum-free Advanced DMEM/F12-based medium supplemented with B27, nicotinamide, N-acetylcysteine, recombinant growth factors including EGF, Noggin, R-spondin 1, Wnt-3A, and FGF-2, -7, and -10, and inhibitors of p38 MAPK and TGF. Medium was changed twice weekly, and organoids were passaged every 1–2 weeks using TrypLE Express. Cultures showing sustained biomass expansion were cryopreserved for experiments after reaching cells.
For drug screening, established PDO lines were dissociated into single cells and resuspended in a 30:70% mixture of GFR Matrigel and growth medium. Approximately 2,000 cells were plated per well in 384-well plates and cultured for 72 hours before treatment. Drugs were prepared in growth medium containing Caspase-3/7 green apoptosis assay reagent and added using an automated pipetting system. Seven 10-fold serial dilutions were prepared for each compound, with a maximum concentration of . Organoids were then exposed to treatment for an additional 72 hours before high-content imaging.
After drug treatment, organoids were incubated with Hoechst 33342 and TO-PRO-3 for 1.5–2 hours and imaged at magnification, capturing approximately 100–200 organoids per well. Z-stack images were projected into two dimensions and analyzed. Organoids and individual cells were identified from Hoechst staining, while dead/dying cells were identified using TO-PRO-3 and Caspase-3/7 signals. Viability was calculated as the percentage of viable cells within each organoid. Organoids containing fewer than three cells, the largest 1% of organoids, and wells containing fewer than 20 organoids were excluded. Mean viability across retained organoids in a well was taken as the viability readout and normalized to vehicle controls on the same plate. The normalized viability values were used to calculate normalized AUCs for the dose range of [100 pM, 100 M].
Cell Line Drug Screening Data
The cell line drug screening data were extracted from the IMPROVE benchmark [10]. The benchmark integrates drug response data from five publicly available cell line drug screening studies: the Cancer Cell Line Encyclopedia (CCLE) [5, 11], the Cancer Therapeutics Response Portal version 2 (CTRPv2) [12, 13], the Genentech Cell Screening Initiative (gCSI) [14, 15], and the Genomics of Drug Sensitivity in Cancer datasets GDSCv1 and GDSCv2 [16, 6, 17]. The numbers of experiments, drugs, and cell lines from each source are summarized in Table 1. Drug responses are represented by the normalized AUC values calculated for the dose range of [100 pM, 100 M].
Omics Data and Drug Feature Data
We performed RNA-seq experiments to generate gene expression data of PDO lines used in the four drug screening studies. TPM (transcripts per million reads mapped) values were first obtained and gene expression values were calculated as . Gene-level copy numbers, somatic mutations, and DNA methylations of HCMI PDO lines were retrieved from the Genomic Data Commons (GDC). Somatic mutations and gene-level copy numbers of PDO lines used in the Pilot 1 drug screening study were obtained using a targeted DNA-seq assay [9]. Cell line omics data were extracted from the IMPROVE benchmark [10]. The benchmark provides multiple omics modalities, including gene expressions, somatic mutations, copy numbers, and DNA methylations.
Drug feature data were also extracted from the IMPROVE benchmark [10], including SMILES strings, binary molecular fingerprints, and numeric molecular descriptors. The 512-bit Extended Connectivity Fingerprints were generated by the Morgan algorithm using the open-source package RDKit [18]. The molecular descriptors were calculated using the open-source Mordred package [19], and only 2D descriptors were kept for analysis. In our computation analysis, DRP models that require alternative drug representations, such as molecular graphs or SMILES-derived substructure sequences, generated these representations based on the drug SMILES strings.
Drug Response Prediction Models
Our analysis utilized six publicly available DRP models: UNO [20], IGTD [21], GraphDRP [2], HiDRA [22], DeepTTC [3], and LightGBM (LGBM) [4]. The models represent a range of machine learning and deep learning architectures, such as graph neural networks, convolutional neural networks, transformers, and gradient-boosted decision trees. All models use both cancer and compound features to predict drug response, with model-specific feature representations and architectures described in the Supplementary Information Section 1. In the current study, gene expression was used as the cancer molecular representation, while drug representations included molecular descriptors, fingerprints, molecular graphs, or SMILES-derived substructure sequences depending on the model. Unless otherwise specified, models were trained using the mean squared error (MSE) loss with a batch size of 256 for up to 200 epochs, with early stopping applied using a patience of 20 epochs. LGBM was trained using a separate set of model-specific hyperparameters and early-stopping criteria. Additional model-specific preprocessing, architecture, and training details are provided in Supplementary Information Section 1.
Analysis Schemes
We built DRP models and evaluated their prediction performance on PDO response under three different scenarios designed to assess how CCL and PDO data can be used for predicting PDO drug response: (1) training exclusively on CCL data, (2) joint training on CCL and PDO data, and (3) transfer learning with pretraining on CCL data followed by fine-tuning on PDO data (Figure 2). Two different data partition schemes, random split and cancer-blind split, were implemented for cross-validation analyses. In cancer-blind cross-validation, the train, validation and test sets did not share cancer cases, while this constrain did not apply in random cross-validation. 10-fold data partitioning with an 8:1:1 train, validation, and test ratio was used on CCL data in both random split and cancer-blind cross-validations. For PDO datasets, 10-fold data partitioning with an 8:1:1 train, validation, and test ratio was used in random cross-validation, while in cancer-blind cross-validation -fold data partitioning with an :1:1 train, validation, and test ratio was used, where is the number of organoid lines in the dataset. An exception was the Pilot 1 PDO dataset with 47 PDO lines, where 10-fold data partitioning with an 8:1:1 train, validation, and test ratio was also applied in the cancer-blind cross-validation. Predictive performance was quantified by comparing predicted and true normalized AUC values using the coefficient of determination () and the Pearson correlation coefficient (PCC). Prediction performances on PDO experiments and CCL experiments in the PDO drug screening studies were calculated separately.

Figure 2. Overview of the DRP analysis framework. Six DRP models were used to predict PDO drug response using three learning strategies: cell line-only training, joint cell line and PDO training, and transfer learning with pretraining on cell lines followed by fine-tuning on PDOs. Random splits performed 10-fold cross-validations, whereas cancer-blind splits used -fold data partitioning, where on IMPROVE datasets is the number of PDO lines in the dataset and on Pilot 1 dataset is 10.
Model Training on Cell Line Data Only
Since CCL drug screening data are available at a substantially greater scale than PDO data (See Table 1), we first evaluated the extent to which models trained exclusively on CCL data could generalize to PDO drug response prediction without target-domain adaptation. In this cell line only training scheme, no PDO molecular features and response data were used for model training. Each of the six DRP models was trained on the pooled CCL dataset comprising CCLE, CTRPv2, GDSCv1, GDSCv2, and gCSI data through cross-validation. Model performance was first evaluated on the CCL test set. The trained models were then applied to each PDO dataset to assess model generalizability to PDO response prediction. This analysis therefore provides a zero-shot evaluation of cross-domain generalization from CCLs to PDOs and serves as the baseline for assessing the benefit of incorporating PDO data through joint training (§ 2.8.2) or transfer learning (§ 2.8.3).
Model Training on Cell Line and PDO Data Jointly
We next evaluated whether incorporating PDO data during model training could improve the prediction on PDO drug response. In the joint-training scheme, a separate model was trained by combining the train and validation sets of a CCL cross-validation partition with the train set of a PDO cross-validation partition, while the validation set of the PDO cross-validation partition was used for hyper-parameter optimization (HPO) and early stop of model training. The test set of the PDO cross-validation partition was used to evaluate the prediction performance on PDO response. The CCL experiments in PDO drug screening studies were never included in model training, but served as an additional test set for performance evaluation. As a quality control measure, we also evaluated the prediction performance on the test set in the CCL cross-validation partition. Under the random split scheme, the 10 CCL cross-validation partitions and the 10 PDO cross-validation partitions produced 100 models for each PDO dataset. Under the cancer-blind scheme, 10 models were trained for each of the IMPROVE PDO datasets and 100 models were trained for the Pilot 1 PDO dataset.
Transfer Learning from Cell Lines to PDOs
In the joint training scheme, the PDO data contribute to only a very small fraction of the train and validation sets used in model development, which might limit the benefit of including PDO data in the model training process. To address this limitation, we applied transfer learning, in which models were first pretrained on CCL data and then fine-tuned using PDO data. Specifically, each of the 10 models pretrained via the CCL cross-validation had its parameters updated through continuous training using the train and validation set of a PDO cross-validation partition. Then, the prediction performance of the updated model was evaluated using the test set of the PDO cross-validation partition. The CCL experiments in PDO drug screening studies were never included in model training, but served as an additional test set for performance evaluation. As a quality control measure, we also evaluated the prediction performance on the test set in the CCL cross-validation partition. Under the random split scheme, 100 fine-tuned models were generated for each PDO dataset. Under the cancer-blind scheme, 10 fine-tuned models were trained for each of the IMPROVE PDO datasets and 100 models were trained for the Pilot 1 PDO dataset.
Transfer learning analyses under both cross-validation schemes were conducted for GraphDRP and LGBM. For GraphDRP, we froze early layers in the model and updated the parameters of late layers during fine-tuning on PDO data [23]. The number of trainable layers was treated as a hyper-parameter to be optimized with other hyper-parameters using the differential evolution approach [24, 25]. For LGBM, transfer learning was implemented by adding additional boosting steps to the pre-trained model. Additional details of the HPO process are provided in Supplementary Information Section 2.
Results
Results
Correlation Between PDO Drug Screening Studies
As Figure 1 shows, the three IMPROVE drug screening studies share significant numbers of experiments, based on which we can evaluate the consistency of drug response measurements across studies. For each pair of the IMPROVE studies, we calculated the PCC of normalized AUC values based on experiments included in both studies, with a scatterplot visualization showing the distributions of AUC values (Figure 3). The correlation of response measurements varies across the three IMPROVE studies. The strongest correlation is observed between Datasets 1 and 3 (PCC = 0.645, = 594), where denotes the number of common experiments, followed by Datasets 1 and 2 (PCC = 0.547, = 240). Datasets 2 and 3 exhibited the weakest correlation (PCC = 0.371, = 280). All three correlations are at a moderate level, indicating a substantial variability remained between AUC measurements generated from studies using different experimental protocols and assay technologies.

Figure 3. Pairwise comparison of drug response measurements across the three IMPROVE studies. Scatterplots show the normalized AUC values of matched experiments between a pair of IMPROVE datasets. Each point represents an experiment measured in both studies. PCC and the number of matched experiments are reported for each comparison.
Prediction Performance
Model Training Using CCL Data Only
Table 2 shows the prediction performance () in random split cross-validations. Models achieve an average ranging from 0.73 to 0.79 on the test set of CCL data with the deep learning models outperforming the conventional machine learning method LGBM. Performance decreases substantially when the same models are applied on data of PDO drug screening studies, including both CCL and PDO experiments in these studies. This demonstrates a challenge on model generalizability caused by both the biological gap between CCLs and PDOs and the difference of experimental protocols and assay technologies used in different drug screening studies. The average PDO prediction performance across models is similar on IMPROVE datasets 1 and 3 ( = 0.25 and 0.24), while the prediction performance on Pilot 1 dataset is lower ( = 0.07) and the prediction performance on IMPROVE dataset 2 is the lowest ( = −0.26). Performance difference across model architectures are more pronounced on PDO data than the test set of CCL data. The PDO prediction performance on IMPROVE dataset 1, IMPROVE dataset 3, and Pilot 1 has ranges of [0.03, 0.34], [0.12, 0.33], and [0.02, 0.12], respectively, while no model can always perform the best on all datasets.
| Model | CCL Test | IMPROVE Dataset 1 CCL | IMPROVE Dataset 1 PDO | IMPROVE Dataset 2 CCL | IMPROVE Dataset 2 PDO | IMPROVE Dataset 3 CCL | IMPROVE Dataset 3 PDO | Pilot 1 PDO |
| DeepTTC | ||||||||
| UNO | ||||||||
| GraphDRP | ||||||||
| HiDRA | ||||||||
| LGBM | ||||||||
| IGTD | ||||||||
| Model Avg. |
Table 2. Prediction performance (, mean s.d.) of models trained exclusively on CCL data in random split cross-validations. CCL Test indicates the test set of CCL data. Prediction performance on CCL experiments and PDO experiments in IMPROVE datasets 1-3 are listed separately.
In cancer-blind cross-validation (Table 3), prediction performance on the test set of CCL data is lower than that observed in random-split cross-validation, with an average across models of 0.68 compared with 0.77. This difference indicates that cancer-blind cross-validation, which simulates a precision oncology application, presents a more challenging prediction task than random-split cross-validation, which simulates a drug repurposing application. The prediction performance on PDO data measured by the model-average does not change much on all PDO datasets compared with the random-split cross-validation, which is reasonable because the PDO lines are not included in the CCL data for modeling training in both random split and cancer-blind cross-validations. In contrast, prediction performance on PDO data, measured by the average across models, does not change much compared with random-split cross-validation for all PDO datasets. This is expected because the PDO lines are not included in the CCL training data under either cross-validation scheme.
| Model | CCL Test | IMPROVE Dataset 1 CCL | IMPROVE Dataset 1 PDO | IMPROVE Dataset 2 CCL | IMPROVE Dataset 2 PDO | IMPROVE Dataset 3 CCL | IMPROVE Dataset 3 PDO | Pilot 1 PDO |
| DeepTTC | ||||||||
| UNO | ||||||||
| GraphDRP | ||||||||
| HiDRA | ||||||||
| LGBM | ||||||||
| IGTD | ||||||||
| Model Avg. |
Table 3. Prediction performance (, mean s.d.) of models trained exclusively on CCL data in cancer-blind cross-validations. CCL Test indicates the test set of CCL data. Prediction performance on CCL experiments and PDO experiments in IMPROVE datasets 1-3 are listed separately.
Prediction performances measured by PCC in the random split and cancer-blind cross-validations are reported in the Supplementary Information Section 3 and show the same general pattern as the prediction performance.
Model Training Using CCL and PDO Data Jointly
Model Training Using CCL and PDO Data Jointly
Tables 4 shows the prediction performance of models trained using CCL and PDO data jointly in random split cross-validations. Compared with training models using only CCL data (Table 2), incorporating PDO data directly into model training substantially improves PDO prediction performance. The model-average for PDO prediction increases from 0.25 to 0.72 for IMPROVE dataset 1, from −0.26 to 0.55 for IMPROVE dataset 2, from 0.24 to 0.71 for IMPROVE dataset 3, and from 0.07 to 0.73 for Pilot 1 dataset. Prediction performance on the test set of CCL data averaged across models and studies is , which is similar to that of using only CCL data to train models (), indicating that the improvement on PDO prediction does not require a substantial loss of performance in predicting CCL response. Prediction performance on the CCL experiments from the IMPROVE PDO drug screening studies also improves slightly when PDO data are included in model training, with increasing from 0.27 to 0.37 for IMPROVE Dataset 1, from −0.51 to −0.49 for IMPROVE Dataset 2, and from −0.14 to 0.04 for IMPROVE Dataset 3. This result suggests that incorporating PDO experiments conducted using the same experimental protocols and assay technologies as the CCL experiments into model training can improve model generalizability to these CCL experiments.
The prediction performance in the cancer-blind cross-validation largely follows the same pattern of the prediction performance obtained in the random split cross-validation. When PDO data are included in model training, PDO prediction performance increases substantially on three out of the four PDO drug screening datasets, increasing from 0.25 to 0.48 on IMPROVE dataset 1, from 0.27 to 0.59 on IMPROVE dataset 2, and from 0.11 to 0.57 on Pilot 1, measured by average across models. However, the PDO prediction performance in cancer-blind cross-validation is lower than that of random split cross-validation, which again indicates model generalizability across cancer cases is more challenging.
Across model architectures, UNO always achieves the highest or tied-highest PDO prediction performance on all four datasets in random split cross-validation and two out of the four datasets in cancer-blind cross-validation. Overall, joint training reduces the CCL-to-PDO prediction gap for most datasets, but its effectiveness depends on whether predictions involve previously observed or entirely unseen PDO lines. Prediction performance measured by PCC generally supports the same trends observed on prediction performance results and is provided in the Supplementary Information Section 3.
Transfer Learning From CCLs To PDOs
| Model | IMPROVE Dataset 1 | IMPROVE Dataset 2 | IMPROVE Dataset 3 | Pilot 1 | |||||||
| CCL Test | Study CCL | Study PDO | CCL Test | Study CCL | Study PDO | CCL Test | Study CCL | Study PDO | CCL Test | Study PDO | |
| GraphDRP | -0.13 ± 1.25 | 0.56 ± 0.06 | 0.79 ± 0.06 | 0.49 ± 0.15 | -0.05 ± 0.15 | 0.55 ± 0.28 | 0.56 ± 0.08 | 0.14 ± 0.13 | 0.73 ± 0.05 | 0.47 ± 0.07 | 0.76 ± 0.04 |
| LGBM | -1.16 ± 0.53 | 0.65 ± 0.02 | 0.75 ± 0.08 | 0.10 ± 0.07 | 0.16 ± 0.17 | 0.64 ± 0.16 | 0.44 ± 0.07 | -0.12 ± 0.08 | 0.73 ± 0.05 | 0.07 ± 0.16 | 0.75 ± 0.04 |
Table 6. Transfer learning prediction performance (, mean ± s.d.) of models in random split cross-validations. CCL Test indicates the test set of CCL data. Study CCL and Study PDO indicate the CCL experiments and the test set of PDO experiments in a PDO drug screening study, respectively. Pilot 1 study does not include Study CCL experiments.
| Model | IMPROVE Dataset 1 | IMPROVE Dataset 2 | IMPROVE Dataset 3 | Pilot 1 | |||||||
| CCL Test | Study CCL | Study PDO | CCL Test | Study CCL | Study PDO | CCL Test | Study CCL | Study PDO | CCL Test | Study PDO | |
| GraphDRP | 0.04 ± 0.39 | 0.59 ± 0.07 | 0.58 ± 0.17 | 0.32 ± 0.18 | 0.06 ± 0.12 | -1.20 ± 3.37 | 0.38 ± 0.13 | 0.14 ± 0.12 | 0.60 ± 0.23 | 0.43 ± 0.14 | 0.57 ± 0.12 |
| LGBM | -0.37 ± 0.75 | 0.61 ± 0.07 | 0.50 ± 0.22 | 0.28 ± 0.29 | 0.17 ± 0.17 | -1.44 ± 3.69 | 0.38 ± 0.12 | -0.27 ± 0.20 | 0.60 ± 0.32 | 0.05 ± 0.18 | 0.63 ± 0.09 |
Table 7. Transfer learning prediction performance (, mean ± s.d.) of models in cancer-blind cross-validations. CCL Test indicates the test set of CCL data. Study CCL and Study PDO indicate the CCL experiments and the test set of PDO experiments in a PDO drug screening study, respectively. Pilot 1 study does not include Study CCL experiments.
Tables 6 shows the transfer learning prediction performance () in random split cross-validations. Compared with joint model training using CCL and PDO data (Table 4), transfer learning further improves the PDO prediction performance for both GraphDRP and LGBM models on all four PDO datasets with an improvement range of [0.03, 0.11]. This result indicates that fine-tuning models on PDO data through transfer learning is more effective in improving PDO prediction performance than training models on combined PDO and CCL data, in which PDO data constitute only a very small fraction of the training and validation data. For the prediction performance on CCL experiments from PDO drug screening studies, transfer learning improves the performance of GraphDRP on all three IMPROVE studies and improve the performance of LGBM on two of the IMPROVE studies. The prediction performance on the CCL test set decreases substantially for both GraphDRP and LGBM across all four PDO studies compared with joint training on CCL and PDO data, indicating that the improvement in PDO prediction performance achieved through transfer learning comes at the cost of forgetting previously learned CCL drug response patterns.
| Model | IMPROVE Dataset 1 | IMPROVE Dataset 2 | IMPROVE Dataset 3 | Pilot 1 | |||||||
| CCL Test | Study CCL | Study PDO | CCL Test | Study CCL | Study PDO | CCL Test | Study CCL | Study PDO | CCL Test | Study PDO | |
| DeepTTC | 0.78 ± 0.01 | 0.39 ± 0.06 | 0.74 ± 0.11 | 0.76 ± 0.02 | −0.51 ± 0.10 | 0.55 ± 0.20 | 0.78 ± 0.01 | −0.03 ± 0.10 | 0.73 ± 0.04 | 0.77 ± 0.01 | 0.74 ± 0.06 |
| UNO | 0.79 ± 0.01 | 0.30 ± 0.04 | 0.77 ± 0.09 | 0.77 ± 0.02 | −0.50 ± 0.08 | 0.58 ± 0.23 | 0.79 ± 0.01 | −0.11 ± 0.10 | 0.74 ± 0.04 | 0.79 ± 0.01 | 0.77 ± 0.04 |
| IGTD | 0.74 ± 0.03 | 0.34 ± 0.05 | 0.70 ± 0.12 | 0.68 ± 0.19 | −0.48 ± 0.13 | 0.51 ± 0.31 | 0.75 ± 0.02 | 0.06 ± 0.12 | 0.71 ± 0.05 | 0.75 ± 0.02 | 0.73 ± 0.05 |
| HiDRA | 0.76 ± 0.00 | 0.37 ± 0.03 | 0.77 ± 0.05 | 0.76 ± 0.00 | −0.59 ± 0.04 | 0.54 ± 0.28 | 0.76 ± 0.00 | 0.16 ± 0.03 | 0.72 ± 0.03 | 0.76 ± 0.00 | 0.70 ± 0.04 |
| LGBM | 0.74 ± 0.01 | 0.45 ± 0.01 | 0.69 ± 0.02 | 0.74 ± 0.00 | −0.37 ± 0.01 | 0.58 ± 0.02 | 0.74 ± 0.01 | 0.16 ± 0.01 | 0.69 ± 0.04 | 0.74 ± 0.00 | 0.72 ± 0.00 |
| GraphDRP | 0.76 ± 0.02 | 0.39 ± 0.04 | 0.68 ± 0.11 | 0.75 ± 0.03 | −0.48 ± 0.06 | 0.52 ± 0.26 | 0.77 ± 0.01 | −0.02 ± 0.10 | 0.68 ± 0.04 | 0.77 ± 0.01 | 0.73 ± 0.05 |
| Model Avg. | 0.76 ± 0.02 | 0.37 ± 0.05 | 0.72 ± 0.04 | 0.74 ± 0.03 | −0.49 ± 0.07 | 0.55 ± 0.03 | 0.77 ± 0.02 | 0.04 ± 0.11 | 0.71 ± 0.02 | 0.76 ± 0.02 | 0.73 ± 0.02 |
Table 4. Prediction performance (, mean ± s.d.) of models trained using CCL and PDO data jointly in random split cross-validations. CCL Test indicates the test set of CCL data. Study CCL and Study PDO indicate the CCL experiments and the test set of PDO experiments in a PDO drug screening study, respectively.
Tables 7 shows the transfer learning prediction performance () in cancer-blind cross-validations. Compared with model training using combined CCL and PDO data (Table 5), transfer learning improves the PDO prediction performance of GraphDRP on three out of the four PDO datasets by a range of [0.01, 0.07] and improves the PDO prediction performance of LGBM on all four PDO datasets by a range of [0.02, 0.18]. As observed in random-split cross-validation, transfer learning also improves prediction performance in cancer-blind cross-validation on the CCL experiments from the PDO drug screening studies, with improvements observed for GraphDRP across all three IMPROVE studies and for LGBM in two of the three studies. Similar to the results observed in random-split cross-validation, prediction performance on the CCL test sets decreases substantially compared with joint training on CCL and PDO data, further highlighting the trade-off between CCL and PDO prediction performance in transfer learning.
| Model | IMPROVE Dataset 1 | IMPROVE Dataset 2 | IMPROVE Dataset 3 | Pilot 1 | |||||||
| CCL Test | Study CCL | Study PDO | CCL Test | Study CCL | Study PDO | CCL Test | Study CCL | Study PDO | CCL Test | Study PDO | |
| DeepTTC | 0.68 ± 0.01 | 0.50 ± 0.04 | 0.57 ± 0.16 | 0.67 ± 0.04 | −0.20 ± 0.09 | −0.98 ± 2.77 | 0.68 ± 0.03 | −0.04 ± 0.11 | 0.61 ± 0.26 | 0.68 ± 0.02 | 0.57 ± 0.13 |
| UNO | 0.69 ± 0.01 | 0.46 ± 0.04 | 0.60 ± 0.13 | 0.68 ± 0.02 | −0.21 ± 0.07 | −1.08 ± 2.97 | 0.69 ± 0.01 | −0.02 ± 0.08 | 0.63 ± 0.26 | 0.69 ± 0.02 | 0.59 ± 0.12 |
| IGTD | 0.31 ± 0.54 | 0.42 ± 0.12 | 0.40 ± 0.11 | 0.34 ± 0.55 | −0.16 ± 0.16 | −1.48 ± 3.74 | 0.64 ± 0.04 | −0.01 ± 0.17 | 0.56 ± 0.31 | 0.64 ± 0.03 | 0.56 ± 0.10 |
| HiDRA | 0.67 ± 0.01 | 0.50 ± 0.02 | 0.46 ± 0.14 | 0.67 ± 0.01 | −0.37 ± 0.05 | −1.82 ± 4.55 | 0.68 ± 0.01 | −0.03 ± 0.07 | 0.60 ± 0.20 | 0.68 ± 0.01 | 0.54 ± 0.08 |
| LGBM | 0.66 ± 0.01 | 0.44 ± 0.01 | 0.32 ± 0.02 | 0.69 ± 0.01 | −0.31 ± 0.03 | −1.61 ± 0.32 | 0.62 ± 0.01 | 0.07 ± 0.02 | 0.54 ± 0.01 | 0.68 ± 0.02 | 0.61 ± 0.02 |
| GraphDRP | 0.68 ± 0.01 | 0.45 ± 0.04 | 0.51 ± 0.15 | 0.66 ± 0.04 | −0.29 ± 0.07 | −1.23 ± 3.45 | 0.68 ± 0.01 | 0.02 ± 0.11 | 0.59 ± 0.25 | 0.68 ± 0.01 | 0.57 ± 0.13 |
| Model Avg. | 0.61 ± 0.15 | 0.46 ± 0.03 | 0.48 ± 0.11 | 0.62 ± 0.14 | −0.26 ± 0.08 | −1.37 ± 0.33 | 0.66 ± 0.03 | 0.00 ± 0.04 | 0.59 ± 0.03 | 0.67 ± 0.02 | 0.57 ± 0.02 |
Table 5. Prediction performance (, mean ± s.d.) of models trained using CCL and PDO data jointly in cancer-blind cross-validations. CCL Test indicates the test set of CCL data. Study CCL and Study PDO indicate the CCL experiments and the test set of PDO experiments in a PDO drug screening study, respectively.
Performance results measured by PCC are generally consistent with the patterns observed in the results and are reported in the Supplementary Information Section 3.
Discussion
A major contribution of our study is the generation and evaluation of new PDO drug screening datasets for DRP research. Large-scale CCL pharmacogenomic datasets have been widely used for developing DRP models, but they do not fully capture the biological context in which these models are ultimately intended to be applied. PDOs provide an experimental system that more closely preserves the biological characteristics of patient tumors. However, PDO drug screening data remain limited, and their utility for drug response modeling has not been extensively investigated. We conducted four PDO drug screening studies using multiple screening platforms and experimental settings, generating drug response measurements along with molecular profiles of the cancer models and drug feature data. These studies focused on colorectal cancer and head and neck cancer, for which existing DRP models have shown limited prediction accuracy. The compounds used in the screening studies were selected to cover major MoAs relevant to current cancer therapies and oncology research.
We evaluated the utility of our datasets for developing and evaluating DRP models for PDO response prediction using three training strategies: (1) training models exclusively on CCL data, (2) training models jointly on CCL and PDO data, and (3) pretraining models on CCL data followed by fine-tuning on PDO data. Models trained exclusively on CCL data generally retained limited predictive power on PDO datasets, indicating that some molecular and pharmacological relationships learned from CCL screening data can transfer to PDO systems. Incorporating a portion of the PDO data into model training improved PDO response prediction performance, demonstrating that even though the target-domain data constituted only a small fraction of the overall training set, they provided valuable information that was not captured by CCL screening data but was important for accurate PDO response prediction. Transfer learning, in which models were pretrained on CCL data and subsequently fine-tuned on PDO data, generally further improved PDO response prediction performance, although the magnitude of improvement varied across models and datasets.
We evaluated the models under two different data partitioning schemes: random-split cross-validation and cancer-blind cross-validation. In general, prediction performance was higher under random-split cross-validation than under cancer-blind cross-validation when all other factors, such as model, training data, and training strategy, were held constant. This finding is expected, because generalizing to unseen cancer cases is more challenging than predicting responses for cancer cases represented in the training data. The patterns of variation in prediction performance across models, datasets, and training strategies are largely consistent between random-split and cancer-blind cross-validation.
PDO prediction performance on IMPROVE Dataset 2 was consistently the lowest among the four drug screening studies, with negative values observed in some cases, indicating that the models performed worse than a predictor that always predicts the mean response. Upon reviewing the experimental procedures, we identified an issue with insufficient mixing of the cell culture medium, which resulted in uneven cell seeding across wells and introduced substantial noise into the data. We believe this issue was a major contributor to the poor prediction performance on this dataset. Interestingly, even when values were low or negative, PCC prediction performance sometimes remained moderate to high, suggesting that the models could capture the relative ranking of drug responses despite failing to accurately predict their magnitudes.
Our IMPROVE drug screening studies include overlapping experiments across studies using different screening platforms and experimental settings, allowing an evaluation on the reproducibility of drug response measurements. Moderate levels of correlations in the range of [0.371, 0.645] are observed on response measurements from different studies, highlighting an important challenge for both drug screening experiments and PDO response modeling. Reproducibility of drug screening experiments need to be improved. Future efforts to expand PDO pharmacogenomic resources should consider experimental standardization, consistent quality-control procedure, and study or site effects as integral components of benchmark design. Most computational DRP models can only learn relationships that are represented consistently in their target data, and variations of response measurements caused by differences in screening platforms may therefore limit model generalizability even when the underlying biological response is similar. AI approaches that explicitly consider measurement variations may be useful for DRP tasks across datasets. Nevertheless, datasets generated across heterogeneous experimental settings remain valuable because they provide a more stringent test of whether predictive relationships are robust to realistic sources of measurement variation.
We have observed that transfer learning can improve prediction performance in the target PDO domain compared with both training exclusively on CCL data and joint training on CCL and PDO data, but at the cost of forgetting previously learned CCL response patterns. More sophisticated transfer learning approaches may improve prediction performance in the target domain while preserving performance in the source domain. One such approach is source replay, which incorporates data from both the source and target domains during model fine-tuning, with target-domain data accounting for a substantial proportion of the fine-tuning dataset. Because source-domain datasets are typically much larger than target-domain datasets, a small subset of source-domain data can be randomly sampled and combined with target-domain data for fine-tuning.
Based on the findings of our study, several directions for future research can be identified. First, new computational methods are needed to improve cross-domain and cross-study generalization, particularly by accounting for systematic differences in experimental platforms and drug response distributions. Second, although our current DRP analyses use only gene expression data to characterize cancer models, we have made additional multi-omics data for CCLs and PDOs, including copy number alterations, somatic mutations, and DNA methylation profiles, available through our online data repository. Incorporating these additional molecular modalities into DRP modeling may improve prediction performance and generalizability. Third, despite our efforts to generate new PDO drug screening and molecular datasets, available data resources for PDO drug response modeling remain limited. Substantially larger and more diverse datasets spanning a broader range of cancer types, PDO models, and compounds are needed to support the development of accurate and generalizable DRP models. Fourth, further research is needed to bridge the gap between PDO drug response modeling and the prediction of treatment responses in patients. Integrating CCL, PDO, and patient-level data for predicting drug responses in patient tumors will require more sophisticated computational approaches, such as multi-domain transfer learning.
Overall, our work provides both new datasets and a computational framework for PDO drug response modeling. Our results demonstrate that large-scale CCL datasets remain valuable for learning generalizable predictive relationships but are insufficient to ensure robust prediction of PDO responses. Incorporating even limited PDO data into model training can substantially improve prediction performance in the target domain. At the same time, variability in drug response measurements arising from differences in experimental protocols and technologies, together with reduced generalization performance on unseen cancer cases, highlights two distinct challenges that must be addressed as DRP models advance toward applications in therapeutic development and patient care. Expanding and harmonizing PDO drug screening datasets, while developing computational methods that explicitly account for these sources of variability, will be important steps toward building DRP models that more effectively leverage patient-derived experimental data.
Data and Code Availability
Code. The code used for training and evaluating DRP models in this paper is available here:
https://github.com/zhuyitan/PDO_DRP_Analysis
Data. The full dataset is available at:
https://doi.org/10.5281/zenodo.22920452
Acknowledgement
This research was part of the Innovative Methodologies and New Data for Predictive Oncology Model Evaluation (IMPROVE) project under the National Cancer Institute - Department of Energy (NCI-DOE) collaboration program that has been funded in whole or in part with federal funds from the National Cancer Institute, National Institutes of Health, Task Order No. 75N91019F00134 and from the Department of Energy under Award Number ACO22002-001-00000.
Supplementary Information
Details of Drug Response Prediction Models
Six publicly available DRP models were evaluated using implementations within the IMPROVE framework [10]. The models differ in their neural network architectures and in how they represent cancers and compounds. Unless otherwise specified, deep learning models were optimized using the mean squared error (MSE) loss, a batch size of 256 when training on CCL data, and a maximum of 200 training epochs, with early stopping applied after 20 epochs without a significant improvement on validation performance.
UNO
UNO [20] is a feed-forward deep neural network consisting of separate fully-connected subnetworks for cancer cases and drugs. The outputs of these subnetworks are concatenated and passed through additional fully-connected layers to predict drug response. Cancer cases were represented using gene expression profiles, while drugs were represented using molecular descriptors and fingerprints. To reduce the dimensionality of the gene expression input, only the LINCS L1000 landmark genes were included [26], which is a curated set of approximately 1,000 genes designed to capture a large proportion of transcriptomic variation.
IGTD
Image Generator for Tabular Data (IGTD) [21] converts tabular features into two-dimensional image representations that can be processed by convolutional neural networks (CNNs). Features are assigned to pixel positions in images such that correlated features are positioned close to each other. In this study, IGTD was applied separately to cancer gene expression profiles and drug molecular descriptors, generating two image representations for each sample that were processed by separate CNN subnetworks. The resulting representations were concatenated and used to predict drug response.
GraphDRP
GraphDRP [2] is a graph-based deep learning model that represents drugs as molecular graphs, with atoms represented as nodes and chemical bonds represented as edges. Graph convolutional layers are used to learn drug representations directly from molecular structure. Cancer cases are represented using fixed-length molecular feature vectors. In this study, gene expressions were used as the cancer input. The learned drug and cancer representations are concatenated and passed through fully-connected layers to predict drug response. Although GraphDRP can incorporate additional omics feature types, including mutations and copy-numbers, only gene expressions were used in our analysis.
HiDRA
HiDRA (Hierarchical Network for Drug Response Prediction with Attention) [22] is an attention-based deep learning model that incorporates prior biological pathway information into the cancer representation. Gene expression features are first aggregated into pathway-level representations using gene-level attention. A second attention mechanism is then applied across pathways to generate an integrated representation. Drug structure is represented using 512-bit molecular fingerprints, and the drug and cancer representations are combined to predict drug response. The hierarchical attention architecture also provides a mechanism for identifying genes and pathways that contribute to model predictions. To improve computational efficiency, we modified the original HiDRA implementation by vectorizing the pathway-level computation. The original implementation processes each of the 186 pathways separately in a sequential order; in the modified implementation, pathway representations are combined into a single tensor so that pathway-level computations can be performed parallelly. This modification reduced model training time by more than 10 folds while producing highly consistent prediction outcomes.
LGBM
Light Gradient-Boosting Machine (LGBM) [4] is a tree-based gradient boosting machine that constructs an ensemble of decision trees sequentially, with each tree trained to reduce the residual prediction error of the preceding ensemble. LGBM uses histogram-based feature binning and leaf-wise tree growth to efficiently model nonlinear relationships between input features and prediction target. Cancer cases were represented using gene expression profiles and drugs were represented using Mordred molecular descriptors. Models were trained for up to 10,000 boosting rounds with early stopping after 500 consecutive rounds without an improvement on validation performance. When training on CCL data, the maximum number of terminal leaves was set to 16, the maximum tree depth was set to 5, and the minimum number of samples per terminal leaf was set to 128 to limit model complexity and reduce overfitting.
DeepTTC
DeepTTC [3] is a transformer-based deep learning model that predicts drug response using cancer gene expression profiles and drug substructure representations. Drug SMILES strings are decomposed into sequences of chemical substructures using an explainable substructure partition fingerprint method based on byte-pair encoding. These substructure sequences are then processed using a transformer encoder, allowing self-attention to model relationships among chemical substructures within each compound. Cancer gene expressions are encoded separately using a fully connected neural network. The resulting drug and cancer-sample representations are concatenated and passed through a final fully-connected prediction network to estimate drug response.
Transfer Learning Implementation and Hyperparameter Optimization
Layer Freezing
The model architecture of GraphDRP consists of three components: a drug encoder, a cancer encoder, and a prediction head that combines the cancer and drug embeddings to predict drug response. Transfer learning was implemented by freezing the early layers in the prediction head and updating its late layers using PDO data. To adjust the degree of adaptation during transfer learning, we varied the number of trainable layers within the prediction head. The drug and cancer encoders remained fixed during fine-tuning. The number of trainable layers was taken as a hyper-parameter and selected based on validation performance.
Hyper-parameter Optimization
Hyper-parameters involved in model fine-tuning were optimized independently for each transfer learning analysis using differential evolution (DE) [24, 25]. DE is a population-based optimization algorithm in which individual members of the population represent candidate hyper-parameter configurations and fitness is determined by model performance on the validation set. Candidate configurations were iteratively generated through mutation and recombination, with configurations yielding improved validation performance retained for subsequent generations. Candidate solutions were initialized using Latin hypercube sampling to provide distributed coverage of the hyper-parameter search space. Mutation was performed using the best 1bin strategy with mutation dithering over the interval . The recombination probability was set to 0.7. For GraphDRP, the optimization jointly considered the number of trainable layers, learning rate, dropout rate, and batch size. Learning rate was optimized from to using logarithmic scaling. Dropout rate was optimized in the range of 0 to 0.90. Batch size was selected from . DE sampled 20 candidate hyper-parameter configurations in each iteration and run for six optimization iterations, resulting in 120 candidate configurations evaluated for each modeling analysis. For LGBM, optimization considered the learning rate and the minimum number of samples per leaf. Learning rate was optimized over the range 0.001–0.05, and the minimum number of samples per leaf was optimized between 2–64. DE sampled 10 candidate hyper-parameter configurations in each iteration and run for six optimization iterations, resulting in 60 candidate configurations evaluated per modeling analysis.
Prediction Performance Measured by PCC
| Model | CCL Test | IMPROVE Dataset 1 | IMPROVE Dataset 1 | IMPROVE Dataset 2 | IMPROVE Dataset 2 | IMPROVE Dataset 3 | IMPROVE Dataset 3 | Pilot 1 |
| CCL | PDO | CCL | PDO | CCL | PDO | PDO | ||
| DeepTTC | ||||||||
| UNO | ||||||||
| GraphDRP | ||||||||
| HiDRA | ||||||||
| LGBM | ||||||||
| IGTD | ||||||||
| Model Avg. |
Table 8. Prediction performance (PCC, mean s.d.) of models trained exclusively on CCL data in random split cross-validations. CCL Test indicates the test set of CCL data. Prediction performance on CCL experiments and PDO experiments in IMPROVE datasets 1-3 are listed separately.
| Model | CCL Test | IMPROVE Dataset 1 | IMPROVE Dataset 1 | IMPROVE Dataset 2 | IMPROVE Dataset 2 | IMPROVE Dataset 3 | IMPROVE Dataset 3 | Pilot 1 |
| CCL | PDO | CCL | PDO | CCL | PDO | PDO | ||
| DeepTTC | ||||||||
| UNO | ||||||||
| GraphDRP | ||||||||
| HiDRA | ||||||||
| LGBM | ||||||||
| IGTD | ||||||||
| Model Avg. |
Table 9. Prediction performance (PCC, mean s.d.) of models trained exclusively on CCL data in cancer-blind cross-validations. CCL Test indicates the test set of CCL data. Prediction performance on CCL experiments and PDO experiments in IMPROVE datasets 1-3 are listed separately.
| Model | IMPROVE Dataset 1 | IMPROVE Dataset 2 | IMPROVE Dataset 3 | Pilot 1 | |||||||
| CCL Test | Study CCL | Study PDO | CCL Test | Study CCL | Study PDO | CCL Test | Study CCL | Study PDO | CCL Test | Study PDO | |
| DeepTTC | |||||||||||
| UNO | |||||||||||
| IGTD | |||||||||||
| HiDRA | |||||||||||
| LGBM | |||||||||||
| GraphDRP | |||||||||||
| Model Avg. |
Table 10. Prediction performance (PCC, mean ± s.d.) of models trained using CCL and PDO data jointly in random split cross-validations. CCL Test indicates the test set of CCL data. Study CCL and Study PDO indicate the CCL experiments and the test set of PDO experiments in a PDO drug screening study, respectively.
| Model | IMPROVE Dataset 1 | IMPROVE Dataset 2 | IMPROVE Dataset 3 | Pilot 1 | |||||||
| CCL Test | Study CCL | Study PDO | CCL Test | Study CCL | Study PDO | CCL Test | Study CCL | Study PDO | CCL Test | Study PDO | |
| DeepTTC | |||||||||||
| UNO | |||||||||||
| IGTD | |||||||||||
| HiDRA | |||||||||||
| LGBM | |||||||||||
| GraphDRP | |||||||||||
| Model Avg. |
Table 11. Prediction performance (PCC, mean ± s.d.) of models trained using CCL and PDO data jointly in cancer-blind cross-validations. CCL Test indicates the test set of CCL data. Study CCL and Study PDO indicate the CCL experiments and the test set of PDO experiments in a PDO drug screening study, respectively.
| Model | IMPROVE Dataset 1 | IMPROVE Dataset 2 | IMPROVE Dataset 3 | Pilot 1 | |||||||
| CCL Test | Study CCL | Study PDO | CCL Test | Study CCL | Study PDO | CCL Test | Study CCL | Study PDO | CCL Test | Study PDO | |
| GraphDRP | |||||||||||
| LGBM |
Table 12. Transfer learning prediction performance (PCC, mean ± s.d.) of models in random split cross-validations. CCL Test indicates the test set of CCL data. Study CCL and Study PDO indicate the CCL experiments and the test set of PDO experiments in a PDO drug screening study, respectively. Pilot 1 study does not include Study CCL experiments.
| Model | IMPROVE Dataset 1 | IMPROVE Dataset 2 | IMPROVE Dataset 3 | Pilot 1 | |||||||
| CCL Test | Study CCL | Study PDO | CCL Test | Study CCL | Study PDO | CCL Test | Study CCL | Study PDO | CCL Test | Study PDO | |
| GraphDRP | |||||||||||
| LGBM |
Table 13. Transfer learning prediction performance (PCC, mean ± s.d.) of models in cancer-blind cross-validations. CCL Test indicates the test set of CCL data. Study CCL and Study PDO indicate the CCL experiments and the test set of PDO experiments in a PDO drug screening study, respectively. Pilot 1 study does not include Study CCL experiments.
References
- [1]A Partin, T S Brettin, Y Zhu, O Narykov, et al. “Deep learning methods for drug response prediction in cancer: Predominant and emerging trends”. In: Frontiers in Medicine 10 (2023).DOI
- [2]T. Nguyen, G. T. T. Nguyen, et al. “Graph Convolutional Networks for Drug Response Prediction”. In: IEEE/ACM Transactions on Computational Biology and Bioinformatics (2021).DOI
- [3]Likun Jiang, Changzhi Jiang, Xinyu Yu, Rao Fu, Shuting Jin, and Xiangrong Liu. “DeepTTA: a transformer-based model for predicting cancer drug response”. In: Briefings in Bioinformatics 23.3 (2022), b bac100. DOI: 10.1093/bib/bbac100.DOI
- [4]Yu Shi, Guolin Ke, Damien Soukhavong, James Lamb, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, Tie-Yan Liu, Nikita Titov, and David Cortes. lightgbm: Light Gradient Boosting Machine. R package version 4.7.0.99. 2026. URL: https://github.com/lightgbm-org/LightGBM.
- [5]Jordi Barretina, Giordano Caponigro, Nicolas Stransky, Kavitha Venkatesan, Adam Margolin, Sungjoon Kim, Christopher Wilson, Joseph Lehár, Gregory Kryukov, Dmitry Sonkin, et al. “The Cancer Cell Line Encyclopedia Enables Predictive Modelling of Anticancer Drug Sensitivity”. In: Nature 483.7391 (2012), pp. 603–607.DOI
- [6]W Yang, J Soares, P Greninger, E J Edelman, H Lightfoot, S Forbes, et al. “Genomics of Drug Sensitivity in Cancer (GDSC): a resource for therapeutic biomarker discovery in cancer cells”. In: Nucleic Acids Research (2013).
- [7]U. Ben-David, B. Siranosian, G. Ha, et al. “Genetic and transcriptional evolution alters cancer cell line drug response”. In: Nature 560 (2018), pp. 325–330. DOI: 10.1038/s41586-018-0409-3.
- [8]M Weil, A Partin, N Kousa, et al. “The IMPROVE framework for automating and standardizing tumor drug response prediction modeling”. In: European Journal of Cancer 211 (2024).
- [9]Brian M Larsen, Madhavi Kannan, Lee F Langer, Benjamin D Leibowitz, Aicha Bentaieb, Andrea Cancino, Igor Dolgalev, Bridgette E Drummond, Jonathan R Dry, Chi-Sing Ho, et al. “A pan-cancer organoid platform for precision medicine”. In: Cell reports 36.4 (2021).DOI
- [10]Alexander Partin, Priyanka Vasanthakumari, et al. “Benchmarking community drug response prediction models: datasets, models, tools, and metrics for cross-dataset generalization analysis”. In: Briefings in Bioinformatics 27.1 (2026). DOI: 10.1093/bib/bbaf667.DOI
- [11]Mahmoud Ghandi, Franklin W Huang, Judit Jane-Valbuena, Gregory V Kryukov, et al. “Next-generation characterization of the Cancer Cell Line Encyclopedia”. In: Nature 569 (2019), pp. 503–508.DOI
- [12]B Seashore-Ludlow, M G Rees, J H Cheah, M Cokol, E V Price, M E Coletti, et al. “Harnessing Connectivity in a Large-Scale Small-Molecule Sensitivity Dataset”. In: Cancer Discovery 5.11 (2015), pp. 1210–1223.
- [13]M G Rees, B Seashore-Ludlow, J H Cheah, D J Adams, E V Price, S Gill, et al. “Correlating chemical sensitivity and basal gene expression reveals mechanism of action”. In: Nature Chemical Biology 12 (2016), pp. 109–116.DOI
- [14]C Klijn, S Durinck, E W Stawiski, P M Haverty, Z Jiang, H Liu, et al. “A comprehensive transcriptional portrait of human cancer cell lines”. In: Nature Biotechnology 33.3 (2015), pp. 306–312.DOI
- [15]P M Haverty, E Lin, J Tan, Y Yu, B Lam, S Lianoglou, et al. “Reproducible pharmacogenomic profiling of cancer cell line panels”. In: Nature 533 (2016), pp. 333–337.DOI
- [16]F Iorio, T A Knijnenburg, D J Vis, G R Bignell, M P Menden, M Schubert, et al. “A Landscape of Pharmacogenomic Interactions in Cancer”. In: Cell (2016).DOI
- [17]G Picco, E D Chen, L G Alonso, F M Behan, E Gonçalves, G Bignell, et al. “Functional linkage of gene fusions to cancer cell fitness assessed by pharmacological and CRISPR-Cas9 screening”. In: Nature Communications (2019).DOI
- [18]RDKit: Open-source cheminformatics. https://www.rdkit.org.
- [19]H Moriwaki, Y-S Tian, et al. “Mordred: A molecular descriptor calculator”. In: Journal of Cheminformatics 10 (2018).
- [20]Fangfang Xia, Jonathan Allen, Prasanna Balaprakash, Thomas Brettin, Cristina Garcia-Cardona, Austin Clyde, Judith Cohn, James Doroshow, Xiaotian Duan, Veronika Dubinkina, Yvonne Evrard, Yun Jae Fan, Jason Gans, Stewart He, Pinyi Lu, Sergei Maslov, Alexander Partin, Maulik Shukla, Eric Stahlberg, Justin M. Wozniak, Hyunseung Yoo, George Zaki, Yitan Zhu, and Rick Stevens. “A cross-study analysis of drug response prediction in cancer cell lines”. In: Briefings in Bioinformatics 23.1 (2022), bbab356. DOI: 10.1093/bib/bbab356.DOI
- [21]Yitan Zhu, Thomas Brettin, Fangfang Xia, Alexander Partin, Maulik Shukla, Hyunseung Yoo, Yvonne A. Evrard, James H. Doroshow, and Rick L. Stevens. “Converting tabular data into images for deep learning with convolutional neural networks”. In: Scientific Reports 11 (2021), p. 11325. DOI: 10.1038/s41598-021-90923-y.DOI
- [22]I Jin and H Nam. “HiDRA: hierarchical network for drug response prediction with attention”. In: J Chem Inf Model 61 (2021), pp. 3858–3867. DOI: 10.1021/acs.jcim.1c00706.DOI
- [23]Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. “How transferable are features in deep neural networks?” In: Advances in Neural Information Processing Systems. Ed. by Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Weinberger. Vol. 27. Curran Associates, Inc., 2014. URL: https://proceedings.neurips.cc/paper_files/paper/2014/file/532a2f85b6977104bc938f8580abb303-Paper.pdf.arxiv.org/abs/1411.1792
- [24]R. Storn and K. Price. “Differential Evolution – A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces”. In: Journal of Global Optimization 11 (1997), pp. 341–359. DOI: 10.1023/A:1008202821328.DOI
- [25]Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, et al. “SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python”. In: Nature Methods 17.3 (2020), pp. 261–272. DOI: 10.1038/s41592-019-0686-2.DOI
- [26]Aravind Subramanian, Rajiv Narayan, Steven M. Corsello, David D. Peck, Ted E. Natoli, Xiaodong Lu, Joshua Gould, John F. Davis, Andrew A. Tubelli, Jacob K. Asiedu, et al. “A Next Generation Connectivity Map: L1000 Platform and the First 1,000,000 Profiles”. In: Cell 171.6 (2017), 1437–1452.e17. DOI: 10.1016/j.cell.2017.10.049.DOI