Deep learning (DL) has become central to modern bioinformatics and health sciences, enabling applications ranging from gene expression -based diagnostics and patient stratification to cell type classification, and multi-omics data integration. Although these models have the potential to substantially improve clinical decision-making and patient outcomes, they often fail to generalize when applied to new cohorts. Without rigorous external validation using independent datasets, DL-based models risk overestimating performance by capturing dataset-specific artifacts rather than underlying biological signals. Biological and technical heterogeneity further exacerbate this issue by introducing substantial distribution shifts between datasets. Such shifts arise from differences in sequencing platforms, library preparation protocols, and patient demographics, leading to unreliable predictions outside the model's original training domain. This challenge is particularly pronounced in transfer learning and domain adaptation settings involving gene expression data, where the goal is often to transfer drug sensitivity predictions from in vitro cancer cell line models to patient tumors. We recently developed a tool, MODAE, which addresses this task by learning transferable representations using data from CCLE, GDSC, CTRP, and TCGA. However, validating the reliability of such models on truly external datasets remains a major challenge.
Here, we propose a diagnostic framework for assessing domain adaptation in deep learning models. This diagnostic was evaluated using MODAE and is designed to support the development of robust validation protocols for deep learning -based de-confounding autoencoders applied to external datasets. Ultimately, this approach provides a principled means to assess whether predictions generated on external cohorts can be considered trustworthy.

Co-authors: Teemu Rintala, Vittorio Fortino