H&E-stained histological images remain the gold standard in clinical pathology, providing cost-effective, rapid, and robust visualization of tissue architecture and cellular morphology. Recent advances in machine learning have substantially expanded their utility, enabling representation learning, cross-modal alignment, and prediction of spatial omics directly from H&E images. Multiple studies have been published, demonstrating that such predictions are feasible for metabolomics, transcriptomics, and proteomics at the slide, spot, and single-cell levels. Here, we focus on models that predict transcriptomics at single-cell resolution.

Despite rapid methodological development, rigorous and objective evaluation of these approaches remains limited. To address this, we benchmarked three methods: SpatialEx, a hypergraph-based model compatible with multiple pretrained image encoders; GHIST, a deep learning framework using a UNet3+ backbone trained from scratch on paired H&E and spatial transcriptomics data; and Pixel2Gene, which uses the HIPT pathology foundation model to extract histological features that are fed into a simple MLP. As controls, we included simple linear regression models trained on embeddings from multiple foundation models, totalling 16 models evaluated in this study.

We evaluated all models on in-house Xenium 5K panel data from acute myeloid leukemia skin tissue. To further validate our findings, we are extending this pipeline to publicly available Xenium skin datasets, including a 5K and a 300-gene panel dataset. Our results show that predictive performance of published models was limited for the majority of genes and, surprisingly, comparable to that of a simple linear regression model trained on foundation model embeddings.

Co-authors: Julia Staller, Namrata Singh, Martin Simon, Sabina Gansberger, Johannes Griss, Philipp Tschandl, Barbara Sternizky, Inigo Oyarzun