Deep learning models have shown strong performance in modelling transcriptional expression patterns but often lack interpretability. In settings where perturbations follow an ordinal structure, such as dosage levels, promoting ordinality in the latent space may improve both robustness and biological insight.
We analyse single-nuclei RNA-seq data from 131,613 mouse liver cells exposed to nine concentration levels of TCDD using a variational autoencoder augmented with ordinal losses. We compare three ordinal loss functions (binomial cross-entropy, ordinal encoding, and cumulative logistic link) against a standard cross-entropy baseline, with random forest classifiers and regressors as additional baselines. Ablation studies evaluate the effect of classification loss weighting and VAE architecture. We further incorporate a pathway-constrained decoder, using Bayesian differential factor analysis to assess pathway activity in the latent space.
While cross-entropy achieves the highest predictive performance across ordinal-unaware and ordinal-aware metrics (including balanced accuracy, MSE, Kendall's tau, and AUOC), ordinal losses substantially alter the latent space structure. Binomial cross-entropy and Cumulative Logistic Link induce a continuous gradient aligned with increasing dosage, whereas Cross-entropy produces clustered representations and Ordinal Encoding partially preserves ordering. These structured embeddings correspond to biologically meaningful pathway activity patterns, including cholesterol biosynthesis. The pathway-constrained decoder improves performance across all models without changing their relative rankings.
These results show that encoding ordinality reveals biologically relevant structure in the latent space, highlighting a trade-off between predictive performance and interpretable representation learning. Future work will explore ordinal latent representations for generating expression profiles of unseen dosage perturbations.

Co-authors: Soufyan Lakbir