RNA post-transcriptional modifications strongly influence the context-dependent processes dictating RNA localization, translation and organelle interactions. Yet, large-scale experimental mapping of RNA modifications is infeasible, in particular when considering different cellular contexts. Accurate and scalable computational methods for inferring potentially modified sites are thus essential.
RNA language models (RNA-LMs), trained on millions of unlabelled sequences by predicting hidden nucleotides have shown promise for modeling RNA regulation. However, the biological information they capture at nucleotide resolution remains largely unexplored. Here, we systematically evaluate RNA-LM representations and show that these models learn universal RNA features, while also encoding fundamental biological properties. We find that RNA-LMs operate in two distinct regimes: low-confidence predictions driven by simple sequence properties, and high-confidence predictions informed partly by structural context. Chemically modified sites affect model confidence, for example, m⁶A sites have a much lower probability of being predicted as Adenosine compared with randomly sampled positions. We introduce a visualization tool to interrogate internal representations and show that RNA-LMs distinguish m⁶A sites, recovering the canonical DRACH motif and additional non-canonical motifs. Building on this, we develop a RNA modification predictor achieving high accuracy and enabling a genome-wide map of predicted modifications, including previously unannotated sites and combinatorial motif patterns.
These results establish RNA-LMs as powerful tools for decoding regulatory logic. Future work will explore the potential of RNA-LMs to explore the combinatorial code through which chemical modifications and RBP binding collectively specify RNA metabolism, including localization and translation.

Co-authors: Lisa Fournier, Michael Jopiti, Lonneke van der Plas, Pascal Frossard, Raphaëlle Luisier