GB.RNA-1.6B is a general-purpose RNA foundation model with 1.6 billion parameters, trained on 42 million non-coding RNA sequences at single-nucleotide resolution. It achieves state-of-the-art performance on a comprehensive set of tasks, including RNA secondary structure prediction, mRNA-related tasks, RNA function prediction, and RNA inverse folding. After domain adaptation, GB.RNA excels in modeling protein-level tasks, highlighting its potential to leverage the central dogma for enhancing biomolecular representations. For more detailed information, please refer to our paper.

Note: If you are interested in a model tailored for coding sequences, we also offer a domain-adapted version: GB.RNA-1.6B-CDS.

GB.RNA is an encoder-only transformer and is pre-trained using masked language modeling (MLM) objective. The model architecture parameters are as follows:

The pre-training data contains 42 million unique ncRNA sequences from RNAcentral version 24.0.

For more information, visit: Model Generator

Please cite GB.RNA using the following BibTeX code:

@inproceedings{zou_large-scale_2024,
	title = {A Large-Scale Foundation Model for RNA Function and Structure Prediction},
	url = {https://www.biorxiv.org/content/10.1101/2024.11.28.625345v1},
	doi = {10.1101/2024.11.28.625345},
	publisher = {bioRxiv},
	author = {Zou, Shuxian and Tao, Tianhua and Mahbub, Sazan and Ellington, Caleb N. and Algayres, Robin and Li, Dian and Zhuang, Yonghao and Wang, Hongyi and Song, Le and Xing, Eric P.},
	year = {2024},
    booktitle = {NeurIPS 2024 Workshop on AI for New Drug Modalities},
}