This repository is the original Orthrus code-and-checkpoints dump from the pre-print period. It is not loadable via AutoModel.from_pretrained — it ships a custom loader (load_model) and multiple raw .ckpt checkpoints under models/.
1from transformers import AutoModel
2
3model = AutoModel.from_pretrained("antichronology/orthrus-4-track", trust_remote_code=True)
4
5model.representation(x, lengths, channel_last=True) # (B, D) pooled
6model.representation_unpooled(x, channel_last=True) # (B, L, D) per-position
7model.predict_tokens(x, lengths, channel_last=True) # (B, L, 4) MLM logits, MLM repos only
See the README on any of the standardized repos for full setup instructions, GenomeKit-based 6-track input construction, and MLM scoring examples.
The files here are preserved for reproducibility of pre-print era work and receive no further updates:
If you specifically need the custom loader (for instance, to reproduce a number from the pre-print), the code matches what is in
bowang-lab/Orthrus on GitHub.
1@article{fradkinShi2026,
2 title = {Orthrus: toward evolutionary and functional RNA foundation models},
3 ISSN = {1548-7105},
4 url = {http://dx.doi.org/10.1038/s41592-026-03064-3},
5 DOI = {10.1038/s41592-026-03064-3},
6 journal = {Nature Methods},
7 publisher = {Springer Science and Business Media LLC},
8 author = {Fradkin, Philip and Shi, Ruian "Ian" and Dalal, Taykhoom and Isaev, Keren and Frey, Brendan J. and Lee, Leo J. and Morris, Quaid and Wang, Bo},
9 year = {2026},
10 month = Apr
11}
