You can deploy this model and start earning money today!

micanonsens/
bigamba-dual-step44000

Name: bigamba-dual-step44000
Availability: InStock

Views

No views yet

BiGamba (MLM + MEM, step 44000)

Repository: micanonsens/bigamba-dual-step44000

The Gamba models are a family of DNA language models from the ArGamba paper that jointly model DNA sequence and evolutionary rate information.

This repository includes the model weights and code for the Caduceus-based bidirectional variant model (BiGamba) trained to predict both the human genome sequence and the Zoonomia 241-mammalian alignment-derived PhyloP scores using both Masked Language Modelling (MLM) and Masked Evolutionary rate Modelling (MEM) tasks. For more details, see the GitHub repo.

Model family

All Gamba family models have checkpoints available at 44,000 steps:

Checkpoint name	Architecture	Training task
ArGamba-dual	ArGamba (Jamba autoregressive)	NTP + CEP
ArGamba-seq_only	ArGamba (Jamba autoregressive)	NTP
ArGamba-cons_only	ArGamba (Jamba autoregressive)	CEP
BiGamba-dual	BiGamba (Mamba bidirectional)	MLM + MEM
BiGamba-seq_only	BiGamba (Mamba bidirectional)	MLM
BiGamba-cons_only	BiGamba (Mamba bidirectional)	MEM

Load

python

from transformers import AutoModel

model = AutoModel.from_pretrained(
    "micanonsens/bigamba-dual-step44000",
    trust_remote_code=True
)

Notes

This repository includes custom modeling code; trust_remote_code=True is required.
Ensure your environment has the necessary project dependencies installed (see GitHub).