Field	Value
Architecture	`Qwen3ForCausalLM`-compatible decoder-only Transformer
Parameters	1,120,772,224
Layers	19
Hidden / intermediate size	1,920 / 7,680
Attention heads / KV heads	15 / 15
Context	256 tokens: one BOS token followed by up to 255 DNA bases
Checkpoint	Final m5.1 checkpoint, step 59,158
Approximate token exposure	166.0B nucleotide tokens over the inherited training lineage
Stored weight dtype	float32
License	Apache-2.0

The source checkpoint is gs://marin-us-east5/checkpoints/dna-bolinas-mix-v0.9-p1B-i24-exp135-zoonomia-m5.1-bef41e/hf/step-59158. The commit-pinned training script defines the experiment.

Loading

python

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "marin-dna/marin-dna-exp135-m5.1"

tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)

The float32 checkpoint is approximately 4.18 GiB. Convert or load it in a lower-precision dtype only if that is appropriate for your downstream use.

Tokenizer and input format

The bundled tokenizer is a case-insensitive, single-nucleotide tokenizer with seven tokens:

Token	ID
`[PAD]`	0
`[UNK]`	1
`[BOS]`	2
`a`	3
`c`	4
`g`	5
`t`	6

Pass raw DNA strings containing A, C, G, and T without spaces or separators. The tokenizer lowercases input and automatically prepends [BOS]; the model has no EOS token and the tokenizer does not append one. Characters outside the four canonical bases map to [UNK]. Because BOS occupies one of the 256 model positions, inputs are limited to 255 DNA bases.

Training lineage and data

m5.1 is a continued-training lineage with approximately 166.0B total inherited and newly seen nucleotide tokens:

approximately 42.0B inherited tokens from the pre-cooldown point of a uniform three-region mixture (CDS, upstream, and downstream);
approximately 62.0B tokens from a continued uniform three-region mixture; and
approximately 62.0B tokens from a uniform five-region mixture adding enhancer and ncRNA sequence.

The five-region inventory below describes the resources used across the m5.1 lineage. It does not mean that all five datasets contributed to every training phase.

Training datasets

Matched training-validation probes

These matched datasets were validation probes, not training data.