This 4,021,773,440-parameter nucleotide-level causal language model is the largest member of the eight-model MarinDNA v0.5 parameter-scaling ladder developed with Marin. This repository contains only the final step-215573 checkpoint from run dna-bolinas-scaling-v0.5-h2944-p4B-fa02c3, with its tokenizer bundled. It accompanies A 1B standard Transformer rivals Evo 2 40B on variant effect prediction. It is the canonical ladder 4B, not one of the separate short dna-bolinas-4b-v0.* AdamH-search runs.

The canonical source checkpoint is gs://marin-us-east5/checkpoints/dna-bolinas-scaling-v0.5-h2944-p4B-fa02c3/hf/step-215573; the byte-identical evals_v2 transfer cache is s3://oa-bolinas/snakemake/analysis/evals_v2/results/checkpoints/scaling-v0.5-h2944-p4B-step-215573. The commit-pinned training script defines the production ladder.

The bundled tokenizer is case-insensitive and represents one nucleotide per token. Its vocabulary is [PAD]=0, [UNK]=1, [BOS]=2, a=3, c=4, g=5, t=6. Pass raw DNA strings containing A, C, G, and T without spaces. The tokenizer lowercases input and prepends [BOS]; it has no EOS token. Other symbols map to [UNK]. Because BOS occupies one of the 256 positions, inputs are limited to 255 DNA bases.

Every model in the ladder used the same batch size, token budget, optimizer hyperparameters, tokenizer, and three-way training mixture: 73.19% CDS, 20.62% upstream, and 6.19% downstream sequence. Lowercase soft-masked positions received loss weight 0.01, versus 1.0 for uppercase positions. The ladder changes model scale, not the training-data recipe.

Training datasets: CDS, upstream, and downstream. The matched CDS, upstream, and downstream datasets were validation probes, not training data. The exact tokenizer, data, and mixture definitions are commit-pinned here.