Evo2 is a large-scale foundation model for genomic sequences. It can generate, complete, and predict functions for DNA/RNA sequences. The model supports pretraining on large-scale genomic datasets and can be rapidly adapted to downstream tasks such as regulatory element prediction and variant effect assessment, providing a unified sequence modeling tool for genomics research.
Model Description
Evo2 is built on a hybrid architecture that combines Hyena operators with convolutional gating, and supports context windows at the million-token scale. The model is pretrained on multi-species genomic data containing trillions of base pairs. It takes raw nucleotide sequences as input and supports autoregressive generation, sequence classification, and embedding extraction. Evo2 is compatible with multi-GPU distributed training and flexible data mixing, achieves leading performance on benchmarks such as GenBench, and is suitable for pretraining, fine-tuning, and downstream functional validation.
Applicable Scenarios
Scenario
Description
DNA/RNA sequence generation
Generate subsequent genomic sequence fragments from an input prompt
FASTA sequence prediction
Read FASTA input and output sequence prediction results
Local offline inference validation
Use the example inputs in this repository and a local checkpoint to verify the inference workflow end to end
Mini training workflow validation
Use data/data_mini/genome_data/ to validate the training scripts, data loading, and checkpoint workflow
Cluster training adaptation
Use scripts/train_slurm.py to run training in a Slurm/cluster environment
Data preprocessing
Generate the .bin/.idx data required by the Byte-Level tokenizer from FASTA/JSON data
Usage
1. Using OneCode
You can try intelligent one-click AI4S programming through the OneCode online environment:
If the scripts need to access the OneScience source code, set:
export ONESCIENCE_ROOT=/path/to/onescience
When run from this directory, the scripts automatically add the project root and model/ to sys.path. You can also manually verify that the local package can be imported:
chr20.fa, chr21.fa, chr22.fa, and their compressed versions
Merged FASTA: chr20_21_22.fa
Preprocessed binary data for training, validation, and testing: preprocessed_data/
The dataset_prefix in config/genome_data_config.yaml is a relative path. During actual runs, it should be used together with:
--dataset-dir data/data_mini/genome_data
By default, the 7B NeMo checkpoint is placed inside the repository:
checkpoints/evo2_nemo_7b/
The inference and prediction entry points read this directory by default; training does not load a checkpoint by default. To fine-tune from existing weights or resume training, explicitly pass --ckpt-dir checkpoints/evo2_nemo_7b in the training command. You can also use EVO2_CKPT_DIR or --ckpt-dir to override it with another checkpoint path:
export EVO2_CKPT_DIR=/path/to/evo2_nemo_7b
The weight files are located under /path/to/evo2/checkpoints/evo2_nemo_7b/weights.
5. Prompt-Based Generation Inference
infer.py writes results to --output-file, but it does not automatically create the parent directory. Create the output directory before the first run:
predict.py automatically creates --output-dir. If --fasta is not passed, it reads data/predict_example.fa by default; if --output-dir is not passed, it writes output to outputs/predict by default.
Common parameters:
Parameter
Default
Description
--fasta
data/predict_example.fa
Input FASTA file
--ckpt-dir
checkpoints/evo2_nemo_7b
NeMo checkpoint directory; can be overridden with EVO2_CKPT_DIR
--output-dir
outputs/predict
Prediction result directory
--batch-size
1
Prediction batch size
--model-size
7b
Model specification; available values are determined by NeMo HYENA_MODEL_OPTIONS
--output-log-prob-seqs
Disabled
Output sequence log probability
data/data_mini/genome_data/chr20.fa, chr21.fa, and chr22.fa are relatively large chromosome-level FASTA files. Direct prediction may consume substantial GPU memory. It is recommended to split them into shorter fragments first, or use a multi-GPU parallel configuration.
7. Training
The training entry point requires one of two options: pass -d/--dataset-config, or use --mock-data. For the built-in example data, config/genome_data_config.yaml and data/data_mini/genome_data are recommended.
Example for training the 1B architecture from scratch or validating the training workflow:
This example does not load a checkpoint and is suitable for validating the training workflow when 1B weights are unavailable. To fine-tune from a 1B NeMo checkpoint, additionally pass --ckpt-dir /path/to/evo2_nemo_1b.
Example for training the 7B long-context architecture from scratch or validating the training workflow:
This example does not load a checkpoint. If you need to fine-tune from an existing 7B NeMo checkpoint or resume training, additionally pass --ckpt-dir checkpoints/evo2_nemo_7b or another checkpoint path.
In a Slurm/cluster environment, replace the entry point with:
python scripts/train_slurm.py ...
Common training parameters:
The table below covers the main parameters used in the training examples above and in scripts/train_evo2_1b.sh and scripts/train_evo2_7b.sh. For more advanced parameters, run python scripts/train.py --help.
Parameter
Default
Description
-d, --dataset-config
Required, or use --mock-data
Training data configuration YAML
--mock-data
Disabled
Do not read real data; use mock data for connectivity testing. Mutually exclusive with -d/--dataset-config
--dataset-dir
EVO2_DATASET_DIR or data/data_mini/genome_data
Data root directory
--ckpt-dir
Empty
Initial checkpoint directory; pass explicitly when fine-tuning from existing weights or resuming training
--model-size
7b
Model specification. Common values include 1b, 7b, and 7b_arc_longcontext; use test for testing
--devices
1
Number of devices on a single node
--num-nodes
1
Number of nodes
--tensor-parallel-size
1
Tensor parallel size
--pipeline-model-parallel-size
1
Pipeline model parallel size
--context-parallel-size
1
Context parallel size
--sequence-parallel
Disabled
Enable sequence parallel, usually used together with tensor parallelism
--seq-length
8192
Training sequence length
--micro-batch-size
1
Micro batch size
--global-batch-size
Automatically inferred
Global batch size
--grad-acc-batches
1
Number of gradient accumulation batches, used to infer the global batch size
--lr
3e-4
Learning rate, explicitly set to 0.0001 in the examples
--min-lr
3e-5
Minimum learning rate of the cosine scheduler
--warmup-steps
2500
Number of warmup steps, explicitly set to 5 in the examples
--constant-steps
80000
Number of steps for which the learning rate stays constant
--max-steps
500000
Number of training steps
--wd
0.01
Optimizer weight decay
--clip-grad
1.0
Gradient clipping threshold
--workers
8
Number of DataLoader workers
--activation-checkpoint-recompute-num-layers
Empty
Override the number of activation recomputation layers; explicitly set to 1 in the examples
--val-check-interval
Empty
Validation and checkpoint check interval; explicitly set to 50 in the examples
--limit-val-batches
20
Maximum number of batches to run per validation; explicitly set to 2 in the examples
--result-dir
./results
Log and result directory
--experiment-name
evo2
Experiment name, which affects the log directory
--disable-checkpointing
Not passed by default; checkpoint callback enabled
Pass this parameter during connectivity tests to disable the checkpoint callback
--no-save-last-checkpoint
Save the last checkpoint by default
Disable saving the last checkpoint; often used together with --disable-checkpointing for quick tests
--save-top-k
5
Number of best checkpoints to save
--ckpt-async-save
Disabled
Enable asynchronous checkpoint saving
8. Data Preprocessing
The current example training data has already been preprocessed. To process new FASTA or JSON data, use:
The original Evo2 code uses the Apache License 2.0. See LICENSE in this repository. Original license source: ArcInstitute/evo2 LICENSE.
The Evo2 7B model weights are sourced from Hugging Face arcinstitute/evo2_7b. In this repository, checkpoints/evo2_nemo_7b is the converted NeMo checkpoint format. The original weight license is Apache-2.0.
The example dataset data/data_mini/genome_data/ was downloaded and preprocessed from the hg38 chr20, chr21, and chr22 chromosome FASTA files provided by the UCSC Genome Browser. It is used only to validate the training and inference workflows. See the data download directory: UCSC hg38 chromosomes.
Use of UCSC hg38 data must comply with the UCSC Genome Browser data usage terms. See: UCSC Terms of Use.
If you use this repository, the model weights, or generated results in research, it is recommended to cite the original Evo2 paper, the model weight source, the UCSC hg38 data source, and relevant OneScience project information, and to add citations for downstream analysis tools or datasets according to the actual task.
