This model is a BERT-based DNA language model trained with the standard byte-pair encoding (BPE) tokenizer with vocabulary size 5,120. It is the baseline control model from the paper
EvoLen: Evolution-Guided Tokenization for DNA Language Model (COLM 2026).
This model uses a BERT-base architecture (12 layers, hidden size 768) with a vocabulary of 5,120 tokens. It was pretrained on human genome sequences using masked language modeling. The tokenizer is the standard BPE baseline described in the EvoLen repository.
This model is primarily intended as a baseline for evaluating the EvoLen tokenizer in DNA language model benchmarks. It can be fine-tuned for downstream genomic tasks, but it is not specifically designed for production use.
The model was pretrained on the human genome (hg38). Training sequences were tokenized into 512-token windows and a held-out evaluation set was used to report the following results.
