You can deploy this model and start earning money today!

dnagpt/
OmniGene-4-CPT-v2-4bit

Name: OmniGene-4-CPT-v2-4bit
Availability: InStock

safetensors

gemma4

Views

No views yet

OmniGene-4-CPT-v2-4bit

BF16 model with automatic 4-bit quantization for RTX 5090 (32GB)

This model automatically quantizes to 4-bit when loaded, requiring only ~13GB GPU memory.

Model Description

OmniGene-4-CPT-v2-4bit is a biological foundation model with:

Base: Gemma-4-26B-A4B-Instruct (MoE, 128 experts, top-8 routing)
Vocabulary: 290,048 tokens (262,020 original + 28,028 bio tokens)
CPT data: 32.5 GB mixed corpus (DNA, Protein, OpenWebText, Structure)
Training: 0.6 epoch, 2,806 steps, 8×H20 GPUs
Storage: BF16 (~49 GB, 32 shards of ~1.5GB each)
Runtime: Automatic 4-bit quantization (~13GB GPU memory)

Quick Start

python

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load model (automatically quantizes to 4-bit)
model = AutoModelForCausalLM.from_pretrained(
    "dnagpt/OmniGene-4-CPT-v2-4bit",
    device_map="auto",  # Automatically applies quantization_config.json
)
tokenizer = AutoTokenizer.from_pretrained("dnagpt/OmniGene-4-CPT-v2-4bit")

# Generate
prompt = "MKTAYIAKQRQISFVKSHFSRQLEERLGLIEVQAPILSRVGDGTQDNLSGAEKAVQVKVKALPDAQFEVVHSLAKWKRQTLGQHDFSAGEGLYTHMKALRPDEDRLSPLHSVYVDQWDWERVMGDGERQFSTLKSTVEAIWAGIKATEAAVSEEFGLAPFLPDQIHFVHSQELLSRYPDLDAKGRERAIAKDLGAVFLVGIGGKLSDGHRHDVRAPDYDDWSTPSELGHAGLNGDILVWNPVLEDAFELSSMGIRVDADTLKHQLALTGDEDRLELEWHQALLRGEMPQTIGGGIGQSRLTMLLLQLPHIGQVQAGVWPAAVRESVPSLL"

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Hardware Requirements

GPU Memory: ~13-15GB (after automatic 4-bit quantization)
Recommended: RTX 5090 (32GB), RTX 4090 (24GB), or better
Minimum: RTX 3090 (24GB)

Quantization Details

This model uses bitsandbytes NF4 quantization with double quantization:

Method: NF4 (Normal Float 4-bit)
Compute dtype: bfloat16
Double quantization: Yes
Quality: Minimal accuracy loss compared to BF16

The quantization happens automatically when you load the model thanks to the included quantization_config.json.

Download Size vs Runtime Size

Download: ~49GB (BF16 weights, 32 shards)
Disk: ~49GB
GPU Memory: ~13GB (after automatic quantization)

The model is stored in BF16 for maximum quality, then quantized to 4-bit at load time.

Model Architecture

Layers: 30 transformer layers
Experts: 128 experts per layer (top-8 routing)
Hidden size: 2816
Attention heads: 22
Active parameters: ~3.8B per token
Total parameters: ~26B

Biological Tokens

The model includes 28,028 additional biological tokens:

DNA BPE: 20,000 tokens (optimized for genomic sequences)
Protein BPE: 8,000 tokens (optimized for amino acid sequences)
3Di alphabet: 20 tokens (Foldseek structural alphabet)
DSSP: 8 tokens (secondary structure: H, E, C, etc.)

Training Data

Source	Size	Tokens	Proportion
DNA (human genome)	8.0 GB	2.1B	24.6%
Protein (UniProt)	8.0 GB	2.1B	24.6%
Protein (LucaOne)	7.5 GB	2.0B	23.1%
OpenWebText	8.0 GB	2.1B	24.6%
Structure (3Di + DSSP)	0.4 GB	0.1B	1.2%
Instruction replay	0.6 GB	0.4B	1.9%

Other Versions

Full BF16 (no quantization): https://huggingface.co/dnagpt/OmniGene-4-CPT-v2-merged
LoRA adapter (requires base model): https://huggingface.co/dnagpt/OmniGene-4-CPT-v2
Instruction-tuned: https://huggingface.co/dnagpt/OmniGene-4-SFT-v3-4bit

Citation

bibtex

@article{wang2026omnigene4,
  title={OmniGene-4: A Unified Bio-Language MoE Model with Router-Level Interpretability},
  author={Wang, Liang},
  journal={bioRxiv},
  year={2026}
}

Paper

Full paper: https://github.com/maris205/omnigene4

License

Apache 2.0

Contact

Liang Wang (wangliang.f@gmail.com)
School of Artificial Intelligence and Automation
Huazhong University of Science and Technology