Base model: google/gemma-4-E2B-it
RNA embeddings: Precomputed TranscriptFormer (2048-d), projected via rna_projection.pt
LoRA: rank 32, alpha 64
Validation loss: 0.9227

Files

File	Description
`model.safetensors`	Merged LLM weights
`rna_projection.pt`	Linear map RNA embed (2048) → text hidden
`rna_model_config.json`	Architecture metadata

Loading

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained("mims-harvard/bio-posttrain-gemma4-e2b-rna-sft", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("mims-harvard/bio-posttrain-gemma4-e2b-rna-sft")
proj = torch.load("rna_projection.pt", map_location="cpu")

RNA sequence embeddings are supplied offline at inference time. See the rna_models code in the BioReason repository.

Collection

Part of the Bio-posttrain collection.