from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

base_model_id = "Qwen/Qwen3-8B"
adapter_id = "Nemo0412/qwen3-8b-lipo"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id, torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(base_model, adapter_id)

Training Details

Base model: Qwen/Qwen3-8B
Method: LoRA (r=16, α=32) on all attention + FFN projection layers
Dataset: ~12,000 LNP virtual library samples with mRNA transfection efficiency labels
Epochs: 3 | Final train loss: 0.367
Hardware: NVIDIA H100 80GB