from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

base_model_id = "Qwen/Qwen2.5-32B-Instruct"
adapter_id = "im-sangwoon/chatprot-qwen2.5-32b-lora"

# Load base model
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

# Load LoRA adapter
model = PeftModel.from_pretrained(model, adapter_id)

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(base_model_id, trust_remote_code=True)

# Inference
messages = [
    {"role": "system", "content": "You are a helpful assistant for protein analysis."},
    {"role": "user", "content": "What is the function of Hemoglobin subunit alpha?"}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, do_sample=True)
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)

With vLLM

bash

python -m vllm.entrypoints.openai.api_server \
    --model Qwen/Qwen2.5-32B-Instruct \
    --enable-lora \
    --lora-modules chatprot=im-sangwoon/chatprot-qwen2.5-32b-lora \
    --dtype bfloat16 \
    --trust-remote-code \
    --max-model-len 8192 \
    --max-lora-rank 64

Training Details

Training Data

im-sangwoon/protein-sft-uniprot - UniProt 데이터베이스에서 추출한 단백질 Q&A 데이터셋 (약 1,551,711건)

질문 유형:

단백질 공식 명칭 (Official names)
단백질 기능 (Function)
단백질 패밀리 분류 (Protein family)
세포 내 위치 (Subcellular location)

LoRA Configuration

Parameter	Value
Rank (r)	64
Alpha	16
Dropout	0.1
Target Modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Task Type	CAUSAL_LM

Training Hyperparameters

Parameter	Value
Epochs	3
Batch size (per device)	1
Gradient accumulation steps	8
Effective batch size	8
Learning rate	2e-4
LR scheduler	Cosine
Warmup ratio	0.03
Weight decay	0.001
Max grad norm	0.3
Optimizer	AdamW (fused)
Precision	bf16
Gradient checkpointing	Enabled

Hardware

GPU: NVIDIA A6000 x 8

Framework Versions

PEFT 0.18.0
TRL (SFTTrainer)
Transformers
PyTorch