Base model: LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct
Method: GRPO with LoRA (rank=64) via TRL
Training framework: PEFT / TRL / Transformers
Hyperparameters: num_generations=8, learning_rate=7e-6, beta=0.04, lora_rank=64
WandB: k2zqndpm

Repository layout

Branch	설명
`main`	학습 종료 시점의 최종 어댑터
`checkpoint-100`	step 100
`checkpoint-200`	step 200
`checkpoint-300`	step 300
`checkpoint-400`	step 400
`checkpoint-500`	step 500
`checkpoint-600`	step 600
`checkpoint-679`	step 679

각 체크포인트는 학습 중간의 LoRA adapter만 포함 (optimizer/scheduler state 제외 — 추론용).

Usage

최종 어댑터 (main)

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

REPO = "xxccho/EXAONE-GRPO-lg_convfin_mcq_grpo_ratio1.0_gen8_bs4_lr7e-6_beta0.04"
BASE = "LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct"

tokenizer = AutoTokenizer.from_pretrained(REPO, trust_remote_code=True)
base = AutoModelForCausalLM.from_pretrained(BASE, trust_remote_code=True, torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, REPO)

prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "What is the price-to-earnings ratio?"}],
    tokenize=False, add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

특정 체크포인트

revision 인자로 branch를 지정:

model = PeftModel.from_pretrained(base, REPO, revision="checkpoint-100")

vLLM (LoRA 모듈로)

bash

vllm serve LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct \
    --enable-lora \
    --lora-modules ckpt=xxccho/EXAONE-GRPO-lg_convfin_mcq_grpo_ratio1.0_gen8_bs4_lr7e-6_beta0.04 \
    --max-lora-rank 64 \
    --trust-remote-code

License

Base model의 라이선스(EXAONE License)를 따른다.