LoRA adapter for Qwen2.5-3B-Instruct, trained for Chinese role-play as part of the Anima project — a reproduction of a verifiable-reward (RLVR) GRPO recipe on a single RTX 4090. This is the SFT arm (source-disjoint RoleBench), used as the warm-start for the GRPO arm and as a baseline in a four-arm (Base / SFT / DPO / GRPO) leakage-controlled evaluation.

The model emits a structured reply <think><focus>…</focus><focus_attr>…</focus_attr></think> \boxed{reply}; the \boxed{} content is the in-character answer and <focus> carries the cognitive tags the rule-based reward optimizes.

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "Qwen/Qwen2.5-3B-Instruct"
tok = AutoTokenizer.from_pretrained(base, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "Bot42/Anima")

Results (four arms, public benchmarks)

axis	Base	SFT	DPO	GRPO
role-play heldout (rule replay)	0.00	0.82	0.82	0.82
quality proxy (4-bit scalar)	0.30	0.41	0.41	0.40
social/role MCQ	0.19	0.23	0.24	0.24
general-capability canary	0.46	0.53	0.51	0.53

SFT teaches the structured output contract and delivers most of the gain over Base; DPO/GRPO match SFT within noise (reported as measured, no superiority claim).

Training data & license

Trained on RoleBench (Apache-2.0) Chinese roles, source-disjoint train/heldout split.
Adapter weights: Apache-2.0. Base model: see the Qwen2.5 license.
Project code & full writeup: the Anima GitHub repo (https://github.com/Firefly0237/Anima).