LoRA adapter for Qwen2.5-72B-Instruct-AWQ, fine-tuned on 1973 Chinese political commentary Q&A pairs to simulate a "小粉红" (Little Pink / Chinese nationalist) persona.

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-72B-Instruct-AWQ",
    torch_dtype="float16",
    device_map="cuda:0"
)
model = PeftModel.from_pretrained(base_model, "lmxxf/little-pink-lora")
tokenizer = AutoTokenizer.from_pretrained("lmxxf/little-pink-lora")

Full project & interactive chat script: github.com/lmxxf/little-pink-ai

Training

Parameter	Value
Base Model	Qwen2.5-72B-Instruct-AWQ
Method	LoRA (r=16, alpha=32)
Training Data	1973 Q&A pairs
Epochs	3
Final Loss	0.2638
Training Time	18 hours
Hardware	NVIDIA DGX Spark (GB10, 128GB)
Framework	LLaMA-Factory

Training Hyperparameters

learning_rate: 1e-4
train_batch_size: 1
gradient_accumulation_steps: 8 (effective batch size = 8)
lr_scheduler_type: cosine
warmup_ratio: 0.1
num_epochs: 3
mixed_precision: fp16

Framework Versions

PEFT 0.17.1
Transformers 4.51.3
PyTorch 2.10.0a0+b558c986e8.nv25.11