Ruming — Custom-Persona LLM (Korean, On-Device)
Qwen2.5-3B fine-tuned with QLoRA on 1,000 curated synthetic dialogue pairs
designed around a 15-year best-friend persona. Quantized to GGUF Q4_K_M
(~2 GB) for offline Android inference via PocketPal AI.
Full code, data, and debugging postmortem:
https://github.com/mintae818/ruming-llm
Usage (PocketPal AI on Android)
- Models → Add from Hugging Face →
mintae818/ruming-qwen2.5-3b-q4km
- Create a Pal with the system prompt
- Chat offline.
Training
- Base: Qwen/Qwen2.5-3B-Instruct (4-bit NF4)
- LoRA: r=8, α=16, all linear projections, assistant-only loss
- Library: Unsloth + trl SFTTrainer
- Hardware: Kaggle T4 (~20 min)
- Data: 1,000 LLM-generated dialogue pairs, hand-curated across 5 topic categories
(dev 30% / gaming 25% / emotion 20% / daily 15% / career 10%)
Quantization
| File | Format | Size |
|---|
ruming.Q4_K_M.gguf | GGUF Q4_K_M | ~2 GB |
Sweet spot for 3B on flagship Android: fits memory, ~25 tok/s on Snapdragon
8 Elite Gen 5 (Galaxy S26 Ultra), minimal quality loss vs FP16.
License
Apache 2.0 (inherited from base model).
