Qwen2.5-14B-Instruct-ORPO-Math

Recipe (SFT → ORPO, all license-clean)

1. Base: Qwen2.5-14B-Instruct (Apache-2.0), trained in bf16 (no quantization loss) on a 96GB GPU.
2. SFT — one epoch, LoRA r=64, LR 2e-4, on 7,474 GSM8K train CoT examples (MIT). Mean train loss 0.226.
3. ORPO — preference tuning on 7,000 GSM8K chosen/rejected pairs (β=0.1, bf16). Start loss ~0.06; log_odds_chosen ~5.6 at finish. Rejected responses derived locally (wrong final answer), no closed model used.
4. Eval — GSM8K test, 40-sample exact-match, native chat format, bf16: 72.5%.

License & provenance (important)

• Base weights: Qwen2.5-14B-Instruct — Apache-2.0 (open).
• Training data: openai/gsm8k — MIT (human-authored). No closed-source model was used in any step.
• Rejected ORPO responses were constructed locally from the correct solutions (swapped final answer) — no GPT-4/Claude/Gemini or proprietary distillation anywhere.
• Redistribution data: gsm8k (MIT).

Files

• adapter_model.safetensors + adapter_config.json — the SFT+ORPO LoRA (applies on Qwen/Qwen2.5-14B-Instruct).
• tokenizer.json / tokenizer_config.json — Qwen2.5 tokenizer.
• (GGUF quant if present in this repo.)

Honest limitations

• This is a math word-problem specialist tuned from GSM8K; it's strong on arithmetic/word problems but not a broad frontier model, and GSM8K is grade-school level.
• 72.5% is exact-match on a 40-sample test subset — directional, not a full benchmark.
• For general chat/creative use, prefer the base Qwen2.5-14B-Instruct.

Reproducibility seed