Model	Coverage	FalsePos	Severity	Action	Overall
GPT-4o (teacher)	2.62	3.46	2.38	3.46	2.62
GPT-4o-mini	2.77	3.38	2.62	3.38	2.77
v1 (base Qwen)	2.08	4.46	2.00	1.62	2.00
v2 (Instruct, this)	2.31	2.77	2.08	3.00	2.38

Usage with vLLM

Base: <a href="https://huggingface.co/unsloth/Qwen2.5-Coder-7B-Instruct" target="_blank" rel="noopener noreferrer" class="text-blue-400 hover:text-blue-300 underline underline-offset-2 transition-colors">unsloth/Qwen2.5-Coder-7B-Instruct
Fine-tuning: QLoRA (rank=32, alpha=32, dropout=0.05, all linear layers)
Quantization: AWQ 4-bit (~4 GB)
Training data: 73 PRs, GPT-4o-generated reviews
Eval: 13 hand-labeled PRs, GPT-4o-as-judge

vllm serve Hardik55/pr-reviewer-7b-instruct-awq --quantization awq

Lessons Learned

v1 vs v2 documents a real distillation lesson: with only 73 examples, fine-tuning a base model cannot teach instruction-following AND review style simultaneously. Switching to the Instruct base variant improved overall score 19% (actionability 85%) with no other changes.

Limitations

73 training examples is small; model often hedges
Distillation ceiling: cannot exceed teacher quality
AWQ calibrated on default wikitext-2 (not domain-specific)

Source

https://github.com/hardik5520/pr-reviewer-distillation