Qwen2.5-Coder-3B Code-Review (TAID) — MLC q4f16_1 (browser / WebGPU)
4-bit (q4f16_1) MLC build for in-browser WebLLM/WebGPU deployment. Layout:
mlc-chat-config.json + ndarray-cache.json + params_shard_*.bin. The student architecture is
stock Qwen2.5-Coder — load with the matching prebuilt WebLLM model lib. Held-out F1 0.478.
Use the prebuilt Qwen2.5-Coder-3B model lib.
Method. Cached-logit knowledge distillation on Apple MLX: run the teacher once, cache top-k=50
logits over its response positions, train a LoRA student (rank 16 / 16 layers, lr 1e-4, seq 768)
against them, fuse. Held-out eval eval_set_100 (100 labeled chunks, 73 buggy / 27 clean; py/js/c/go).
Prompt contract. Given a numbered code chunk, emit one JSON object
{"findings":[{category, subtype, severity, confidence, title, body, evidence, line}]}; evidence
is a verbatim source substring, line 1-based. Use repetition_penalty ~1.15 to keep JSON valid.
Project findings (honest). Teacher selection dominated: a 3B distilled from Gemma-2-9B ties the
Qwen-7B teacher (F1 0.509); per-category teacher routing added nothing (-0.003); 1.6x more
in-distribution data did not raise the ceiling. TAID (logit KD) > SFT with the Qwen-7B teacher
(0.478 vs 0.410) but both over-train past ~3000 steps on this small corpus.