Qwen2.5-Coder-14B — GRPO fine-tune for code-edit generation
GRPO (reinforcement) fine-tune of Qwen2.5-Coder-14B-Instruct for structured code editing: given a request and the current file state, the model emits a <file_changes> edit. Experimental, not deployed.
Method
GRPO with 4 rollouts per prompt; advantage = each candidate's reward relative to the group mean. Trained as a LoRA adapter, then merged.
| Setting | Value |
|---|
| Algorithm | GRPO |
| Rollouts per prompt | 4 |
| LoRA rank / alpha / dropout | 8 / 16 / 0.05 |
| Learning rate | 1e-4 |
| Epochs | 1 |
| QAT | on |
| KL coefficient | 0.001 |
| Clip ratio (dual-clip) | 0.2 / 0.2, c = 3.0 (DAPO-style) |
| Importance sampling | token-level |
| Aggregation | token-mean |
Reward
Rule-based, no LLM judge. The generated edit is parsed and scored 0–1 on: file-coverage F1 (0.30), code validity (0.25), output structure (0.15), action correctness (0.15), and similarity to ground truth (0.15).
Data
A proprietary, internal dataset of code-edit examples from Magically (an AI app builder). Not released. Largest training tier (~5,000 examples).
Intended use and limits
Research model for a code-editing / app-builder setting. Not safety-tuned, not production-grade, evaluated only on the internal reward above.
