Qwen3-8B-Critic-SFT-Qwen-only
An 8B critic from
Steer, Don't Solve: Training Small Critic Models for Large Code Agents, trained on critiques of Qwen3-Next-80B-A3B trajectories only. It is one arm of the training-corpus ablation (Table 3). The main 8B critic, trained on CWM plus Qwen3-Next trajectories, is
Qwen3-8B-Critic-SFT.
The critic reads a coding agent's trajectory every k steps and returns a structured critique: detected error categories, evidence, a recovery action, task status, and one line of overall guidance. It does not write the patch.
All released models and datasets are listed on the
organization page. Code and configs are in the
critic-training repository.
Where it appears in the paper
| Paper location | Row label |
|---|
| Table 3, corpus ablation | 8B, Qwen-only |
Original run name: qwen3-8b-full-sft-prm-r2egym-swebench-instructions-k5-qwen-only-opus-distill-32k-lr5e6-multiturn.
Training data
- Tasks: 483 R2E-Gym instances from matplotlib, moto, and sympy, disjoint from SWE-bench Verified.
- Agent that produced the trajectories: Qwen3-Next-80B-A3B-Instruct, 483 trajectories.
- Teacher: Claude Opus 4.6, queried every 5 agent steps with the high-level prompt.
This corpus is the Qwen half of critic-sft-cwm-qwen. It is smaller than the CWM half because Qwen3-Next finishes tasks in fewer steps, so fewer critique points are collected per trajectory.
Training setup
Identical to Qwen3-8B-Critic-SFT apart from the data. Full-parameter SFT with LLaMA-Factory, config finetuning/qwen3_8b_critic_full_sft_l40s_train_multiturn_resumable.yaml.
| Setting | Value |
|---|
| Base model | Qwen/Qwen3-8B |
| Chat template | qwen3_nothink |
| Sequence length | 32,768 tokens |
| Loss | final critique turn only (mask_history: true) |
| Hardware | 8 x L40S, effective batch 8 |
| Optimizer | AdamW, lr 5e-6, cosine, warmup ratio 0.1 |
| Epochs | 3 |
| Precision | bf16 |
The trainer saved this run's weights in fp32. The checkpoint here was cast to bf16 before upload, which is the precision it was served in for every result in the paper.
Results
Resolve rate on SWE-bench Verified, from Table 3 of the paper.
| Coding agent | No critic | + this critic | + Qwen3-8B-Critic-SFT (CWM + Qwen data) |
|---|
| Qwen3-Next-80B-A3B | 20.0 | 26.2 | 25.2 |
| Qwen3-32B | 8.8 | 10.6 | 13.8 |
Training only on Qwen3-Next trajectories gives the best result on Qwen3-Next itself and the weakest on Qwen3-32B. The mixed corpus is the better default across agents.
How to use
Same serving and launch procedure as
Qwen3-8B-Critic-SFT: serve with vLLM in bf16 and pass the served name to
scripts/run_critic_max150.sh with
--prm. The served name must have an entry in
mini-swe-agent/configs/litellm_model_registry.json; add one for this model if you use a new name.
Citation
1@misc{gandhi2026steerdontsolvetraining,
2 title={Steer, Don't Solve: Training Small Critic Models for Large Code Agents},
3 author={Shubham Gandhi and Yiqing Xie and Atharva Naik and Ruichen Zhu and Carolyn Rose},
4 year={2026},
5 eprint={2606.21811},
6 archivePrefix={arXiv},
7 primaryClass={cs.SE},
8 url={https://arxiv.org/abs/2606.21811}
9}
