Reviewer-7B is a fine-tuned on DeepSeek-R1-Distill-Qwen-7B, optimized for selecting the best patch among multiple patches generated by our DARS agent while solving software engineering problems.

We use vLLM to deploy and infer the model. Please follow this tutorial here to use our LoRA weights with vLLM.

We use our code review dataset where each instance contains several git patches with critiques for each each patch. The model learns to generate critiques for multiple patches and select the best patch.

We use training script provided in Qwen-2.5 codebase.

Using this model as a reviewer with DARS trajectories generated using Claude 3.5 Sonnet V2 achieves 38.7% on SWE-Bench Lite.