OS-Shepherd-9B is an open multimodal reward model for judging computer-use agent trajectories. Given a task instruction, screenshots, and the agent's reasoning and actions, it determines whether the task was completed and returns a reasoned SUCCESS or FAIL verdict.
It is fine-tuned from
Qwen3.5-9B on
OS-Shepherd-100K using SFT followed by GRPO, with the RL stage focused on reducing false-success judgments.
Use the canonical prompt and trajectory format from the
OSReward repository. A recent Transformers, vLLM, or SGLang version with Qwen3.5 multimodal support is required.
This model is intended for trajectory evaluation, data filtering, and reward-model research. It is not a computer-control policy and may still miss fine-grained visual failures, especially on hard cases.
Apache License 2.0. See LICENSE.
1@article{sun2026osreward,
2 title={OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models},
3 author={Sun, Qiushi and others},
4 journal={arXiv preprint arXiv:2607.28609},
5 year={2026}
6}
