Best GRPO checkpoint of the real-only condition (round 5).
Part of the Halluminate blog post on sim-to-real transfer for web agents.
Tasks:
goodbuy-bestbuy-mirrored-tasks.
Training logs:
W&B report.
1from peft import PeftModel
2from transformers import Qwen2_5_VLForConditionalGeneration
3
4base = Qwen2_5_VLForConditionalGeneration.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct")
5model = PeftModel.from_pretrained(base, "victoria-kp/qwen25vl-7b-simtoreal-grpo-bestbuy")
GRPO from the corresponding SFT adapter: 8 rollouts per task, 8 tasks per
round, 10 rounds, binary matched-criterion reward from a deterministic
page-state verifier, KL 0.03 to the frozen SFT adapter, clip 0.2, lr 5e-6,
temperature 1.0. Trained on the Bestbuy split. This is the checkpoint with
the best held-out success (round 5).
