Views
No views yet
search_plan_actQwen/Qwen3-14B, trained with GRPO
(via TRL's multi-turn environment_factory tool-calling API) on
search_plan_act: a procedurally-generated, closed-book multi-turn task
requiring the model to search for facts, read_record/update_record/
link_records on retrieved entities, and finish, looping through tool
calls until the goal is achieved or it gives up. Reward decomposes into
outcome / grounding / stop-behavior components, with an explicit
adversarial-pilot check that no reward-hacking strategy (reckless guessing,
spamming searches, never finishing) scores above genuine goal-directed play.Qwen/Qwen3-14B) vs post-RL (this checkpoint) on
held-out real-world and synthetic benchmarks, all using vLLM-accelerated
generation with the model's own chat template (apples-to-apples generation
path in both rows):| Benchmark | Pre-RL | Post-RL (this checkpoint) |
|---|---|---|
| MuSiQue (EM / F1, n=2417) | 0.342 / 0.466 | 0.343 / 0.469 |
| 2WikiMultihopQA (EM / F1, n=12576) | 0.578 / 0.676 | 0.578 / 0.676 |
| search_plan_ood (mean reward, n=100) | 0.030 | 0.040 |
| BALROG BabyAI (mean episode return, n=50) | 0.000 | 0.000 |
search_plan_ood and is flat everywhere else, including BALROG (where
smaller Qwen3 sizes showed a small real gain). Compare against the sibling
bmonikraj/qwen3-8b-search-plan-act checkpoint, which showed much stronger,
more consistent transfer under the identical training recipe -- RL transfer
here is genuinely model-specific, not simply a function of scale.model.safetensors, config.json, tokenizer.json, tokenizer_config.json,
generation_config.json, chat_template.jinja -- the full merged checkpoint.adapter/ -- the raw LoRA adapter (pre-merge), kept for reference.