Views
No views yet
Qwen/Qwen3-Coder-30B-A3B-Instruct using RLOO over agentic
software-engineering tasks executed by OpenCode in sandboxed environments.Qwen/Qwen3-Coder-30B-A3B-Instruct (Qwen3 MoE, 48 layers)global_step_20rl-tasktrove-dq-sweep-30b-terminus2-qwen-20260725-163115-1ae770advantage_estimator: rloo_n) with megatron backend, tensor-parallel 4, pipeline-parallel 2,
expert-parallel 4, across 32 H100s. The objective is deliberately unregularized: use_kl_loss: false, use_entropy_loss: false, and policy_update_steps: 1, which leaves the PPO clip ratio
inert at 0.0. That choice makes entropy dynamics the primary failure mode across the sweep, and it
is why several arms ended early.torch.distributed.checkpoint megatron checkpoint by re-running the trainer's own
export path (bridge.save_hf_weights) at the checkpoint's own step, so no offline conversion was
involved. Shard count, index total_size and weight_map completeness were verified against the
object store before upload.last episode of each trial (per
make_and_upload_trace_dataset --episodes last) -- the rollouts the policy was trained on.training_logs/ holds the parsed metric surface for this run -- per-step training metrics,
vLLM engine metrics, a summary report, and the reward-vs-steps plot -- alongside the raw
trainer log. Capability tokens have been redacted from the logs.