
Optimizer step	130 / 150
In-training validation success rate	82.0%
Backbone	`Qwen/Qwen2.5-1.5B-Instruct` (1.54B params, bf16)
Environment	ALFWorld `alfworld/AlfredTWEnv` (in-process Ray actors)
Hardware	2x NVIDIA A100 80GB
Wall-clock (full 150 steps)	~26 hours

Final evaluation (step 150)

Standalone vLLM harness, 128 ALFWorld valid_seen tasks, 3 seeds, temperature 0.4, max 50 env steps:

	value
success rate	84.38% ± 1.69
base model (`Qwen2.5-1.5B-Instruct`)	2.86% ± 0.37
mean episode length	16.42 (base: 49.15)
format rate	99.49% (base: 98.12%)

Training curve

In-training validation success rate (val/success_rate), measured every 5 optimizer steps on a 32-task validation batch. Note this is a smaller and noisier probe than the final 128-task evaluation above.

optimizer step	success rate (%)
5	6.2
40	29.7
50	40.6
55	35.2
60	49.2
65	57.0
70	51.6
75	54.7
80	51.6
85	55.5
90	58.6
95	61.7
100	65.6
105	63.3
110	75.0
115	72.7
120	75.0
125	85.2
130	82.0
135	80.5
140	73.4
145	78.9
150	80.5

Hyperparameters

group	parameter	value
RL	`adv_estimator`	gigpo
RL	`gigpo.mode`	mean_std_norm
RL	`gigpo.step_advantage_w`	1.0
RL	`algorithm.gamma`	0.95
RL	`actor.use_kl_loss` / `kl_loss_coef`	True / 0.01
RL	`kl_loss_type`	low_var_kl
RL	invalid action penalty	True, coef 0.1
RL	`algorithm.use_kl_in_reward`	False
Optim	learning rate	1e-6
Optim	`ppo_mini_batch_size`	256
Optim	`ppo_micro_batch_size_per_gpu`	32
Data	`train_batch_size` / `val_batch_size`	16 / 32
Data	group size (`env.rollout.n`)	8
Data	episodes per step	16 x 8 = 128
Data	`max_prompt_length` / `max_response_length`	2048 / 512
Rollout	engine / TP / `gpu_memory_utilization`	vLLM / 2 / 0.6
Rollout	`val_kwargs.temperature`	0.4
Env	max steps per episode / seed	50 / 0
Train	total steps / `save_freq`	150 / 10

Files

path	contents
`model.safetensors`, `config.json`, tokenizer files	bf16 weights, merged from the FSDP shards with `scripts/model_merger.py`. `lm_head` is omitted because `tie_word_embeddings=true` (same layout as the base Qwen release).
`training_state/model_world_size_2_rank_{0,1}.pt`	FSDP-sharded weights
`training_state/optim_world_size_2_rank_{0,1}.pt`	Adam optimizer state
`training_state/extra_state_world_size_2_rank_{0,1}.pt`	RNG and scheduler state
`training_state/fsdp_config.json`	FSDP layout descriptor

training_state/ lets you resume RL training from this exact optimizer step. The shards are written for world_size=2; resuming on a different number of GPUs requires resharding.

Available checkpoints

step 130 · step 140 · step 150

Only steps 130/140/150 survive: the run used trainer.max_actor_ckpt_to_keep=3, so earlier checkpoints were pruned during training.

Comparison caveat

A StraTA reproduction on the same task and backbone is published alongside these models (chanyoungkim/strata-qwen2.5-1.5b-alfworld-step*). The two are not directly comparable — they use different agent scaffolds:

	GiGPO (this model)	StraTA
eval tasks	128 (`valid_seen` subset)	140 (`valid_seen`, all)
temperature	0.4	0.7
`max_prompt_length`	2048	7168
`max_response_length`	512	1024
prompt format	verl-agent ReAct-style	StraTA strategy-conditioned
seeds	3	2

To compare the methods fairly, re-evaluate both under a single scaffold.

Caveats

Do not use is_correct / pass@1 from verl logs; they are hardcoded to 1.0. Use val/average/metrics/success_rate (or val/success_rate in verl-agent).
Evaluated on valid_seen only; valid_unseen was not run.

Usage

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "chanyoungkim/gigpo-qwen2.5-1.5b-alfworld-step130"
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto")
tok = AutoTokenizer.from_pretrained(repo)