qwen3-4b-grpo-alfworld-v3 – AI Model by Kaito-F | AlphaNeural AI

You can deploy this model and start earning money today!

Kaito-F/
qwen3-4b-grpo-alfworld-v3

like

0

reinforcement-learning

text-generation

Kaito-F/qwen3-4b-instruct-lora-v2

Views

No views yet

qwen3-4b-gigpo-alfworld-v3

LoRA adapter trained with GiGPO v5 (Group-in-Group Policy Optimization) from Kaito-F/qwen3-4b-instruct-lora-v2.

Training Method

GiGPO v5: Task-Focused + Dense Milestone Rewards

Task-focused rollout: examine + pick_two only (weakest tasks)
Dense milestone rewards: RLVMR-inspired verifiable process rewards
Step-level advantage: Anchor state grouping + milestone enrichment
Per-task exploration budget: examine=15, pick_two=30 steps

v5 Improvements over v4

Task-focused: only examine/pick_two → no regression on strong tasks
Dense milestone rewards (take/put/use_desklamp → positive signal)
Penalty for "Nothing happens" and wasteful examine/close
Partial credit for failed trajectories (ADMIRE-inspired)
K=6 trajectories per task (v4: 4)

References

Wang & Ammanabrolu (2025): Practitioner's Guide to Multi-turn Agentic RL
RLVMR (Zhang et al., 2025): Verifiable meta-reasoning rewards
ADMIRE (Zheng et al., 2025): Adaptive milestone reward

Config

Parameter	Value
Base model	`Kaito-F/qwen3-4b-instruct-lora-v2`
LoRA r / alpha	8 / 16
Learning rate	5e-07
Epochs	1
GiGPO lambda	0.3
Advantage clip	+/-2.0
Rollout K	6
Training samples	1182 (examine=300, puttwo=882)
Rollout success rate	30.4%

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("Kaito-F/qwen3-4b-grpo-alfworld-v3")
tokenizer = AutoTokenizer.from_pretrained("Kaito-F/qwen3-4b-grpo-alfworld-v3")