qwen3-4b-grpo-alfworld-v2 – AI Model by Kaito-F | AlphaNeural AI

You can deploy this model and start earning money today!

Kaito-F/
qwen3-4b-grpo-alfworld-v2

like

0

reinforcement-learning

text-generation

Kaito-F/qwen3-4b-instruct-lora-v2

Views

No views yet

qwen3-4b-gigpo-alfworld-v2

LoRA adapter trained with GiGPO v4 (Group-in-Group Policy Optimization) from Kaito-F/qwen3-4b-instruct-lora-v2.

Training Method

GiGPO (verl-agent, ICML 2025) with v4 enhancements:

Episode-level advantage: Compare K=4 trajectories per task (GRPO)
Step-level advantage: Anchor state grouping + action-quality bias
Action-quality aware: Explicit penalty for wasteful examine/close actions
Conservative training: 1 epoch, lower LR, advantage clipping

v4 Improvements over v3

Enhanced loop detection (action-level, not output-level)
Wasteful action penalty in reward function
Action-type bias in step advantages
Advantage clipping +/-2.0
Reduced GiGPO lambda: 0.5 -> 0.3
Conservative hyperparameters: lr=5e-07, LoRA r=8, epochs=1

Config

Parameter	Value
Base model	`Kaito-F/qwen3-4b-instruct-lora-v2`
LoRA r / alpha	8 / 16
Learning rate	5e-07
Epochs	1
GiGPO lambda	0.3
Advantage clip	+/-2.0
Rollout K	4
Max tasks	100
Training samples	389
Rollout success rate	44.2%

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("Kaito-F/qwen3-4b-grpo-alfworld-v2")
tokenizer = AutoTokenizer.from_pretrained("Kaito-F/qwen3-4b-grpo-alfworld-v2")