Views
No views yet
tooluse GRPO batch-size-32 run. The repository name uses the project GRPO-TR naming convention, but the actual training method for this checkpoint is GRPO.mean@16. checkpoints/last/ contains the final checkpoint.| Dataset | Method | Base model | Train batch size | Best val mean@16 | Best checkpoint | Final val mean@16 | Final checkpoint |
|---|---|---|---|---|---|---|---|
| Tool-use / tooluse | GRPO | Qwen3-4B | 32 | 60.29% | 40 | 56.89% | 100 |
| step | val_mean16 | percent |
|---|---|---|
| 10 | 0.596507352941 | 59.65% |
| 20 | 0.579044117647 | 57.90% |
| 30 | 0.590992647059 | 59.10% |
| 40 | 0.602941176471 | 60.29% |
| 50 | 0.594669117647 | 59.47% |
| 60 | 0.569852941176 | 56.99% |
| 70 | 0.549632352941 | 54.96% |
| 80 | 0.559742647059 | 55.97% |
| 90 | 0.564338235294 | 56.43% |
| 100 | 0.568933823529 | 56.89% |
| Section | Parameter | Value | Source |
|---|---|---|---|
| Run identity | Base model | Qwen/Qwen3-4B | queue/script override |
| Run identity | Dataset | Tool-use / tooluse | run_qwen3_generalization.sh |
| Run identity | Method | GRPO | run_qwen3_generalization.sh |
| Run identity | Config | baseline_grpo | run_qwen3_generalization.sh |
| Run identity | Experiment | qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 | run_qwen3_generalization.sh |
| Run identity | W&B run | run-20260702_145459-dx38nnn7 | wandb |
| Data | Train file | datasets/tooluse/train.parquet | script override |
| Data | Validation file | datasets/tooluse/test.parquet | script override |
| Data | Train batch size | 32 | queue/script override |
| Data | Train max samples | 3200 | queue/script override |
| Schedule | Total training steps | 100 | queue/script override |
| Schedule | Validation before train | False | queue/script override |
| Schedule | Save frequency | 10 | queue/script override |
| Schedule | Validation frequency | 10 | queue/script override |
| Sequence | Max prompt length | 2048 | queue/script override |
| Sequence | Max response length | 8192 | queue/script override |
| Sequence | Max model length | 10240 | queue/script override |
| Rollout | Train rollout n | 8 | queue/script override |
| Rollout | Validation rollout n | 16 | queue/script override |
| Rollout | vLLM GPU memory utilization | 0.8 | queue/script override |
| Optimization | Learning rate | 1e-6 | GRPO method override |
| Optimization | Weight decay | 0.01 | script override |
| PPO/GRPO | PPO mini batch size | 8 | queue/script override |
| PPO/GRPO | Normalize GRPO advantages by std | False | baseline_grpo.yaml / script override |
| Rollout correction | Importance sampling mode | token | script override |
| Rollout correction | IS threshold | 2.0 | script override |
| Checkpoint/Logging | Checkpoint root | checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 | script override |
| Checkpoint/Logging | Latest checkpointed iteration | 100 | latest_checkpointed_iteration.txt |
| Checkpoint/Logging | External actor archive | checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/_actor_archive | preserve_actor_checkpoints.py |
| Checkpoint/Logging | Logger | console, wandb | ppo_trainer.yaml |
| PPO/GRPO | Policy loss mode | vanilla | method override |
| PPO/GRPO | Actor KL loss coef | 0.0 | method override |
results/validation_mean16.csvresults/training_scores.csvresults/hyperparameters.csvresults/training_score.pngresults/training_score.svgartifacts/config.yamlartifacts/wandb-summary.jsonartifacts/wandb-metadata.jsonartifacts/output.logartifacts/queue.log1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3repo_id = "SeongryongJung/Qwen3-4B-Tooluse-GRPO-TR"
4tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
5model = AutoModelForCausalLM.from_pretrained(
6 repo_id,
7 torch_dtype="auto",
8 device_map="auto",
9 trust_remote_code=True,
10)checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8checkpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/_actor_archive/global_step_40/actorcheckpoints/datasets/tooluse/qwen3gen-tooluse-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_100/actorrun-20260702_145459-dx38nnn7artifacts/queue.log