Views
No views yet
Materials SRPO-TR run trained with batch size 32.global_step_20), selected by validation mean@16.
checkpoints/last/ contains the final checkpoint.| Dataset | Method | Base model | Train batch size | Best val mean@16 | Best checkpoint | Final val mean@16 | Final checkpoint |
|---|---|---|---|---|---|---|---|
| Materials | SRPO-TR | Qwen3-4B | 32 | 76.46% | 20 | 68.75% | 100 |
results/validation_mean16.csvresults/training_scores.csvartifacts/config.yamlartifacts/wandb-summary.json| Hyperparameter | Value |
|---|---|
| Base model | Qwen/Qwen3-4B |
| Method | SRPO-TR |
| Train batch size | 32 |
| Train max samples | 3200 |
| Total training steps | 100 |
| Save frequency | 10 |
| Test frequency | 10 |
| Teacher regularization | trust-region |
| Teacher update rate | 0.1 |
| Rollout samples per prompt | 8 |
| Validation samples per prompt | 16 |
| Learning rate | 5e-6 |
| vLLM GPU memory utilization | 0.8 |
1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3repo_id = "SeongryongJung/Qwen3-4B-Materials-SRPO-TR"
4tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
5model = AutoModelForCausalLM.from_pretrained(
6 repo_id,
7 torch_dtype="auto",
8 device_map="auto",
9 trust_remote_code=True,
10)