Views
No views yet
\boxed{N} where N is the option number.\boxed{N} matches ground truth)\boxed{N} exists but wrong answer)\boxed{} found)| Parameter | Value |
|---|---|
| base_model | meta-llama/Llama-3.2-3B-Instruct |
| algorithm | Async GRPO (TRL AsyncGRPOTrainer) |
| thinking | disabled (nothink) |
| learning_rate | 3e-6 |
| lr_scheduler | cosine |
| warmup_steps | 60 |
| max_steps | 2000 |
| global_step_saved | 2000 |
| per_device_train_batch_size | 1 |
| gradient_accumulation_steps | 19 |
| num_train_processes | 7 |
| effective_batch_size | 133 prompts/step |
| num_generations | 9 |
| max_completion_length | 512 |
| temperature | 1.0 |
| epsilon | 0.2 |
| epsilon_high | 0.2 |
| max_staleness | 4 |
| weight_sync_steps | 1 |
| max_grad_norm | 1.0 |
| precision | bf16 |
| parallelism | FSDP2 (7 GPUs training) + vLLM TP=1 enforce-eager (1 GPU inference) |
| final_reward | ~0.89 |
| final_mean_completion_length | ~203 tokens |
1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model = AutoModelForCausalLM.from_pretrained("EnergyAI/llama-3.2-3b-instruct-agrpo-nothink-lr3e-6", torch_dtype="auto")
4tokenizer = AutoTokenizer.from_pretrained("EnergyAI/llama-3.2-3b-instruct-agrpo-nothink-lr3e-6")