LoRA adapters trained with
GRPO on top of
meta-llama/Llama-3.2-1B using the
Tinker cloud training service.
Part of the TinkerRL-Bench release for our NeurIPS submission
"A Unified Benchmark for RL Post-Training of Language Models"
(
repo).
1from peft import PeftModel
2from transformers import AutoModelForCausalLM, AutoTokenizer
3
4base = "meta-llama/Llama-3.2-1B"
5adapter = "arvindcr4/tinker-rl-arithmetic_trajectory-llama-3.2-1b"
6
7tok = AutoTokenizer.from_pretrained(base)
8model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="auto", device_map="auto")
9model = PeftModel.from_pretrained(model, adapter, subfolder="final") # or "<step>"
1@misc{tinkerrlbench2026,
2 title = {A Unified Benchmark for RL Post-Training of Language Models},
3 author = {Arvind, C. R. and Jeyaraj, Sandhya},
4 year = {2026},
5 note = {NeurIPS submission, https://github.com/pes-llm-research/tinker-rl-lab}
6}
Apache 2.0. The underlying base model retains its original license —
please check meta-llama/Llama-3.2-1B for any usage restrictions.
