Fine-tuned from Qwen/Qwen2.5-3B-Instruct using GRPO with an evolving skill bank on the Interphyre physics-puzzle environment, level down_to_earth.

The skill bank is updated during training and injected into the model's context, combining gradient-based policy optimization with in-context skill accumulation.

Training Configuration

Field	Value
Base model	Qwen/Qwen2.5-3B-Instruct
Fine-tuning method	GRPO + evolving skill bank
Level	`down_to_earth`
Training seeds	1–50
Val seeds	51–100
Checkpoint	global_step_50 (epoch 1)
Batch size	1 seed / step
Rollouts per step	n=4
Learning rate	1 × 10⁻⁶
Max turns per episode	25
Hardware	1 × A100 80 GB
Reward	Binary {0, 1} — environment success predicate
Observation masking	Yes (`mask_observations=True`)

Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("vgandhi13/Qwen2.5-3B-Interphyre-Evolving-DownToEarth")
model = AutoModelForCausalLM.from_pretrained("vgandhi13/Qwen2.5-3B-Interphyre-Evolving-DownToEarth", device_map="auto")

This model operates inside the Interphyre ReAct tool loop. At inference time, the evolved skill bank should be prepended to the system prompt.