You can deploy this model and start earning money today!

Pradheep1647/
qwen2.5-0.5b-instruct-openai-gsm8k-ppo

Name: qwen2.5-0.5b-instruct-openai-gsm8k-ppo
Availability: InStock

safetensors

qwen2

Views

No views yet

Pradheep1647/qwen2.5-0.5b-instruct-openai-gsm8k-ppo

small single-gpu rl post-training run on a gsm8k subset.

summary

base model: Qwen/Qwen2.5-0.5B-Instruct
method: ppo
dataset: openai/gsm8k config main
train / eval samples: 400 / 100
prompt format: step-by-step reasoning with final answer after ####

reward

+1.0 if the final numeric answer is correct
+0.1 if the output has a parseable final answer
0.0 otherwise

final metrics

eval_acc: 0.14
eval_reward: 0.2260000000000003
train_reward: 0.0
kl_mean: 0.0
entropy_mean: 7.1146955490112305
ratio_max: 1.0
response_length: 256.0
gpu_hours: 0.0

artifacts

training log: training.log
plots:
training_curves.png

usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "Pradheep1647/qwen2.5-0.5b-instruct-openai-gsm8k-ppo"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)

notes

this is a small controlled experiment, not a benchmark
see training.log for the full tracked metrics over time