qwen2.5-1.5b-grpo-gsm8k – AI Model by shiva-sankeerth | AlphaNeural AI

You can deploy this model and start earning money today!

shiva-sankeerth/
qwen2.5-1.5b-grpo-gsm8k

like

0

Qwen/Qwen2.5-1.5B-Instruct

Views

No views yet

Qwen2.5-1.5B GRPO GSM8K

LoRA adapter fine-tuned with GRPO on GSM8K grade-school math. No judge model — pure Python reward functions only.

Results

Model	GSM8K Test Accuracy
Base (Qwen2.5-1.5B-Instruct)	0.0%*
+ GRPO 100 steps	29.2%

*Base scores 0% since it never uses the required <answer> tags without training.

Training details

Base model: Qwen/Qwen2.5-1.5B-Instruct (4-bit)
LoRA rank: 16 (1.18% of weights trained)
Dataset: GSM8K train split (7,473 examples)
Steps: 100, batch 8, 8 generations per question
Rewards: correct answer = +2.0, format tags present = +0.5

Usage

python

from unsloth import FastLanguageModel
from peft import PeftModel

model, tokenizer = FastLanguageModel.from_pretrained(
    "Qwen/Qwen2.5-1.5B-Instruct", load_in_4bit=True
)
model = PeftModel.from_pretrained(model, "shiva-sankeerth/qwen2.5-1.5b-grpo-gsm8k")

Output format

<reasoning>
step-by-step working...
</reasoning>
<answer>
42
</answer>

Code: ShivaSankeerth/llm-reasoning