A highly customized version of google/gemma-2-2b-it, extended through Reinforcement Learning (GRPO) to excel at structured mathematical reasoning and logic tasks. The model's reasoning capability was further enhanced via knowledge distillation from google/gemma-2-27b-it.

Model Lineage: This GRPO model is fine-tuned on top of our earlier SFT foundation: Phonsiri/gemma-2-2b-SFT-Reasoning-full-Model

Model Highlights

Unlike standard instruction-tuned models that jump straight to an answer, this model is trained to show its work. By explicitly laying out each reasoning step before answering, it performs significantly better on math, logic, and analytical problem-solving.

Output Format:

Section	Tag	Description
Chain-of-Thought	`<reasoning> ... </reasoning>`	Step-by-step internal reasoning
Final Answer	`<answer> ... </answer>` or `\boxed{}`	Concise, boxed final answer

Training Pipeline

This model is the result of a two-stage training framework:

Stage 1 Supervised Fine-Tuning (SFT)

The base model (Phonsiri/gemma-2-2b-SFT-Reasoning-full-Model) was fine-tuned on open-r1/OpenR1-Math-220k, formatted explicitly with <reasoning> and <answer> tags. This stage taught the model the structure of reasoning syntax without yet enforcing correctness.

Stage 2 GRPO (Generative Reward Policy Optimization)

Building on the SFT base, the model underwent RL training with a custom reward structure:

Reward	Condition
+2.0	Final boxed answer mathematically matches ground truth
+1.0	Strict adherence to `<reasoning>` XML formatting structure

Quickstart

Important: Use skip_special_tokens=False in the streamer to see the <reasoning> tags in output.

python

from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
import torch

model_id = "Phonsiri/gemma-2-2b-GRPO-Reasoning-full"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
 model_id,
 device_map="auto",
 torch_dtype=torch.bfloat16
)

system_prompt = (
 "You are a helpful assistant. Please reason step by step, "
 "and put your thoughts within <reasoning> and </reasoning> tags, "
 "and your final answer within <answer> and </answer> tags or \\boxed{}."
)
prompt = "Solve for x: 3x + 5 = 20"

messages = [{"role": "user", "content": f"{system_prompt}\n\n{prompt}"}]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=False)

with torch.no_grad():
 model.generate(
 **inputs,
 streamer=streamer,
 max_new_tokens=4096,
 temperature=0.6,
 top_p=0.9,
 repetition_penalty=1.1,
 )

Example Output

<reasoning>
We need to isolate x on one side of the equation.

Step 1: Subtract 5 from both sides.
 3x + 5 - 5 = 20 - 5
 3x = 15

Step 2: Divide both sides by 3.
 x = 15 / 3
 x = 5
</reasoning>

<answer> x = 5 </answer>

Model Details

Property	Value
Base Model	`google/gemma-2-2b-it`
Teacher Model (Distillation)	`google/gemma-2-27b-it`
SFT Base	`Phonsiri/gemma-2-2b-SFT-Reasoning-full-Model`
Training Method	GRPO (Reinforcement Learning)
Training Dataset	`open-r1/OpenR1-Math-220k`
Precision	bfloat16
Max New Tokens	4096
Primary Task	Mathematical Reasoning
Language	English

Acknowledgements

Authors:

Phonsiri Thabunsri @Phonsiriwillbejommarn
CYP777 @CYP777

Project Advisor:

Supaporn Bunrit, Ph.D. Suranaree University of Technology

Institutions & Credits:

Suranaree University of Technology (SUT) Research support and computing resources
Google DeepMind Open-weights Gemma 2 model
DeepSeek AI GRPO methodology

If you find this model useful, please consider leaving a and citing the work!

Phonsiri/gemma-2-2b-Distillation-gemma-2-27b-it

Gemma-2-2B Reasoning Edition (GRPO)