This model is a fine-tuned version of Qwen/Qwen2.5-0.5B, first supervised fine-tuned (SFT) on a mixture of math reasoning datasets, then further optimized with reinforcement learning (RL).

Evaluation Results

The table below compares the base model, the SFT checkpoint, and the final RL checkpoint (2600 steps). All numbers are proportions (0–1 scale).

Math Benchmarks

Benchmark	Base	SFT	SFT+RL (ckpt-2600)	Δ over Base
GSM8K	0.2161	0.3616	0.3624	+0.1463
MATH-500	0.1740	0.1500	0.1640	-0.0100
TheoremQA	0.1000	0.1088	0.1138	+0.0138

General QA Benchmarks

Benchmark	Base	SFT	SFT+RL (ckpt-2600)	Δ over Base
ARC-Easy	0.4533	0.4933	0.4882	+0.0349
ARC-Challenge	0.3131	0.3592	0.3592	+0.0461
MMLU	0.2514	0.4185	0.4259	+0.1745
TruthfulQA	0.2950	0.2681	0.2644	-0.0306

Evaluation details: GSM8K and MATH-500 were evaluated with flexible-extract matching; TheoremQA used the built-in scoring; ARC and MMLU used accuracy; TruthfulQA used truthful answer selection. The RL checkpoint corresponds to step 2600 of the reinforcement learning stage.

Version History

0.3 (SFT + RL)

Applied reinforcement learning on top of the SFT math model to improve reasoning consistency and final answer accuracy.
RL training details (algorithm, reward design, datasets) can be provided on request – broadly, it targeted better alignment of chain-of-thought with correct final answers.

0.2 (SFT)

Mixed small subset of multiple math-domain datasets, total 25k examples.

Dataset	Target Size	Role
`openai/gsm8k` (train)	~7.5K	Foundation arithmetic and word-problem reasoning
`AI-MO/NuminaMath-CoT`	~8K	Competition-math coverage for MATH-500-style problems
`TIGER-Lab/MathInstruct` (CoT-only)	~5K	Diverse math reasoning and theorem-style supervision
`hendrycks/competition_math` (train, L3-L5)	~3K	Higher-difficulty competition math
`TIGER-Lab/TheoremQA`-aligned slice	~1.5K	Basic theorem-application exposure

Training Procedure

SFT Hyperparameters

learning_rate: 1e-05
train_batch_size: 2
eval_batch_size: 8
seed: 42
distributed_type: multi-GPU
per_device_train_batch_size: 2
gradient_accumulation_steps: 8
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 100
num_epochs: 2.0

RL Stage

The SFT checkpoint was further trained with reinforcement learning (e.g., online/off-policy preference optimization) to improve solution correctness.
The best performing checkpoint at step 2600 is reported in the tables above.
Detailed RL hyperparameters and reward design available on request.

Framework Versions

Transformers 5.2.0
PyTorch 2.11.0+cu128
Datasets 4.0.0
Tokenizers 0.22.2

Sepolian/qwen2.5-0.5B-math

Model Card for Qwen2.5-0.5B-SFT-RL-math