Granite GRPO Mathematical Reasoning Model

Model Details

• Base Model: IBM Granite 3.1-2b
• Training Approach: Grouped Reward Policy Optimization (GRPO)
• Dataset: GSM8K (Grade School Math 8K)
• Training Progress: 400/1000 steps (40% complete)
• Training Configuration:
  • Learning rate: 3e-6
  • Batch size: 1 per device
  • Gradient accumulation steps: 8
  • Mixed precision: bfloat16
  • DeepSpeed ZeRO-3 optimization

Training Methodology

1. Correctness Reward (1.0 max): Exact match with reference answer
2. Integer Format (0.5 max): Validates numerical answer format
3. Strict Format (0.5 max): Enforces XML-style response structure
4. Soft Format (0.5 max): Ensures basic response organization

Performance Metrics (at step 400)

• Correctness Reward: ~0.65-0.70 (improving)
• Format Rewards:
  • Strict format: ~0.40
  • Soft format: ~0.45
  • Integer format: ~0.45
• Total Reward: ~2.0 (stable)

Input Format

Output Format

Limitations

• Training is incomplete (40% of planned steps)
• Experimental research model
• Performance may vary on complex mathematical problems
• Limited to grade-school level mathematics
• May occasionally produce incorrect reasoning despite correct answers

Intended Use

• Mathematical problem-solving assistance
• Educational support for grade-school math
• Research in mathematical reasoning capabilities of language models

Training Infrastructure

• Framework: DeepSpeed ZeRO-3
• Hardware: 7 GPUs
• Mixed Precision: bfloat16

License

Citation

Acknowledgments

• IBM for the base Granite model
• OpenAI for the GSM8K dataset