Qwen3-1.7B-RL-final is a reinforcement learning (RL) post-trained version of Qwen/Qwen3-1.7B. It is optimized for multilingual reasoning, mathematics, and algorithmic problem-solving.

This model was trained using the veRL framework with GRPO (Group Relative Policy Optimization) on procedural datasets generated by the Reasoning Gym (r-gym) framework.

Training Methodology

Base Model: Qwen/Qwen3-1.7B
RL Framework: veRL
RL Algorithm: GRPO (Group Relative Policy Optimization)
Reward Function: Algorithmic verification via Reasoning Gym (rg_reward.py:compute_score)
Dataset: Procedural dataset generated via reasoning-gym containing multiple algorithmic and logic tasks across multiple languages, available at MauroPello/reasoning-gym-verl-datasets. The training set was dynamically interleaved across tasks and languages to maintain a balanced data distribution.
Hyperparameters:
- Learning Rate: 1e-6
- Batch Size: 256
- Max Prompt Length: 4096
- Max Response Length: 8192
- PPO Epochs: 2
- KL Coefficient: 0.001

Evaluations & Performance

We evaluated the model against the base Qwen/Qwen3-1.7B across several benchmarks, showing substantial improvements in reasoning accuracy, particularly on the Reasoning Gym validation split and held-out validation tasks/languages.

Benchmark / Dataset	Base model Accuracy	RL model Accuracy	Absolute Improvement
Reasoning Gym Validation Split (`val.parquet`)	35.29%	50.51%	+15.22%
Held Out Languages (Dutch `nl`, Turkish `tr`)	30.20%	36.60%	+6.40%
Held Out Tasks	37.88%	40.29%	+2.41%
MMLU Pro Lite (Multilingual subset)	38.48%	38.97%	+0.49%
Polymath (Multilingual Mathematics)	33.24%	33.44%	+0.20%

Key Takeaways

Massive Validation Gains: Accuracy on the Reasoning Gym validation set jumped by 15.22% (from 35.29% to 50.51%).
Strong Language Generalization: On held-out languages (Dutch and Turkish) not specifically targeted during translation tuning, generalization accuracy improved by 6.40%.
Robust Task Generalization: Accuracy on held-out tasks increased by 2.41%, proving that GRPO training generalizes to unseen rule-based verification tasks.

MauroPello/Qwen3-1.7B-RL-final

Qwen3-1.7B-RL-final

Training Methodology

Evaluations & Performance

Key Takeaways

MauroPello/
Qwen3-1.7B-RL-final