This model is fine-tuned from Qwen/Qwen3-4B-Instruct-2507 using the ThinkTwice framework.

Paper: ThinkTwice: Jointly Optimizing Large Language Models for Reasoning and Self-Refinement (arXiv: 2604.01591)

Code: https://github.com/CSSLab/ThinkTwice

ThinkTwice is a simple two-phase GRPO-based framework that jointly trains LLMs to (1) solve reasoning problems and (2) refine their own solutions. In each pair of training steps, the model is first optimized on solving a reasoning problem, then optimized on refining its own solution to the same problem — using the same binary correctness reward in both phases, with no correctness signals or critique annotations required.

ThinkTwice reveals an implicit rectify-then-fortify curriculum: early in training, refinement predominantly corrects errors; as the model improves, it naturally shifts toward preserving already-correct solutions, yielding a more rectified reward signal.

On AIME, ThinkTwice-Qwen3-4B-Instruct outperforms GRPO-trained Qwen3-4B:

Results span five mathematical reasoning benchmarks across two model families (Qwen3-4B and OLMo3-7B).

This model supports both direct solving and self-refinement. Use it in two passes:

See the GitHub repository for full usage instructions and evaluation scripts.