Official code and data for Hint Tuning, a lightweight SFT data construction method that constructs long and short chain-of-thought traces by using the corresponding instruct model as an ideal difficulty probe: the minimal reasoning hint required for the instruct model to solve a problem directly reflects how hard that problem is, and determines the length of CoT assigned to it.
Each record in hint_tuning_1k.json follows the Alpaca format:
json
1{2"instruction":"Let $f(x) = x^2 + ...$",3"input":"",4"output":"<think>\nI may need some deep thinking.\n...\n</think>\n\nThe answer is $\\boxed{42}$."5}
The <think> prefix encodes the reasoning state assigned during data construction (see below).
Data Construction
The 1,000 problems are drawn from s1K.
The corresponding instruct model serves as an ideal difficulty probe: the minimal hint prefix from the think model's trace that allows the instruct model to reach the correct answer measures problem difficulty, and directly determines the length of CoT assigned to each problem.
Step 1 — Both models attempt all problems independently.
Step 2 — For problems the instruct model cannot solve alone,
inject cumulative prefixes from the think model's trace
and ask the instruct model to complete from there.
Grading (LLM-as-judge) determines the minimal prefix k
that leads to a correct answer.
Step 3 — Classify each problem:
instruct correct (k=0) → State 1 – No-Hint
<think>Let me think. ...</think>
instruct correct (k>0) → State 2 – Sparse-Hint
<think>I may need some deep thinking. [prefix]...</think>
no prefix worked → State 3 – Full-Hint (fall back to full think trace)
<think>This is a complex or challenging question... [full trace]</think>
Our experiments use Relax, an open-source post-training framework supporting both SFT and RL.
The dataset (hint_tuning_1k.json) is in Alpaca format (instruction / input / output fields).
The script automatically loads evaluation/custom_tasks.py via --custom-tasks, which defines the prompt format used at training time:
{problem}
Please reason step by step, and put your final answer within \boxed{}.
Use this script — not lighteval's built-in task names — to reproduce our numbers. Lighteval's default prompts differ from the above and will produce inconsistent results.
The script also exports EVAL_MODEL_PATH so custom_tasks.py can load the correct tokenizer for measuring output token length.
Note on instruction robustness: The 1K dataset uses a fixed prompt style (math-oriented, \boxed{} format). If you want the model to generalize to a wider variety of instruction phrasings, synthesize additional prompt variants on top of the 1K samples before training — e.g. replacing the instruction with paraphrases like "Solve:", "Think step by step.", "Q: … A:", etc.
Citation
If you find this work useful, please cite:
bibtex
1@article{fan2026hint,
2 title={Hint Tuning: Less Data Makes Better Reasoners},
3 author={Fan, Siqi and Li, Minghao and Ma, Xiaoqian and Huang, Xiusheng and Chen, Zhuo and Qin, Bowen and Zhang, Liujie and Shang, Shuo and Chen, Weihang},
4 journal={arXiv preprint arXiv:2605.08665},
5 year={2026}
6}
We are grateful to the authors of s1 for curating and open-sourcing the s1K problem set that forms the foundation of our dataset, and to the Relax team for building and maintaining the post-training framework used in our experiments.
bibtex
1@inproceedings{muennighoff2025s1,
2 title={s1: Simple test-time scaling},
3 author={Muennighoff, Niklas and Yang, Zitong and Shi, Weijia and Li, Xiang Lisa and Fei-Fei, Li and Hajishirzi, Hannaneh and Zettlemoyer, Luke and Liang, Percy and Cand{\`e}s, Emmanuel and Hashimoto, Tatsunori B},
4 booktitle={Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
5 pages={20286--20332},
6 year={2025}
7}
bibtex
1@software{relax2026,
2 title = {Relax: An Asynchronous Reinforcement Learning Engine for Omni-Modal Post-Training at Scale},
3 author = {Relax Contributors},
4 url = {https://arxiv.org/abs/2604.11554},
5 year = {2026}
6}
