Qwen3-4B-sft-en10k
SFT of Qwen3-4B-Base on 10k high-quality English math chain-of-thought traces
(correctness-filtered via math_verify, decontaminated vs AIME2026/AMC23, length-stratified),
intended as an RL cold-start model.
- Base: Qwen/Qwen3-4B-Base
- Data: 10k English long-CoT (R1-style
<think>...</think> + boxed answer)
- Recipe: 2 epochs, lr 2e-5 cosine, max_len 32768, FSDP bf16, keep-think chat template
- eos: fixed to
<|im_end|> (151645) for proper generation stopping
- AIME2026 (think on, 12k tokens): pass@1 20.6%, pass@16 43.3% (converged-only acc ~52%)
