Views
No views yet
Llama-3.2-3B-Instruct fine-tuned on 43.5K explicit reasoning traces
from OpenThoughts-114k
(math subset, thinking format). No continual pre-training.| Init | meta-llama/Llama-3.2-3B-Instruct |
| Data | OpenThoughts-114k math subset (43.5K examples) |
| Epochs | 3 |
| Sequence length | 8,192 |
| Effective batch size | 256 sequences |
| Learning rate | 1.5e-5, cosine decay, 10% warmup |
| Optimizer | AdamW, weight decay 0.01 |
| Precision | BF16 + Flash Attention 2 |
1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model = AutoModelForCausalLM.from_pretrained("pavelslab-nyu/Llama-3.2-3B-ThinkSFT")
4tokenizer = AutoTokenizer.from_pretrained("pavelslab-nyu/Llama-3.2-3B-ThinkSFT")1@article{rahman2026when,
2 title = {When Can LLMs Learn to Reason with Weak Supervision?},
3 author = {Rahman, Salman and Shen, Jingyan and Mordvina, Anna and
4 Palangi, Hamid and Gabriel, Saadia and Izmailov, Pavel},
5 journal = {Preprint},
6 year = {2026}
7}