OLMo-2-1B checkpoint midtrained with SAM optimizer with rho = 5e-2.
Paper:"Sharpness-Aware Pretraining Mitigates Catastrophic Forgetting"
Ishaan Watts*, Catherine Li*, Sachin Goyal, Jacob Mitchell Springer, Aditi Raghunathan
ICML2026
https://arxiv.org/abs/2605.02105
