Ace-2504/slm-125m-base continued for one epoch on ~515M additional tokens (389M FineWeb-Edu + 131M US case law) that the base model had never seen.

Constant learning rate 1e-4, optimizer state reset, 982 steps on one A100-40GB.

This model exists to answer a research question: does continued pretraining on new data improve a small model overall, or does it cause catastrophic forgetting?

Measured against the base model on its own held-out data, forgetting tracked each domain's share of the new data almost exactly:

Use Ace-2504/slm-125m-base if you want the original. Neither model is instruction-tuned; both are base models.