sfanm/

d24-sft-mixture

The supervised-fine-tuning conversation mixture used to train the d24 (0.757B, nanochat-style depth-24) chat models. It is a faithful port of nanochat's speedrun SFT mix (SmolTalk + MMLU aux + GSM8K + spelling + identity), with all content stored as plain-string ChatML conversations.

848,656 unique conversations (this dataset). Training blend ≈ 1,071,759 after nanochat's epoch oversampling (see the epochs column — applied at train time, not… See the full description on the dataset page: https://huggingface.co/datasets/sfanm/d24-sft-mixture.