SFT (supervised fine-tuned) model — 1B Transformer++ base (kenpeter123/small-1b-pretrain) fine-tuned on quality-filtered mixed instruction shards (513 verified-code + 2500 generated), 8-bit Adam optimizer, stage-isolated checkpoint.

Spec	Value
Params	1,031,898,624 (~1.03B)
Hidden	1536 · 32 layers · 12 attn heads · 4 KV (GQA) · head_dim 128
FFN	4608 SwiGLU
Seq len	2048 (train) / 8192 (config)
Vocab	49152 (SmolLM2-135M tokenizer, BPE)
Optimizer	bitsandbytes Adam8bit, bf16, cosine LR
SFT loss	20.5897 @ step 1000

Raw training checkpoint included as sft_best.pt (model_state_dict + optimizer_state_dict).

kenpeter123/small-1b-sft

Small-1B SFT Model

kenpeter123/
small-1b-sft