A supervised fine-tuned (SFT) version of
nanochat-d20, a 896 million parameter GPT-style language model trained from scratch at the Anuradha and Vikas Sinha Department of Data Science housed in the College of Information at the University of North Texas.
This model demonstrates the transformation from a raw base model to a conversational assistant through instruction tuning — the same technique used to build ChatGPT from GPT-3.
Many thanks to
Andrej Karpathy for the
nanochat repository, which made this entire workshop possible.
The base model knew facts but had no idea how to respond to questions. SFT taught it conversational structure, when to stop, and how to follow instructions.
