A ~200M-parameter transformer
pretrained from random initialization on mathematical text,
using Karpathy's
nanochat. The deep-learning artifact of
the
MathNano project — trained on a single RTX 4090.
Generates coherent mathematical/English text with real factual recall (e.g. "the chemical symbol
of gold is Au", "if yesterday was Friday, then tomorrow will be Saturday"). It does not
reliably solve problems or follow instructions — that requires SFT/RL (see the project's Track
B). This is a base model for study and further fine-tuning, not an instruct model.
Loading requires the nanochat codebase. See the project repo for details:
https://github.com/adimunot21/mathnano
