Quantization made by Richard Erkhov.
This is
karpathy's model from
the llm.c project converted to HF format to investigate
bfloat16 performance.
The training run was 150B tokens, 1.5 epochs over the 100B FineWeb sample dataset.
There's active work underway at
https://github.com/karpathy/llm.c so I'd suggest following the developments there!
