Quantization made by Richard Erkhov.
llmc-gpt2-774M-150B - bnb 8bits
Original model description:
license: mit
datasets:
- HuggingFaceFW/fineweb
language:
- en
llm.c checkpoint: GPT-2 774M
This is a HF/safetensors conversion of the llm.c checkpoint of a 774M parameter run on 150B tokens from FineWeb.
Training was conducted on a single 8xA100 80GB SXM node for ~6 days.
See
discussion on GitHub for more information.
