You can deploy this model and start earning money today!

Xerneas3318/
gpt2-124m-edu-fineweb10b

Name: gpt2-124m-edu-fineweb10b
Availability: InStock

Views

No views yet

GPT-2 124M — edu-FineWeb 10B

A 124M-parameter GPT-2 trained from scratch on 10B tokens of the edu-FineWeb sample.

Architecture: GPT-2 (12 layers, 12 heads, 768 dim, 1024 context)
Vocab: 50304 (GPT-2 BPE, padded)
Training: 19,072 steps, ~0.5M tokens/batch (10B tokens total)
Final val loss: 3.10

Files

ckpt_019072.pt — checkpoint (model, optimizer, scheduler, step, vocab_size)
inference.py — self-contained model definition + generation script

Usage

bash

huggingface-cli download Xerneas3318/gpt2-124m-edu-fineweb10b ckpt_019072.pt inference.py --local-dir .

python3 inference.py \
    --ckpt ckpt_019072.pt \
    --prompt "Once upon a time" \
    --num-samples 4 --max-new-tokens 100 --temperature 0.9 --top-k 50

The script primes on the real prompt tokens and grows the context autoregressively, with temperature + top-k sampling. Runs on GPU (CUDA) by default, CPU with --device cpu.