marcoshernanz/

llm-lab-fineweb-edu-sample10bt-bpe-16384

Public tokenizer + token shard artifacts for llm-lab phase 2 milestone 21. Contents:

metadata.json fineweb_edu_sample10bt_bpe_16384.json train_.npy validation_.npy

These files are intended to be downloaded into a local directory and then passed to the milestone-021 experiment script.