SentencePiece BPE tokenizer (vocab_size=16384) and pre-tokenized
FineWeb-10B shards
canonical/ — used by the submission script. Plain sp16384 BPE
tokenization of FineWeb-10B (50K validation docs + 80 train shards).
caseops/ — not used by the submission. Experiment artifacts from
a CaseOps lossless-case-transform variant.
The submission's setup.sh pulls only the canonical/ subdirectory:
hf download… See the full description on the dataset page:
https://huggingface.co/datasets/deniskurlov/parameter-golf-fineweb-sp16384.
