Parallel-packed pretraining data built from FineWeb sample-10BT.
Core idea: blocks within each sample are semantically related but not duplicates; block order is shuffled to break privileged sequential ordering.
Each line in train.jsonl is a JSON object:
{
"text": "
block 1 textblock 2 textblock 3 text",
"blocks": ["block 1 text", "block 2 text", "block 3 text"],
"metadata": {… See the full description on the dataset page:
https://huggingface.co/datasets/pengxiang/nap-parallel-packing-demo.
