LongBlocks is a multilingual synthetic dataset for training long-context language models. It contains approximately 194K long-context question–answer examples generated from long documents spanning books, web text, Wikipedia, arXiv papers, programming/code data, and community Q&A.
The dataset was created to support long-context adaptation for tasks that require reasoning over extended inputs, including:
multi-hop reasoning over long documents,
contextual grounding… See the full description on the dataset page:
https://huggingface.co/datasets/utter-project/LongBlocks.
