This is the dataset used in the paper Multi-Agent Collaborative Data Selection for Efficient LLM Pretraining.
It is a labeled version of the SlimPajama-627B train dataset.
An example in this dataset:
{
"id": "BkiUdvk25V5jCITp144_",
"content": "At the time of Federation most Australian colonies had introduced income taxes, each with its own rules and administered in its own way. This was further complicated with some jurisdictions recording tax according to a taxpayer's residence, and… See the full description on the dataset page:
https://huggingface.co/datasets/beccabai/slimpajama_labeled.
