Tulu 3 SFT subset (50K examples) for pre-pretraining eval. Applied identically to all experimental conditions.
messages
List({'content': Value('string'), 'role': Value('string')})
Conversation turns [{role, content}] in Tulu 3 format
{
"script_name": "data/upload_data.py",
"model": "N/A (SFT training data, not model… See the full description on the dataset page:
https://huggingface.co/datasets/reasoning-degeneration-dev/prepretraining-sft-v1.