ontocord/

MixtureVitae-v1-decontaminated

	MixtureVitae (Decontaminated)




	
	Dataset Summary

MixtureVitae is a 422B-token open pretraining dataset introduced in the paperMixtureVitae: Open Web-Scale Pretraining Dataset With High Quality Instruction and Reasoning Data Built from Permissive-First Text Sources. The dataset is designed to answer a key question:

Can we train competitive large language models using only permissive-licensed and low-risk data, without resorting to unrestricted web scrapes?

To this end… See the full description on the dataset page: https://huggingface.co/datasets/ontocord/MixtureVitae-v1-decontaminated.