You can deploy this model and start earning money today!

0x-genesys/
mix_wiki_chat_data_300M_tokens_curriculum

Views

No views yet

Curriculum Learning Dataset - 300M Tokens

This dataset is prepared for curriculum learning with separate source files for dynamic mixing during training.

Dataset Information

Total Tokens: 300,000,000 Total Documents: 1,896,043 Tokenizer: tiktoken cl100k_base (GPT-4) Vocabulary Size: 100,277

	Sources




	
	Wikitext

Tokens: 240,000,000 (80.0%) Documents: 1,648,808 File: wikitext_train.bin

Ultrachat

Tokens: 60,000,000 (20.0%) Documents: 247,235… See the full description on the dataset page: https://huggingface.co/datasets/0x-genesys/mix_wiki_chat_data_300M_tokens_curriculum.