Ba2han/

pt-1501-tokenized-2

Base Tokenizer: unsloth/Llama-3.2-1B Max Length: 4000 Last Update: 2026-01-15 17:45:17

	Statistics




	

Metric Count

Total Input Rows 3,414,026

Deduplicated (Dropped) 16,052

Final Rows Kept 3,188,409

Total Tokens 2,432,321,066

	Dataset Breakdown




	

Dataset Source Tokens Contributed

OpenDataArena/ODA-Mixture-100k 27,505,162

codelion/fineweb-edu-1B 738,481,203… See the full description on the dataset page: https://huggingface.co/datasets/Ba2han/pt-1501-tokenized-2.