anrilombard/

sa-languages

	South African Languages Dataset




	
	Dataset Overview




	

Language Training Documents Training GPT2 Tokens Avg Tokens/Doc Max Tokens Test Documents Test GPT2 Tokens Test Avg Tokens/Doc Test Max Tokens

isiZulu 116,693 192,622,799 1,650.68 335,530 687 1,080,961 1,573.45 15,691

Sesotho 83,329 144,337,938 1,732.15 98,542 841 1,393,086 1,656.4614,071

isiXhosa 99,567 141,484,241 1,421.00 113,710 788 1,161,296 1,473.73 17,220

isiNdebele 21,922 17,533,799 799.83 42,701… See the full description on the dataset page: https://huggingface.co/datasets/anrilombard/sa-languages.