MasakhaNEWS is the largest publicly available dataset for news topic classification in 16 languages widely spoken in Africa. The train/validation/test sets are available for all the 16 languages.
You can evaluate an embedding model on this dataset using the following… See the full description on the dataset page:
https://huggingface.co/datasets/mteb/masakhanews.