This dataset is a comprehensive pull of all Japanese wikipedia article data as of 20220808.
Note: Right now its uploaded as a single cleaned gzip file (for faster usage), I'll update this in the future to include a huggingface datasets compatible class and better support for japanese than the existing wikipedia repo.
import pandas as pd
from datasets import load_dataset
df =… See the full description on the dataset page:
https://huggingface.co/datasets/inarikami/wikipedia-japanese.
