This repository hosts a German wordlist constructed from data from the Wortschatz Leipzig project.
The wordlist is shuffled and contains 9,064,144 words.
First, we obtain several data dumps from the Wortschatz Leipzig project. All URLs are documented here.
After that, we perform several filtering steps. The filtering:
Removes quotes from beginning/end
Removes leading hyphens
Removes duplicates
Filters out words with broken encoding… See the full description on the dataset page:
https://huggingface.co/datasets/stefan-it/nanochat-german-wordlist.
