Deduplicated Kazakh web text corpus collected from 6 public HuggingFace datasets. Contains only texts not present in kz-transformers/multidomain-kazakh-dataset (12.4M texts were used as the dedup reference).

	Stats

Field Value

Total unique texts 9,475,089

Format Parquet (142 shards)

Columns text, source

Dedup method MD5 hash (exact match)

Dedup reference kz-transformers/multidomain-kazakh-dataset (12.4M hashes)

Date… See the full description on the dataset page: https://huggingface.co/datasets/stukenov/sozkz-corpus-dedup-kk-web-v1.