Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
sozkz-corpus-segmented-kk-v1 – Dataset by stukenov | AlphaNeural AI
You can deploy this model and start earning money today!
stukenov
/
sozkz-corpus-segmented-kk-v1
like
0
stukenov/ekitil-corpus-annotated-kk-v1
kk
mit
10M<n<100M
parquet
optimized-parquet
text
datasets
dask
polars
mlcroissant
2404.01954
us
morpheme-segmentation
kazakh
agglutinative
nlp
corpus
Views
No views yet
Model card
Files and Versions
Community
API
sozkz-corpus-segmented-kk-v1
55.5M Kazakh texts with morpheme boundaries marked by a BiLSTM neural segmenter. Built for training morpheme-aware tokenizers.
Quick Start
from datasets import load_dataset
ds = load_dataset("stukenov/sozkz-corpus-segmented-kk-v1", split="train", streaming=True) sample = next(iter(ds))["text_segmented"]
"Қазақстан\x1fның экономика\x1fсы тұрақты дам\x1fып кел\x1fеді."
The morpheme boundary marker is \x1F (ASCII Unit Separator). To… See the full description on the dataset page:
https://huggingface.co/datasets/stukenov/sozkz-corpus-segmented-kk-v1
.