Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
CCMatrix-v1-Ja_Zh-filtered – Dataset by larryvrh | AlphaNeural AI
You can deploy this model and start earning money today!
larryvrh
/
CCMatrix-v1-Ja_Zh-filtered
like
0
translation
zh
ja
1M<n<10M
parquet
text
datasets
dask
mlcroissant
polars
us
Views
No views yet
Model card
Files and Versions
Community
API
Dataset Card for "CCMatrix-v1-Ja_Zh-filtered"
Filtered and modified version of Japanese/Chinese language pair data from CCMatrix v1.
Process steps:
Basic regex based filtering / length checking to remove abnormal pairs.
Semantic similarity filtering with a threshold value of 0.6, based on sentence-transformers/LaBSE.
Convert all Traditional Chinese sentences into Simplified Chinese with zhconv.
经过过滤和修改的日语/中文语言对数据，来自CCMatrix v1。
处理步骤：
1.… See the full description on the dataset page:
https://huggingface.co/datasets/larryvrh/CCMatrix-v1-Ja_Zh-filtered
.
