Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
Malaysian-Emilia-Dedup – Dataset by Scicom-intl | AlphaNeural AI
You can deploy this model and start earning money today!
Scicom-intl
/
Malaysian-Emilia-Dedup
like
0
ms
ta
zh
1M<n<10M
text
us
Views
No views yet
Model card
Files and Versions
Community
API
Malaysian-Emilia-Dedup
Global dedup of,
https://huggingface.co/datasets/mesolitica/Malaysian-Emilia-v2
https://huggingface.co/datasets/Scicom-intl/Malaysian-Chinese-Emilia
https://huggingface.co/datasets/Scicom-intl/Malaysian-Tamil-Emilia
Using FAISS for 80%, 85% and 90% similarities.