Beta
Explore
Marketplace
Neural Labs
Playground
Wallet
Docs
lang-segmentation-roberta – AI Model by nikitast | AlphaNeural AI | AlphaNeural AI
You can deploy this model and start earning money today!
nikitast
/
lang-segmentation-roberta
like
0
transformers
pytorch
xlm-roberta
token-classification
language classification
text segmentation
ru
uk
be
kk
az
hy
ka
he
en
de
multilingual
open_subtitles
tatoeba
oscar
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
RoBERTa for Multilabel Language Segmentation
Training
RoBERTa fine-tuned on small parts of Open Subtitles, Oscar and Tatoeba datasets (~9k samples per language).
Implemented heuristic algorithm for multilingual training data creation with generation of target masks-
https://github.com/n1kstep/lang-classifier
data source
language
open_subtitles
ka, he, en, de
oscar
be, kk, az, hu
tatoeba
ru, uk
Validation
The metrics obtained from validation on the another part of dataset (~1k samples per language).
Validation Loss
Precision
Recall
F1-Score
Accuracy
0.029172
0.919623
0.933586
0.926552
0.991883
