NB: this version of the model is the improved version of
EIStakovskii/french_toxicity_classifier_plus.
To see the source code of training and the data please follow
the github link.
This model was trained for toxicity labeling.
The model was fine-tuned based off
the CamemBERT language model.
1from transformers import pipeline
2
3classifier = pipeline("text-classification", model = 'EIStakovskii/french_toxicity_classifier_plus_v2')
4
5print(classifier("Foutez le camp d'ici!"))
6
This model was compared against the Google's
Perspective API that similarly detects toxicity.
Two models were tested on two datasets: the size of
200 sentences and
400 sentences.
The first one (arguably harder) was collected from the sentences of the
JigSaw and
DeTox datasets.
The second one (easier) was collected from the combination of sources: both from JigSaw and DeTox as well as
Paradetox translations and sentences extracted from
Reverso Context by keywords.
**I suspect that Perspective has such a low score in the case of the FR dataset (400) because it refuses to trigger on the words "merde" and "putain" and some more rarer words in French like "cul" and so on.
