You can deploy this model and start earning money today!

yrrhall/
bert-mini-toxicity

Name: bert-mini-toxicity
Availability: InStock

Views

No views yet

bert-mini-toxicity

This is a toxicity classifier fine-tuned using the gravitee-io/textdetox-multilingual-toxicity-dataset. The model supports a wide range of languages and is trained for toxicity classification ("not-toxic", "toxic").

We perform an 85/15 train-test split per language based on the textdetox dataset. All credits go to the authors of the original corpora.

Performance Overview

While the model performance differs from gravitee-io/distilbert-multilingual-toxicity-classifier Some languages still make the cut, even with the base model being lightweight and trained on English as per the model card.

Original model

Language	eval F1	train F1	Δ F1
en	0.955823	0.992248	-0.036425
fr	0.898551	0.975587	-0.077036
de	0.873016	0.958611	-0.085595
hi	0.869221	0.923933	-0.054713
ru	0.836565	0.842410	-0.005845
es	0.807245	0.918570	-0.111325
it	0.804124	0.903806	-0.099683
uk	0.815013	0.818527	-0.003514
tt	0.799443	0.840136	-0.040693
ja	0.761658	0.774920	-0.013262
hin	0.740053	0.845366	-0.105313
ar	0.675386	0.710673	-0.035287
am	0.639659	0.688475	-0.048816
zh	0.634211	0.668962	-0.034752
he	0.444444	0.554080	-0.109635

Quantized model (ONNX)

Language	eval F1	train F1	Δ F1
en	0.955705	0.992241	-0.036536
fr	0.899329	0.972551	-0.073222
de	0.875000	0.954798	-0.079798
hi	0.866310	0.922319	-0.056009
ru	0.831933	0.837851	-0.005918
uk	0.801642	0.819167	-0.017525
it	0.801061	0.900024	-0.098963
tt	0.793696	0.838308	-0.044612
es	0.783673	0.908962	-0.125288
ja	0.759358	0.775797	-0.016439
hin	0.723810	0.847162	-0.123352
ar	0.678788	0.706675	-0.027887
am	0.641026	0.688030	-0.047005
zh	0.632844	0.658548	-0.025704
he	0.409357	0.527363	-0.118006

🤗 Usage

python

from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForSequenceClassification
import numpy as np
# Load model and tokenizer using optimum
model = ORTModelForSequenceClassification.from_pretrained(
 "gravitee-io/bert-mini-toxicity",
 file_name="model.quant.onnx"
)
tokenizer = AutoTokenizer.from_pretrained("gravitee-io/bert-mini-toxicity")
# Tokenize input
text = "Your text here"
inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True)
# Run inference
outputs = model(**inputs)
logits = outputs.logits
# Optional: convert to probabilities
probs = 1 / (1 + np.exp(-logits))
print(probs)

Github Repository

You can check details on how the model was fine-tuned and evaluated on the Github Repository

License

This model is licensed under OpenRAIL++

Citation

bibtex

@misc{bhargava2021generalization,
      title={Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics}, 
      author={Prajjwal Bhargava and Aleksandr Drozd and Anna Rogers},
      year={2021},
      eprint={2110.01518},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

@article{DBLP:journals/corr/abs-1908-08962,
  author    = {Iulia Turc and
               Ming{-}Wei Chang and
               Kenton Lee and
               Kristina Toutanova},
  title     = {Well-Read Students Learn Better: The Impact of Student Initialization
               on Knowledge Distillation},
  journal   = {CoRR},
  volume    = {abs/1908.08962},
  year      = {2019},
  url       = {http://arxiv.org/abs/1908.08962},
  eprinttype = {arXiv},
  eprint    = {1908.08962},
  timestamp = {Thu, 29 Aug 2019 16:32:34 +0200},
  biburl    = {https://dblp.org/rec/journals/corr/abs-1908-08962.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}