Load the Model

from transformers import pipeline

pipe = pipeline(model='minuva/MiniLMv2-toxic-jigsaw', task='text-classification')
pipe("This is pure trash")
# [{'label': 'toxic', 'score': 0.9383478164672852}]

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 6e-05
train_batch_size: 48
eval_batch_size: 48
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 10
warmup_ratio: 0.1

Metrics (comparison with teacher model)

Teacher (params)	Student (params)	Set (metric)	Score (teacher)	Score (student)
unitary/toxic-bert (110M)	MiniLMv2-toxic-jigsaw (23M)	Test (ROC_AUC)	0.98636	0.98600

Deployment

Check out fast-nlp-text-toxicity repository for a FastAPI based server to deploy this model in CPU devices.