This model is a fine-tuned version of the
bert-base-uncased model to classify toxic comments.
The BERT model is finetuned using adversarial training to boost robustness against textual adversarial attacks.
You can use the model with the following code.
1from transformers import BertForSequenceClassification, BertTokenizer, TextClassificationPipeline
2model_path = "JiaqiLee/robust-bert-jigsaw"
3tokenizer = BertTokenizer.from_pretrained(model_path)
4model = BertForSequenceClassification.from_pretrained(model_path, num_labels=2)
5pipeline = TextClassificationPipeline(model=model, tokenizer=tokenizer)
6print(pipeline("You're a fucking nerd."))
The training data comes from this
Kaggle competition. We use 90% of the
train.csv data to train the model.
We augment original training data with adversarial examples generated by PWWS, TextBugger and TextFooler.
The model achieves 0.95 AUC in a 1500 rows held-out test set.
