The experiments conduced for assessment of racial bias shows that this model can perpetuate racial bias based on brazilian portuguese dialect called "pretuguês".

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-4
train_batch_size: 16
eval_batch_size: 16
optimizer: Adam
num_epochs: 20

Usage

python

from transformers import AutoTokenizer  # Or BertTokenizer
from transformers import AutoModelForPreTraining  # Or BertForPreTraining for loading pretraining heads
from transformers import AutoModel  # or BertModel, for BERT without pretraining heads
model = AutoModelForPreTraining.from_pretrained('cassiasilvaR/bertimbau-hate-detector')
tokenizer = AutoTokenizer.from_pretrained('cassiasilvaR/bertimbau-hate-detector', do_lower_case=False)