This model takes the XLM-Roberta-base model which has been continued to pre-traine on a large corpus of Twitter in multiple languages.
It was developed following a similar strategy as introduced as part of the
Tweet Eval framework.
The model is further finetuned on the french part of the XNLI training dataset.
This model was developed to do Zero-Shot Text Classification in the realm of Hate Speech Detection. It is focused on the language of french as it was finetuned on data in saild languages. Since the base model was pre-trained on 100 different languages it has shown some effectiveness in other languages. Please refer to the list of languages in the
XLM Roberta paper1from transformers import pipeline
2classifier = pipeline("zero-shot-classification",
3 model="morit/french_xlm_xnli")
After loading the model you can classify sequences in the languages mentioned above. You can specify your sequences and a matching hypothesis to be able to classify your proposed candidate labels.
1sequence_to_classify = "Je pense que Marcon va gagner les elections?"
2
3
4# we can specify candidate labels and hypothesis:
5candidate_labels = ["politique", "sport"]
6hypothesis_template = "Cet example est {}"
7
8# classify using the information provided
9classifier(sequence_to_classify, candidate_labels, hypothesis_template=hypothesis_template)
10
11
12# Output
13#{'sequence': 'Je pense que Marcon va gagner les elections?',
14#'labels': ['politique', 'sport'],
15#'scores': [0.8195879459381104, 0.18041200935840607]}
16
This model was pre-trained on a set of 100 languages and follwed further training on 198M multilingual tweets as described in the original
paper. Further it was trained on the training set of XNLI dataset in french which is a machine translated version of the MNLI dataset. It was trained on 5 epochs of the XNLI train set and evaluated on the XNLI eval dataset at the end of every epoch to find the best performing model. The model which had the highest accuracy on the eval set was chosen at the end.
using a GPU (NVIDIA GeForce RTX 3090) resulting in a training time of 1h 47 mins.
The best performing model was evaluatated on the XNLI test set to get a comparable result
