The XLM-RoBERTa model was proposed in
Unsupervised Cross-lingual Representation Learning at Scale by Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer and Veselin Stoyanov. It is based on Facebook's RoBERTa model released in 2019. It is a large multi-lingual language model, trained on 2.5TB of filtered CommonCrawl data. This model is
XLM-RoBERTa-large fine-tuned with the
CoNLL-2002 dataset in Dutch.
The model is a language model. The model can be used for token classification, a natural language understanding task in which a label is assigned to some tokens in a text.
Potential downstream use cases include Named Entity Recognition (NER) and Part-of-Speech (PoS) tagging. To learn more about token classification and other potential downstream use cases, see the Hugging Face
token classification docs.
The model should not be used to intentionally create hostile or alienating environments for people.
Significant research has explored bias and fairness issues with language models (see, e.g.,
Sheng et al. (2021) and
Bender et al. (2021)).
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.
Carbon emissions can be estimated using the
Machine Learning Impact calculator presented in
Lacoste et al. (2019).
1@article{conneau2019unsupervised,
2 title={Unsupervised Cross-lingual Representation Learning at Scale},
3 author={Conneau, Alexis and Khandelwal, Kartikay and Goyal, Naman and Chaudhary, Vishrav and Wenzek, Guillaume and Guzm{\'a}n, Francisco and Grave, Edouard and Ott, Myle and Zettlemoyer, Luke and Stoyanov, Veselin},
4 journal={arXiv preprint arXiv:1911.02116},
5 year={2019}
6}
This model card was written by the team at Hugging Face.
Use the code below to get started with the model. You can use this model directly within a pipeline for NER.
