The XLM-RoBERTa model was proposed in
Unsupervised Cross-lingual Representation Learning at Scale by Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer and Veselin Stoyanov. It is based on Facebook's RoBERTa model released in 2019. It is a large multi-lingual language model, trained on 2.5TB of filtered CommonCrawl data. This model is
XLM-RoBERTa-large fine-tuned with the
conll2003 dataset in English.
Hong Kong Legal Information Institute
HKILL is a free, independent, non-profit document database providing the public with legal information relating to Hong Kong. We finetune the XLM-RoBERTa on the HKILL datasets. It contains docments
The model is a pretrained-finetuned language model. The model can be used for document classification, Named Entity Recognition (NER), especially on legal domain.
1>>> from transformers import pipeline,AutoTokenizer,AutoModelForTokenClassification
2>>> tokenizer = AutoTokenizer.from_pretrained("hklegal-xlm-r-base")
3>>> model = AutoModelForTokenClassification.from_pretrained("hklegal-xlm-r-base")
4>>> classifier = pipeline("ner", model=model, tokenizer=tokenizer)
5>>> classifier("Alya told Jasmine that Andrew could pay with cash..")
1@article{conneau2019unsupervised,
2 title={Unsupervised Cross-lingual Representation Learning at Scale},
3 author={Conneau, Alexis and Khandelwal, Kartikay and Goyal, Naman and Chaudhary, Vishrav and Wenzek, Guillaume and Guzm{\'a}n, Francisco and Grave, Edouard and Ott, Myle and Zettlemoyer, Luke and Stoyanov, Veselin},
4 journal={arXiv preprint arXiv:1911.02116},
5 year={2019}
6}
