Glot500 model (Glot500-m) pre-trained on 500+ languages using a masked language modeling (MLM) objective. It was introduced in
this paper (ACL 2023) and first released in
this repository.
1>>> from transformers import pipeline
2>>> unmasker = pipeline('fill-mask', model='cis-lmu/glot500-base')
3>>> unmasker("Hello I'm a <mask> model.")
1>>> from transformers import AutoTokenizer, AutoModelForMaskedLM
2
3>>> tokenizer = AutoTokenizer.from_pretrained('cis-lmu/glot500-base')
4>>> model = AutoModelForMaskedLM.from_pretrained("cis-lmu/glot500-base")
5
6>>> # prepare input
7>>> text = "Replace me by any text you'd like."
8>>> encoded_input = tokenizer(text, return_tensors='pt')
9
10>>> # forward pass
11>>> output = model(**encoded_input)
1@article{imanigooghari-etal-2023-glot500,
2 title={Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages},
3 author={ImaniGooghari, Ayyoob and Lin, Peiqin and Kargaran, Amir Hossein and Severini, Silvia and Jalili Sabet, Masoud and Kassner, Nora and Ma, Chunlan and Schmid, Helmut and Martins, Andr{\'e} and Yvon, Fran{\c{c}}ois and Sch{\"u}tze, Hinrich},
4 journal={arXiv preprint arXiv:2305.12182},
5 year={2023}
6}
