We are sharing smaller versions of
bert-base-multilingual-cased that handle a custom number of languages.
Unlike
distilbert-base-multilingual-cased, our versions give exactly the same representations produced by the original model which preserves the original accuracy.
This model handles the following languages: english, french, spanish, german, chinese, arabic, russian, portuguese, italian, and urdu. It produces the same representations as
bert-base-multilingual-cased while being 22.5% smaller in size.
1from transformers import AutoTokenizer, AutoModel
2
3tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-10lang-cased")
4model = AutoModel.from_pretrained("Geotrend/bert-base-10lang-cased")
5
To generate other smaller versions of multilingual transformers please visit
our Github repo.
1@inproceedings{smallermbert,
2 title={Load What You Need: Smaller Versions of Multilingual BERT},
3 author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
4 booktitle={SustaiNLP / EMNLP},
5 year={2020}
6}
Please contact
amine@geotrend.fr for any question, feedback or request.
