Views
No views yet
1from transformers import AutoTokenizer, AutoModel
2
3# load the tokenizer and the model
4tokenizer = AutoTokenizer.from_pretrained("racai/distilbert-base-romanian-cased")
5model = AutoModel.from_pretrained("racai/distilbert-base-romanian-cased")
6
7# tokenize a test sentence
8input_ids = tokenizer.encode("Aceasta este o propoziție de test.", add_special_tokens=True, return_tensors="pt")
9
10# run the tokens trough the model
11outputs = model(input_ids)
12
13print(outputs)bert-base-romanian-cased-v1.| Model | Size (MB) | Params (Millions) |
|---|---|---|
| bert-base-romanian-cased-v1 | 477 | 124 |
| distilbert-base-romanian-cased | 312 | 81 |
| Model | UPOS | XPOS | NER | SAPN | SAR | DI | STS |
|---|---|---|---|---|---|---|---|
| bert-base-romanian-cased-v1 | 98.00 | 96.46 | 85.88 | 98.07 | 79.61 | 95.58 | 80.30 |
| distilbert-base-romanian-cased | 97.97 | 97.08 | 83.35 | 98.20 | 80.51 | 96.31 | 80.57 |
1@article{avram2021distilling,
2 title={Distilling the Knowledge of Romanian BERTs Using Multiple Teachers},
3 author={Andrei-Marius Avram and Darius Catrina and Dumitru-Clementin Cercel and Mihai Dascălu and Traian Rebedea and Vasile Păiş and Dan Tufiş},
4 journal={ArXiv},
5 year={2021},
6 volume={abs/2112.12650}
7}