XLMali | |
|
Вишејезични модел, 279 милиона параметара
Обучаван над корпусима српског и српскохрватског језика - 20 милијарди речи
Једнака подршка уноса на ћирилици и латиници!
|
Multilingual model, 279 million parameters
Trained on Serbian and Serbo-Croatian corpora - 20 billion words
Equal support for Cyrillic and Latin input!
|
1>>> from transformers import pipeline
2>>> unmasker = pipeline('fill-mask', model='te-sla/teslaXLM')
3>>> unmasker("Kada bi čovek znao gde će pasti on bi<mask>.")1>>> from transformers import AutoTokenizer, AutoModelForMaskedLM
2>>> from torch import LongTensor, no_grad
3>>> from scipy import spatial
4>>> tokenizer = AutoTokenizer.from_pretrained('te-sla/teslaXLM')
5>>> model = AutoModelForMaskedLM.from_pretrained('te-sla/teslaXLM', output_hidden_states=True)
6>>> x = " pas"
7>>> y = " mačka"
8>>> z = " svemir"
9>>> tensor_x = LongTensor(tokenizer.encode(x, add_special_tokens=False)).unsqueeze(0)
10>>> tensor_y = LongTensor(tokenizer.encode(y, add_special_tokens=False)).unsqueeze(0)
11>>> tensor_z = LongTensor(tokenizer.encode(z, add_special_tokens=False)).unsqueeze(0)
12>>> model.eval()
13>>> with no_grad():
14>>> vektor_x = model(input_ids=tensor_x).hidden_states[-1].squeeze()
15>>> vektor_y = model(input_ids=tensor_y).hidden_states[-1].squeeze()
16>>> vektor_z = model(input_ids=tensor_z).hidden_states[-1].squeeze()
17>>> print(spatial.distance.cosine(vektor_x, vektor_y))
18>>> print(spatial.distance.cosine(vektor_x, vektor_z))
Евалуација XLMR-base модела за српски језик |
Serbian XLMR-base models evaluation results |
   |
1@incollection{skoric2025:juznoslovenskijezici,
2 author = {Škorić, Mihailo and Petalinkar, Saša},
3 orcid = {0000-0003-4811-8692 and 0009-0007-9664-3594},
4 title = {Quality Textual Corpora and New South Slavic Language Models},
5 license = {https://creativecommons.org/licenses/by/4.0/},
6 booktitle = {Proceedings of the International Conference South Slavic Languages in the Digital Environment JuDig : Thematic Collection of Papers},
7 editor = {Moskovljević Popović, Jasmina and Stanković, Ranka},
8 isbn = {978-86-6153-791-2},
9 series = {South Slavic Languages in the Digital Environment JuDig},
10 publisher = {University of Belgrade — Faculty of Philology},
11 address = {Belgrade},
12 year = {2025},
13 volume = {1},
14 pages = {337--348},
15 note = {19},
16 doi = {10.18485/judig.2025.1.ch19},
17 doiurl = {http://doi.fil.bg.ac.rs/volume.php?pt=eb_ser&issue=judig-2025-1&i=19},
18 url = {http://doi.fil.bg.ac.rs/pdf/eb_ser/judig/2025-1/judig-2025-1-ch19.pdf}
19}
|
Истраживање jе спроведено уз подршку Фонда за науку Републике Србиjе, #7276, Text Embeddings – Serbian Language Applications – TESLA
|
This research was supported by the Science Fund of the Republic of Serbia, #7276, Text Embeddings - Serbian Language Applications - TESLA
|