Views
No views yet
distilbert-base-multilingual-cased for morphological analysis of the Tatar language. It was trained on a subset of 80,000 sentences from the Tatar Morphological Corpus. The model predicts fine-grained morphological tags (e.g., N+Sg+Nom, V+PRES(Й)+3SG).| Metric | Value | 95% CI |
|---|---|---|
| Token Accuracy | 0.9850 | [0.9841, 0.9860] |
| Micro F1 | 0.9851 | [0.9841, 0.9860] |
| Macro F1 | 0.4324 | [0.4744, 0.5093]* |
| POS | Accuracy |
|---|---|
| PUNCT | 1.0000 |
| NOUN | 0.9836 |
| VERB | 0.9535 |
| ADJ | 0.9626 |
| PRON | 0.9896 |
| PART | 0.9973 |
| PROPN | 0.9754 |
| ADP | 1.0000 |
| CCONJ | 1.0000 |
| ADV | 0.9845 |
1from transformers import AutoTokenizer, AutoModelForTokenClassification
2import torch
3
4model_name = "TatarNLPWorld/distilbert-tatar-morph"
5tokenizer = AutoTokenizer.from_pretrained(model_name)
6model = AutoModelForTokenClassification.from_pretrained(model_name)
7
8tokens = ["Татар", "теле", "бик", "бай", "."]
9inputs = tokenizer(tokens, is_split_into_words=True, return_tensors="pt", truncation=True)
10outputs = model(**inputs)
11predictions = torch.argmax(outputs.logits, dim=2)
12
13# Get tag mapping from model config
14id2tag = model.config.id2label
15
16word_ids = inputs.word_ids()
17prev_word = None
18for idx, word_idx in enumerate(word_ids):
19 if word_idx is not None and word_idx != prev_word:
20 tag_id = predictions[0][idx].item()
21 if isinstance(id2tag, dict):
22 tag = id2tag.get(str(tag_id), id2tag.get(tag_id, "UNK"))
23 else:
24 tag = id2tag[tag_id] if tag_id < len(id2tag) else "UNK"
25 print(tokens[word_idx], "->", tag)
26 prev_word = word_idxТатар -> N+Sg+Nom
теле -> N+Sg+POSS_3(СЫ)+Nom
бик -> Adv
бай -> Adj
. -> PUNCT1@misc{arabov-distilbert-tatar-morph-2026,
2 title = {DistilBERT multilingual fine-tuned for Tatar Morphological Analysis},
3 author = {Arabov Mullosharaf Kurbonovich},
4 year = {2026},
5 publisher = {Hugging Face},
6 url = {https://huggingface.co/TatarNLPWorld/distilbert-tatar-morph}
7}