BERT-Small finetuned on KazNERD (Kazakh Named Entity Recognition) starting from Abzalbek89/kk-bert-small-unigram — pretrained with the unigram tokenizer.

Part of a comparative study of subword tokenizers for Kazakh small language models.

Test set results

metric	value
F1 (span-level, seqeval)	87.99
Precision	86.36
Recall	89.68
Token accuracy	97.75

Training


Base model	Abzalbek89/kk-bert-small-unigram
Dataset	IS2AI/KazNERD (IOB2, 25 entity types)
Epochs	5
Batch size	16
Learning rate	5e-5 (warmup 10%, weight decay 0.01)
Precision	fp16
Hardware	RTX A4000
Time	13.6 min
Seed	42

Usage

python

from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline
tok = AutoTokenizer.from_pretrained("Abzalbek89/kk-bert-small-unigram-kaznerd")
model = AutoModelForTokenClassification.from_pretrained("Abzalbek89/kk-bert-small-unigram-kaznerd")
ner = pipeline("ner", model=model, tokenizer=tok, aggregation_strategy="simple")
ner("Қазақстан Республикасының Президенті Қасым-Жомарт Тоқаев Алматыға барды.")

Companion artifacts

Pretrained base: Abzalbek89/kk-bert-small-unigram
Aggregate metrics: Abzalbek89/kk-tokenizer-fertility-baseline