BERT-Small finetuned on KazNERD (Kazakh Named Entity Recognition) starting from Abzalbek89/kk-bert-small-bpe — pretrained with the bpe tokenizer.

Part of a comparative study of subword tokenizers for Kazakh small language models.

Test set results

metric	value
F1 (span-level, seqeval)	86.96
Precision	85.22
Recall	88.77
Token accuracy	97.55

Training


Base model	Abzalbek89/kk-bert-small-bpe
Dataset	IS2AI/KazNERD (IOB2, 25 entity types)
Epochs	5
Batch size	16
Learning rate	5e-5 (warmup 10%, weight decay 0.01)
Precision	fp16
Hardware	RTX A4000
Time	9.0 min
Seed	42

Usage

python

from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline
tok = AutoTokenizer.from_pretrained("Abzalbek89/kk-bert-small-bpe-kaznerd")
model = AutoModelForTokenClassification.from_pretrained("Abzalbek89/kk-bert-small-bpe-kaznerd")
ner = pipeline("ner", model=model, tokenizer=tok, aggregation_strategy="simple")
ner("Қазақстан Республикасының Президенті Қасым-Жомарт Тоқаев Алматыға барды.")

Companion artifacts

Pretrained base: Abzalbek89/kk-bert-small-bpe
Aggregate metrics: Abzalbek89/kk-tokenizer-fertility-baseline