The first punctuation restoration model for the Kyrgyz language, achieving 94.1% precision and 90.3% F1-score — surpassing benchmarks for other low-resource languages.
📄 Published research:"AI-Based Punctuation Restoration using Transformer Model for Kyrgyz Language" — Uvalieva Z., Muhametjanova G. (SCOPUS-indexed)
Highlights
🏆 F1-score: 90.3% — outperforms comparable low-resource language models
🌍 First-of-its-kind for Kyrgyz (Turkic language family, ~7M speakers)
⚡ ONNX format — optimized for fast inference across frameworks
🎙️ ASR post-processing — designed to restore punctuation in speech-to-text output
Performance
Metric
Score
Precision
94.1%
Recall
86.8%
F1-Score
90.3%
Cross-Lingual Comparison
Model
Language
F1-Score
Ours (XLM-RoBERTa)
Kyrgyz
90.3%
Alam et al. (2020)
English (clean)
87.0%
Alam et al. (2020)
Bangla
69.5%
Nagy et al. (2021)
Hungarian
~82.0%
The model demonstrates strong performance on frequent punctuation marks (periods, commas) with reduced accuracy on rare marks (question marks, exclamation points) due to class imbalance.
Model Architecture
Parameter
Value
Base model
XLM-RoBERTa-base
Parameters
~270M
Transformer layers
12
Hidden dimensions
768
Attention heads
12
Export format
ONNX
Training Details
Dataset
A custom-built 200 MB Kyrgyz text corpus, collected over 2 months:
Source
Size
Description
Kyrgyz-Turkish Manas University Library
135 MB
Books (literature, math, physics)
Kyrgyz Wikipedia
40 MB
Encyclopedia articles
News portals
25 MB
Journalistic text
Preprocessing pipeline: PDF → EasyOCR text extraction → manual cleaning → JSON formatting with punctuation labels.
Data Augmentation
Specialized augmentation techniques designed for Kyrgyz agglutinative morphology:
Back-translation: Kyrgyz → English → Kyrgyz (simulating ASR-like errors)
Token-level modifications: Random insertions, deletions, swaps
Morphological transformations: Case form and morpheme modifications preserving grammatical correctness
Hyperparameters
Parameter
Value
Batch size
32
Epochs
10
Optimizer
Adam
Learning rate
5e-5
Regularization
Dropout
Hardware
Google Colab TPU
Training time
42 hours
How to Use
python
1import onnxruntime as ort
2import numpy as np
34# Load the ONNX model5session = ort.InferenceSession("model.onnx")67# Prepare input (see config.yaml for tokenizer settings)8# The model predicts punctuation labels for each token:9# O (no punctuation), COMMA, PERIOD, QUESTION, EXCLAMATION1011# Example inference12input_text ="бул кыргыз тилиндеги текст"13# Tokenize and run inference (see main.py for full pipeline)
Repository Structure
├── model.onnx # Trained model in ONNX format (1.11 GB)
├── main.py # Inference pipeline
├── env.py # Environment configuration
├── config.yaml # Hyperparameters and model config
├── requirements.txt # Python dependencies
└── Files/ # Additional model files
Intended Use
Use Case
Description
ASR post-processing
Restore punctuation in speech-to-text output for Kyrgyz
Text normalization
Clean and format raw Kyrgyz text with proper punctuation
