Welcome to burmese-pos-xlmr-base. This is a robust Part-of-Speech (POS) Tagging model specifically trained and optimized for the Burmese (Myanmar) language.

Rather than building from scratch, this project leverages the strong multilingual representations of xlm-roberta-base and adapts it to the specific grammatical nuances of Burmese syntax. The primary goal of this model is to reliably break down raw Burmese sentences and classify each token into its correct grammatical category (Nouns, Verbs, Adjectives, etc.), serving as a foundational tool for downstream NLP pipelines, data augmentation, and linguistic analysis.

📚 Dataset & Acknowledgement

A model is only as good as the data it learns from. This project relies entirely on the LULab/myPOS dataset, which contains 42,196 meticulously annotated Burmese sentences.

I would like to deeply acknowledge and credit the Language Understanding Laboratory (LULab) for their immense contribution to the Burmese NLP community. Their open-source dataset provided the essential ground truth required to train this model.

⚙️ Data Preprocessing & Alignment Strategy

Handling Burmese text presents unique tokenization challenges, particularly with compound words. In the raw dataset, some complex terms were annotated as fragmented sub-parts (e.g., ပိဋကတ်သုံးပုံ/n|စာပေ/n).

Instead of passing these directly to the model—which can confuse the contextual embeddings—I developed a custom parser to dynamically fuse these segments during the data loading phase. For example, ပိဋကတ်သုံးပုံ/n|စာပေ/n is unified into a single, cohesive noun entity (ပိဋကတ်သုံးပုံစာပေ/n). This ensures that the model learns semantically meaningful representations rather than fragmented sub-tokens, greatly improving its contextual awareness.

🧠 Training Methodology & Hyperparameters

To achieve high reliability without requiring massive computational resources, the training pipeline was heavily optimized. Here is the rationale behind the training configuration:

Base Architecture: xlm-roberta-base was chosen for its proven capability in cross-lingual transfer learning, providing a strong initialization for low-resource languages.
Memory-Safe Batching: The per_device_train_batch_size was set to 16, paired with gradient_accumulation_steps of 2. This results in an effective batch size of 32, which stabilizes the gradient updates and helps the model converge smoothly without exceeding standard GPU memory limits (such as Kaggle's T4 GPUs).
Mixed Precision (FP16): Enabled to accelerate computation and reduce memory footprint while maintaining numerical stability during backpropagation.
Max Sequence Length: Capped at 128 tokens, which comfortably accommodates the vast majority of standard Burmese sentence structures while avoiding wasted compute on heavy padding.
Learning Rate: A conservative 3e-5 was used to ensure the pre-trained backbone slowly adapts to the POS task without catastrophic forgetting of its original language understanding.
Optimization: Trained for 3 Epochs with a weight decay of 0.01 to prevent overfitting.

📊 Evaluation Results

The model was evaluated strictly on a validation split using the standard seqeval metric to ensure sequence-level integrity. The training logs demonstrate a steady, stable convergence:

Epoch	Training Loss	Validation Loss	Precision	Recall	F1-Score	Accuracy
1	0.6850	0.2706	94.09%	95.08%	94.58%	96.03%
2	0.5245	0.2095	95.33%	96.00%	95.66%	96.84%
3	0.4424	0.1871	95.64%	96.36%	96.00%	97.14%

Note: By Epoch 3, the model achieves a highly balanced Precision and Recall, indicating that it is not merely guessing the majority classes (like Nouns) but is effectively distinguishing complex grammatical boundaries.

⚠️ Limitations & Known Biases

While the model achieves high accuracy, users should be aware of the following limitations:

Pre-segmentation Required: Burmese is traditionally written without spaces. This model does not perform automatic word segmentation. Passing unspaced, continuous Burmese text will degrade accuracy. Always use a tokenizer (e.g., pyidaungsu) before running inference.
Encoding Constraint: The model is trained strictly on standard Unicode. Text written in the legacy Zawgyi font must be converted to Unicode prior to tokenization and inference.
Domain Shift: The training data heavily consists of formal writing, literature, and news. Accuracy may naturally drop when applied to highly informal social media comments, slang, or heavily abbreviated text.

🚀 How to Use

You can easily load this model via the Hugging Face pipeline. Since XLM-RoBERTa uses subword tokenization, we have included a helper function to aggregate the subwords back into coherent words.

Note: As Burmese is traditionally written without spaces, this model performs best when the input text is pre-segmented into words.

python

from transformers import pipeline

# 1. Load the POS Tagger pipeline
nlp = pipeline("token-classification", model="kalixlouiis/burmese-pos-xlmr-base")

# 2. Define aggregation function to clean subwords and merge tokens
def clean_and_aggregate(results):
    clean_results = []
    current_word = ""
    current_tag = ""
    
    for entity in results:
        # XLM-R subword prefix '▁' (or ' ') and spaces are removed
        word = entity['word'].replace(' ', '').replace('▁', '')
        tag = entity['entity']
        
        if word:
            if not current_word:
                current_word = word
                current_tag = tag
            else:
                # If the tag is the same, aggregate the subwords
                if tag == current_tag:
                    current_word += word
                else:
                    clean_results.append((current_word, current_tag))
                    current_word = word
                    current_tag = tag
    
    if current_word:
        clean_results.append((current_word, current_tag))
    return clean_results

# 3. Test with a pre-segmented Burmese sentence
text = "တော်သလင်း ကို ရာသီ အား ဖြင့် ကန်ရာသီ ဟု ခေါ် သည် ။ "
results = nlp(text)

# 4. Process and view the extracted grammatical tags
final_output = clean_and_aggregate(results)

for word, tag in final_output:
    print(f"Word: {word:<15} | POS Tag: {tag}")

# output
# Word: တော်သလင်း       | POS Tag: n
# Word: ကို             | POS Tag: ppm
# Word: ရာသီ            | POS Tag: n
# Word: အားဖြင့်        | POS Tag: ppm
# Word: ကန်ရာသီ         | POS Tag: n
# Word: ဟု              | POS Tag: part
# Word: ခေါ်            | POS Tag: v
# Word: သည်             | POS Tag: ppm
# Word: ။               | POS Tag: punc

🏷️ Supported POS Tags (Label Guide)

The model outputs tags based on the standard LULab/myPOS tagset. Here are some of the most common tags you will encounter:

n : Noun (နာမ်)
v : Verb (ကြိယာ)
adj : Adjective (နာမဝိသေသန)
pron : Pronoun (နာမ်စား)
part : Particle (ပစ္စည်း)
ppm : Postpositional Marker (ဝိဘတ်)
conj : Conjunction (သမ္ဗန္ဓ)

(For the complete exhaustive list of tags, please refer to the myPOS dataset page.)

📜 License

This project is open-sourced under the Apache License 2.0. You are free to use, modify, and distribute this model for both academic and commercial purposes, provided proper attribution is given.

👨‍💻 About the Author

This model was developed by Khant Sint Heinn. Working under the pseudonym Kalix Louis, he is a Machine Learning Engineer focused on Natural Language Processing (NLP), data foundations, and open-source AI development. His work is heavily centered on improving support for the Burmese language in modern AI systems by building high-quality datasets, practical tools, and scalable infrastructure for language technology.

🖋️ Citation

If you use this model in your research or applications, please cite it using the following BibTeX entry:

bibtex

@misc{kalixlouiis2026burmesepos,
  author       = {Khant Sint Heinn},
  title        = {burmese-pos-xlmr-base: A high-accuracy Part-of-Speech tagging model for Burmese},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{[https://huggingface.co/kalixlouiis/burmese-pos-xlmr-base](https://huggingface.co/kalixlouiis/burmese-pos-xlmr-base)}}
}

Please also ensure you cite the original dataset creators (LULab) who made this work possible:

bibtex

@misc{lulab2024mypos,
  author       = {Language Understanding Laboratory (LULab)},
  title        = {myPOS: A Part-of-Speech Dataset for Myanmar Language},
  publisher    = {Hugging Face},
  howpublished = {\url{[https://huggingface.co/datasets/LULab/myPOS](https://huggingface.co/datasets/LULab/myPOS)}}
}