Gemma3 Singlish → Sinhala Transliteration Model
Overview
This model performs Singlish (Romanized Sinhala) → Sinhala script transliteration.
It is designed to correctly handle:
- phonetic Singlish
- code-mixed Sinhala-English text
- adhoc spellings
- rare Sinhala conjunct characters
Examples of difficult conjunct clusters handled by the model:
Example:
| Singlish | Sinhala |
|---|
| gnathin | ඥාතින් |
| jnanaya | ඥානය |
| mannyokka | මඤ්ඤොක්කා |
| kshana | ක්ෂණ |
| shraddha | ශ්රද්ධා |
Model Architecture
Base model: Gemma
Fine-tuning method:
- LoRA (Low Rank Adaptation)
- Efficient fine-tuning for large language models
Training Strategy
The model was trained using a 3-phase curriculum training approach to improve performance on both common and rare transliteration patterns.
Phase 1 — Phonetic Learning
Datasets used:
- Phonetic dataset (1M rows)
Goal:
Learn general Singlish → Sinhala phonetic mapping
Example:
amma → අම්මා
gama → ගම
ratak → රටක්
Phase 2 — Adhoc + Code-Mix Learning
Datasets used:
- Adhoc dataset
- Code-mixed Sinhala-English dataset
Goal:
Handle:
- informal spellings
- mixed language sentences
- real-world Singlish usage
Example:
mama office ekata yanawa → මම office එකට යනවා
today mama busy → අද මම busy
Phase 3 — Rare Conjunct Booster
Datasets used:
- Adjunct dataset
- Replay samples from phonetic dataset
- Replay samples from adhoc dataset
Goal:
Improve difficult Sinhala conjunct clusters:
Example:
gnathin → ඥාතින්
kshana → ක්ෂණ
mannyokka → මඤ්ඤොක්කා
Datasets Used
Training data consists of multiple dataset types:
1. Phonetic Dataset
Romanized Sinhala → Sinhala script
Examples:
amma → අම්මා
ratak → රටක්
gama → ගම
2. Adhoc Dataset
Common Singlish spellings used in real communication.
Examples:
machan → මචං
mokadda → මොකද්ද
3. Code-Mixed Dataset
Mixed Sinhala + English sentences.
Examples:
mama meeting ekata yanawa → මම meeting එකට යනවා
api project eka finish karamu → අපි project එක finish කරමු
4. Adjunct Dataset
Synthetic dataset focused on rare Sinhala conjunct clusters.
Training Details
| Parameter | Value |
|---|
| Model | Gemma |
| Fine-tuning | LoRA |
| Batch Size | 2 |
| Gradient Accumulation | 8 |
| Learning Rate | 1.5e-4 |
| Scheduler | Cosine |
| Max Length | 256 |
Example Usage
1import torch
2from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
3
4repo_id = "Pudamya/small100-singlish-sinhala-3phase-final"
5
6tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
7model = AutoModelForSeq2SeqLM.from_pretrained(repo_id, trust_remote_code=True)
8
9sentences = [
10 "kohomada oyata",
11 "mama bath kanawa",
12 "api heta hamuwemu",
13 "mama gnathin hambenna yanawa",
14 "eyala ekka mannyokka kanna ymu",
15 "kshana",
16 "oyt gnanaya naha"
17]
18
19for s in sentences:
20 inputs = tokenizer(s, return_tensors="pt")
21 outputs = model.generate(**inputs, max_length=128)
22 result = tokenizer.decode(outputs[0], skip_special_tokens=True)
23
24 print("Singlish:", s)
25 print("Sinhala :", result)
26 print()
Output:
Singlish: kohomada oyata
Sinhala : කොහොමද ඔයාට
Singlish: mama bath kanawa
Sinhala : මම බත් කනවා
Singlish: api heta hamuwemu
Sinhala : අපි හෙට හමුවෙමු
Singlish: mama gnathin hambenna yanawa
Sinhala : මම ඥාතීන් හම්බෙන්න යනවා
Singlish: eyala ekka mannyokka kanna ymu
Sinhala : එයාලා එක්ක මඤ්ඤොක්කා කන්න යමු
Singlish: kshana
Sinhala : ක්ෂණ
Singlish: oyt gnanaya naha
Sinhala : ඔයාට ඥානය නැහැ
Applications
This model can be used for:
- Sinhala input systems
- Chat applications
- Social media text normalization
- Transliteration tools
- NLP preprocessing for Sinhala
Author
Pudamya Vidusini Rathnayake
Singlish → Sinhala Transliteration Research
Notes
Training time is dependent on dataset size and hardware configuration.
The model was trained using large phonetic datasets and specialized conjunct boosters to improve accuracy for complex Sinhala orthography.
