Fine-tuned and quantized small-100 seq2seq model for translating Roman-script Sinhala (Singlish) mixed with English into pure Sinhala Unicode script. Exported to ONNX and quantized to INT8 for fast CPU inference.

What it does

Takes informal Singlish text typed by Sri Lankans on social media and converts it to Sinhala script. English words are translated to their Sinhala meaning — not transliterated phonetically.

Input (Singlish)	Output (Sinhala)
mama hungry, kanna yamu	මම බඩගිනියි, කෑම කමු
bro API eka call karanna puluwanda	බ්‍රෝ API එක කෝල් කරන්න පුළුවන්ද
today weather eka honda ne	අද කාලගුණය හොඳයි නේ
mage salary eka late una	මගේ වැටුප ප්‍රමාද වුණා
traffic eka godak awul	වාහන තදබදය ගොඩක් අවුල්

Training details

Base model: alirezamsh/small100 (encoder-decoder seq2seq)
Fine-tuning: LoRA (r=64, alpha=128) on combined Singlish-Sinhala dataset
Training data:
- Real social media code-mix: ~26,700 rows × 4 upsample
- Phonetic transliteration replay: 20,000 rows
- Adhoc pairs replay: 5,000 rows
- Total: ~211,000 training samples
Evaluation: Golden test set of 2,549 real Singlish sentences
Best CER: ~13.8% on golden test set

Export details

Exported from merged LoRA fp32 model to ONNX
INT8 dynamic quantization using AVX512 VNNI kernels via optimum
Each ONNX file quantized separately: encoder, decoder, decoder_merged
Greedy decoding (num_beams=1) recommended for CPU inference

Installation

pip install optimum[onnxruntime] transformers sentencepiece

Usage

Basic inference

python

from optimum.onnxruntime import ORTModelForSeq2SeqLM
from transformers import AutoTokenizer

model_id = "savinugunarathna/small-100-Singlish-Sinhala-CodeMix2-ONNX-INT8"

model    = ORTModelForSeq2SeqLM.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)
tokenizer.src_lang = "en"
tgt_lang_id = tokenizer.lang_code_to_id["si"]

def translate(text):
    inputs = tokenizer(
        text,
        return_tensors="pt",
        truncation=True,
        max_length=128,
    )
    outputs = model.generate(
        **inputs,
        max_new_tokens=64,
        num_beams=1,           # greedy — fastest on CPU
        forced_bos_token_id=tgt_lang_id,
    )
    return tokenizer.decode(outputs[0], skip_special_tokens=True).strip()

print(translate("mama hungry, kanna yamu"))
# → මම බඩගිනියි, කෑම කමු

Batch inference

python

texts = [
    "mama hungry, kanna yamu",
    "bro API eka call karanna puluwanda",
    "today weather eka honda ne",
]

inputs = tokenizer(
    texts,
    return_tensors="pt",
    padding=True,
    truncation=True,
    max_length=128,
)
outputs = model.generate(
    **inputs,
    max_new_tokens=64,
    num_beams=1,
    forced_bos_token_id=tgt_lang_id,
)
for out in outputs:
    print(tokenizer.decode(out, skip_special_tokens=True).strip())

FastAPI server example

python

from fastapi import FastAPI
from pydantic import BaseModel
from optimum.onnxruntime import ORTModelForSeq2SeqLM
from transformers import AutoTokenizer

app       = FastAPI()
model_id  = "savinugunarathna/small-100-Singlish-Sinhala-CodeMix2-ONNX-INT8"
model     = ORTModelForSeq2SeqLM.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)
tokenizer.src_lang  = "en"
tgt_lang_id = tokenizer.lang_code_to_id["si"]

class Request(BaseModel):
    text: str

@app.post("/translate")
def translate(req: Request):
    inputs = tokenizer(req.text, return_tensors="pt", truncation=True, max_length=128)
    outputs = model.generate(
        **inputs,
        max_new_tokens=128,
        num_beams=3,
        forced_bos_token_id=tgt_lang_id,
    )
    result = tokenizer.decode(outputs[0], skip_special_tokens=True).strip()
    return {"input": req.text, "output": result}

Run with:

uvicorn app:app --host 0.0.0.0 --port 8000

Performance

Benchmarked on CPU (single thread, greedy decoding, max_new_tokens=64):

Setup	Latency per sentence
PyTorch fp32 + beam=3	~3-5 seconds
ONNX INT8 + greedy	~200-400ms

Limitations

Optimized for short conversational sentences (5-20 words)
Very long sentences (50+ words) may see quality degradation
Rare English words without common Sinhala equivalents may be transliterated rather than translated
Conjunct consonants (e.g. ශ්‍රී, ක්‍රිකට්) may occasionally render without ZWJ joiner depending on font rendering

Related models

small-100-Singlish-Sinhala-CodeMix2 — full fp32 PyTorch version of this model
Small100-Singlish-Sinhala-Merged — base merged model before code-mix fine-tuning