A fine-tuned and INT8 quantized NLLB-200-distilled-600M model for high-quality translation between English, Vietnamese, and Japanese. Optimized for fast CPU inference with ONNX Runtime.

🎯 Highlights

75% smaller: 7GB → 1.8GB (INT8 quantization)
48% faster: Optimized for CPU inference
All 6 directions: EN↔VI, EN↔JA, VI↔JA
Production ready: ONNX format with Optimum integration

📊 Performance

Metric	FP32	INT8 (this model)
Model Size	7 GB	1.8 GB
Short Text	0.44s	0.26s
Long Text	2.56s	1.33s

Benchmarked on CPU with num_beams=1

🌐 Supported Languages

Source	Target	Code
English	Vietnamese	`eng_Latn` → `vie_Latn`
English	Japanese	`eng_Latn` → `jpn_Jpan`
Vietnamese	English	`vie_Latn` → `eng_Latn`
Vietnamese	Japanese	`vie_Latn` → `jpn_Jpan`
Japanese	English	`jpn_Jpan` → `eng_Latn`
Japanese	Vietnamese	`jpn_Jpan` → `vie_Latn`

📝 Example Translations

Direction	Input	Output
EN→VI	Hello, how are you?	Chào, bạn khỏe không?
EN→JA	Hello, how are you?	こんにちは、お元気ですか？
VI→EN	Thời tiết hôm nay rất đẹp.	The weather is very beautiful today.
VI→JA	Thời tiết hôm nay rất đẹp.	今日の天気はとても美しいです。
JA→EN	今日の天気はとても良いです。	The weather is very good today.
JA→VI	今日の天気はとても良いです。	Thời tiết hôm nay rất tốt.

🚀 Quick Start

Python (Optimum + ONNX Runtime)

python

from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForSeq2SeqLM

model_id = "sotalab/nllb-trilingual-en-vi-ja-onnx"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = ORTModelForSeq2SeqLM.from_pretrained(
    model_id,
    encoder_file_name="encoder_model_quantized.onnx",
    decoder_file_name="decoder_model_quantized.onnx",
    decoder_with_past_file_name="decoder_with_past_model_quantized.onnx",
)

def translate(text, src_lang, tgt_lang):
    tokenizer.src_lang = src_lang
    inputs = tokenizer(text, return_tensors="pt", max_length=256, truncation=True)
    tgt_lang_id = tokenizer.convert_tokens_to_ids(tgt_lang)
    
    outputs = model.generate(
        **inputs,
        forced_bos_token_id=tgt_lang_id,
        max_new_tokens=256,
        num_beams=1,
    )
    return tokenizer.decode(outputs[0], skip_special_tokens=True)

# English to Vietnamese
print(translate("Hello, how are you?", "eng_Latn", "vie_Latn"))
# Output: Chào, bạn khỏe không?

# English to Japanese
print(translate("Hello, how are you?", "eng_Latn", "jpn_Jpan"))
# Output: こんにちは、お元気ですか？

# Vietnamese to Japanese
print(translate("Tôi thích học tiếng Nhật.", "vie_Latn", "jpn_Jpan"))
# Output: 私は日本語を学ぶのが好きです。

Batch Translation

python

def translate_batch(texts, src_lang, tgt_lang):
    tokenizer.src_lang = src_lang
    inputs = tokenizer(texts, return_tensors="pt", padding=True, truncation=True, max_length=256)
    tgt_lang_id = tokenizer.convert_tokens_to_ids(tgt_lang)
    
    outputs = model.generate(**inputs, forced_bos_token_id=tgt_lang_id, max_new_tokens=256, num_beams=1)
    return [tokenizer.decode(out, skip_special_tokens=True) for out in outputs]

texts = ["Good morning", "How are you?", "Thank you very much"]
results = translate_batch(texts, "eng_Latn", "vie_Latn")
for text, result in zip(texts, results):
    print(f"{text} → {result}")

Translator Class

python

class TrilingualTranslator:
    LANG_CODES = {"en": "eng_Latn", "vi": "vie_Latn", "ja": "jpn_Jpan"}
    
    def __init__(self, model_id="sotalab/nllb-trilingual-en-vi-ja-onnx"):
        from transformers import AutoTokenizer
        from optimum.onnxruntime import ORTModelForSeq2SeqLM
        
        self.tokenizer = AutoTokenizer.from_pretrained(model_id)
        self.model = ORTModelForSeq2SeqLM.from_pretrained(
            model_id,
            encoder_file_name="encoder_model_quantized.onnx",
            decoder_file_name="decoder_model_quantized.onnx",
            decoder_with_past_file_name="decoder_with_past_model_quantized.onnx",
        )
    
    def translate(self, text, src="en", tgt="vi"):
        src_code = self.LANG_CODES[src]
        tgt_code = self.LANG_CODES[tgt]
        
        self.tokenizer.src_lang = src_code
        inputs = self.tokenizer(text, return_tensors="pt", max_length=256, truncation=True)
        tgt_lang_id = self.tokenizer.convert_tokens_to_ids(tgt_code)
        
        outputs = self.model.generate(**inputs, forced_bos_token_id=tgt_lang_id, max_new_tokens=256, num_beams=1)
        return self.tokenizer.decode(outputs[0], skip_special_tokens=True)

# Usage
translator = TrilingualTranslator()
print(translator.translate("Hello world", "en", "vi"))
print(translator.translate("Hello world", "en", "ja"))

📁 Model Files

File	Size	Description
`encoder_model_quantized.onnx`	399 MB	Encoder (INT8)
`decoder_model_quantized.onnx`	698 MB	Decoder (INT8)
`decoder_with_past_model_quantized.onnx`	674 MB	Decoder with KV-cache (INT8)
`tokenizer.json`	31 MB	Tokenizer
Total	~1.8 GB

🔧 Training Details

Base Model: facebook/nllb-200-distilled-600M
Fine-tuning Method: LoRA (Low-Rank Adaptation)
Training Hardware: NVIDIA H100 / A100
Quantization: Dynamic INT8 (ONNX Runtime)
Optimization: Optimum library

🎮 Live Demo

Try the model: Trilingual Translator Space

⚠️ Limitations

Optimized for general-purpose translation
May not handle highly specialized technical content perfectly
Best results with sentences under 256 tokens

📜 License

This model is released for Research Only.

🙏 Acknowledgments

Meta AI for NLLB-200
Hugging Face for Optimum and Transformers
ONNX Runtime for inference optimization

📖 Citation

bibtex

@misc{nllb-trilingual-2024,
  author = {SotaLab},
  title = {NLLB Trilingual Translation Model (EN-VI-JA) - INT8 ONNX},
  year = {2024},
  publisher = {Hugging Face},
  url = {https://huggingface.co/sotalab/nllb-trilingual-en-vi-ja-onnx}
}

belumind/nllb-trilingual-en-vi-ja-onnx

NLLB Trilingual Translation Model (English-Vietnamese-Japanese)

🎯 Highlights

📊 Performance

🌐 Supported Languages

📝 Example Translations

🚀 Quick Start

Python (Optimum + ONNX Runtime)

Batch Translation

Translator Class

📁 Model Files

🔧 Training Details

🎮 Live Demo

⚠️ Limitations

📜 License

🙏 Acknowledgments

📖 Citation

belumind/
nllb-trilingual-en-vi-ja-onnx