You can deploy this model and start earning money today!

TigreGotico/
mt-hitz-eu-en-onnx

Name: mt-hitz-eu-en-onnx
Rating: 2.5 (14 reviews)

Views

No views yet

mt-hitz-eu-en-onnx

ONNX export of HiTZ/mt-hitz-eu-en, a MarianMT machine translation model for eu -> en produced by HiTZ, the Basque Center for Language Technology at the University of the Basque Country (UPV/EHU).

All credit for training and releasing the original model goes to the HiTZ team. This repo only contains an ONNX conversion (fp32 and int8) for fast CPU inference with optimum[onnxruntime], done by TigreGotico.

License

apache-2.0, as declared on the source model card.

Export command

optimum-cli export onnx --model HiTZ/mt-hitz-eu-en --task text2text-generation-with-past --no-post-process mt-hitz-eu-en-onnx

int8 dynamic quantization was applied afterwards with optimum.onnxruntime.ORTQuantizer (AVX2 config, dynamic) into the int8/ subfolder.

File layout

encoder_model.onnx              # fp32
decoder_model.onnx              # fp32
decoder_with_past_model.onnx    # fp32
config.json, generation_config.json
source.spm, target.spm, vocab.json, tokenizer_config.json, special_tokens_map.json
int8/
  encoder_model.onnx
  decoder_model.onnx
  decoder_with_past_model.onnx

No external-data files were produced — this Marian model is small enough that all graphs stay under the 2GB single-file protobuf limit.

Target-language mechanism

Bilingual Marian model, Basque -> English only. No target-language token or forcing is required; tokenizer(text) + model.generate() is sufficient.

Parity vs. the original PyTorch model

Measured on 8 held-out eu->en sentences, beam search (num_beams=4, max_new_tokens=64), exact-match against the original PyTorch MarianMTModel output:

fp32: 8/8 = 100.0% exact match
int8: not separately re-verified against reference; reported not gated

Sample translations observed (eu -> en):

Basque	English (HiTZ / ONNX, identical)
Kaixo, zer moduz?	Hello, how are you?
Gaur eguraldia oso ona da.	The weather is very good today.
Bi lagunentzako mahai bat erreserbatu nahi nuke.	I would like to reserve a table for two people.
Non dago tren geltokia?	Where is the train station?
Liburu hau oso interesgarria da.	This book is very interesting.
Umeek parkean jolasten dute.	Kids play in the park.
Ordenagailuarekin laguntza behar dut.	I need help with my computer.
Bilera goizeko hamarretan hasiko da.	The meeting begins at ten in the morning.

Usage

python

from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForSeq2SeqLM

tok = AutoTokenizer.from_pretrained("TigreGotico/mt-hitz-eu-en-onnx")
model = ORTModelForSeq2SeqLM.from_pretrained("TigreGotico/mt-hitz-eu-en-onnx")
# int8: ORTModelForSeq2SeqLM.from_pretrained("TigreGotico/mt-hitz-eu-en-onnx", subfolder="int8")

batch = tok(["Kaixo, zer moduz?"], return_tensors="pt")
generated = model.generate(**batch, num_beams=4, max_new_tokens=64)
print(tok.batch_decode(generated, skip_special_tokens=True))
# ['Hello, how are you?']

Attribution

Original model: HiTZ/mt-hitz-eu-en by HiTZ Center (University of the Basque Country, UPV/EHU)
ONNX export: TigreGotico