encoder_model.onnx
decoder_model.onnx
decoder_with_past_model.onnx
config.json / generation_config.json
source.spm / target.spm / vocab.json / tokenizer_config.json / special_tokens_map.json
int8/
  encoder_model.onnx
  decoder_model.onnx
  decoder_with_past_model.onnx

Sanity check

3 short sentences, PyTorch original vs this ONNX export, num_beams=4, max_new_tokens=48:

input	PyTorch	ONNX fp32	ONNX int8
Ola, como estas?	Ola, como estas?	Ola, como estas?	Ola, como estas?
O cao corre no parque.	O cao corre no parque.	O cao corre no parque.	O cao corre no parque.
Bom dia a todos.	Bos días a todos.	Bos días a todos.	Bos días a todos.

fp32 matches PyTorch on: 3/3. int8 output recorded above (not gated).

Usage

python

from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForSeq2SeqLM

model_id = "TigreGotico/opus-mt-pt-gl-onnx"
tok = AutoTokenizer.from_pretrained(model_id)
model = ORTModelForSeq2SeqLM.from_pretrained(model_id)

inputs = tok("Ola, como estas?", return_tensors="pt")
out = model.generate(**inputs, num_beams=4, max_new_tokens=48)
print(tok.decode(out[0], skip_special_tokens=True))

For the int8 quantized version, load from the int8/ subfolder:

model = ORTModelForSeq2SeqLM.from_pretrained(model_id, subfolder="int8")