encoder_model.onnx
decoder_model.onnx
decoder_with_past_model.onnx
config.json / generation_config.json
source.spm / target.spm / vocab.json / tokenizer_config.json / special_tokens_map.json
int8/
  encoder_model.onnx
  decoder_model.onnx
  decoder_with_past_model.onnx

Sanity check

3 short sentences, PyTorch original vs this ONNX export, num_beams=4, max_new_tokens=48:

input	PyTorch	ONNX fp32	ONNX int8
Hola, como estas?	Kaixo, zer moduz?	Kaixo, zer moduz?	Kaixo, zer moduz?
El perro corre en el parque.	Txakurra korrika dabil parkean.	Txakurra korrika dabil parkean.	Txakurra korrika dabil parkean.
Buenos dias a todos.	Egun on denoi.	Egun on denoi.	Egun on denoi.

fp32 matches PyTorch 3/3. int8 output recorded above (not gated).

Usage

python

from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForSeq2SeqLM

model_id = "TigreGotico/opus-mt-es-eu-onnx"
tok = AutoTokenizer.from_pretrained(model_id)
model = ORTModelForSeq2SeqLM.from_pretrained(model_id)

inputs = tok("Hola, como estas?", return_tensors="pt")
out = model.generate(**inputs, num_beams=4, max_new_tokens=48)
print(tok.decode(out[0], skip_special_tokens=True))

For the int8 quantized version, load from the int8/ subfolder:

model = ORTModelForSeq2SeqLM.from_pretrained(model_id, subfolder="int8")