encoder_model.onnx
decoder_model.onnx
decoder_with_past_model.onnx
config.json / generation_config.json
source.spm / target.spm / vocab.json / tokenizer_config.json / special_tokens_map.json
int8/
  encoder_model.onnx
  decoder_model.onnx
  decoder_with_past_model.onnx

Sanity check

3 short sentences, PyTorch original vs this ONNX export, num_beams=4, max_new_tokens=48:

input	PyTorch	ONNX fp32	ONNX int8
Bonjour, comment vas-tu?	Hola, com estàs?	Hola, com estàs?	Hola, com estàs?
Le chien court dans le parc.	El gos corre cap al parc.	El gos corre cap al parc.	El gos corre cap al parc.
Bonne journee a tous.	Que tingui un bon dia.	Que tingui un bon dia.	Que tingui un bon dia.

fp32 matches PyTorch on: 3/3. int8 output recorded above (not gated).

Usage

python

from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForSeq2SeqLM

model_id = "TigreGotico/opus-mt-fr-ca-onnx"
tok = AutoTokenizer.from_pretrained(model_id)
model = ORTModelForSeq2SeqLM.from_pretrained(model_id)

inputs = tok("Bonjour, comment vas-tu?", return_tensors="pt")
out = model.generate(**inputs, num_beams=4, max_new_tokens=48)
print(tok.decode(out[0], skip_special_tokens=True))

For the int8 quantized version, load from the int8/ subfolder:

model = ORTModelForSeq2SeqLM.from_pretrained(model_id, subfolder="int8")