You can deploy this model and start earning money today!

R4kSo1997/
opus-mt-en-pt-onnx-int8

Name: opus-mt-en-pt-onnx-int8
Availability: InStock

Views

No views yet

opus-mt-en-pt-onnx-int8

ONNX export with dynamic int8 quantization (AVX2) of Helsinki-NLP/opus-mt-tc-big-en-pt.

Files

encoder_model_quantized.onnx
decoder_model_quantized.onnx
decoder_with_past_model_quantized.onnx (KV-cache decoder)
tokenizer.json (Unigram, derived from merged vocab.json + source.spm/target.spm scores)
source.spm, target.spm, vocab.json, config.json, generation_config.json

Usage with optimum

python

from optimum.onnxruntime import ORTModelForSeq2SeqLM
from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("Helsinki-NLP/opus-mt-tc-big-en-pt")
model = ORTModelForSeq2SeqLM.from_pretrained(
    "R4kSo1997/opus-mt-en-pt-onnx-int8",
    encoder_file_name="encoder_model_quantized.onnx",
    decoder_file_name="decoder_model_quantized.onnx",
    decoder_with_past_file_name="decoder_with_past_model_quantized.onnx",
)
out = model.generate(**tok("Hello, how are you?", return_tensors="pt"), num_beams=4)
print(tok.batch_decode(out, skip_special_tokens=True))

License

Inherits CC-BY-4.0 from the upstream Helsinki-NLP model.