Views
No views yet
por-cat,
opus-2020-06-17). This repository only republishes an ONNX conversion of their weights; no model changes
were made.>>xxx<< target token is required or
supported.encoder_model.onnx, decoder_model.onnx, decoder_with_past_model.onnx (fp32, repo root)
source.spm, target.spm, vocab.json, tokenizer_config.json, special_tokens_map.json, config.json, generation_config.json
int8/
encoder_model.onnx, decoder_model.onnx, decoder_with_past_model.onnx (dynamic QUInt8 quantization)
(+ copies of the tokenizer files)1optimum-cli export onnx --model Helsinki-NLP/opus-mt-pt-ca \
2 --task text2text-generation-with-past --no-post-process ./fp32onnxruntime.quantization.quantize_dynamic(weight_type=QUInt8) applied to each of the three graphs
independently, written to int8/.MarianMTModel) vs ONNX (ORTModelForSeq2SeqLM), num_beams=4,
max_new_tokens=64, exact string match after decoding:1from transformers import AutoTokenizer
2from optimum.onnxruntime import ORTModelForSeq2SeqLM
3
4tok = AutoTokenizer.from_pretrained("TigreGotico/opus-mt-pt-ca-onnx")
5model = ORTModelForSeq2SeqLM.from_pretrained("TigreGotico/opus-mt-pt-ca-onnx")
6
7text = "O tempo está bom hoje."
8ids = model.generate(**tok(text, return_tensors="pt"), num_beams=4, max_new_tokens=64)
9print(tok.decode(ids[0], skip_special_tokens=True))
10# El temps està bé avui.subfolder="int8" instead.