Views
No views yet
1import onnxruntime as ort
2from optimum.onnxruntime import ORTModelForSeq2SeqLM
3from transformers import AutoTokenizer
4
5so = ort.SessionOptions()
6so.intra_op_num_threads = 1 # single thread is fastest for this model
7so.inter_op_num_threads = 1
8
9model = ORTModelForSeq2SeqLM.from_pretrained(
10 "klebster/g2p_multilingual_byT5_small_onnx",
11 provider="CPUExecutionProvider",
12 session_options=so,
13)
14tokenizer = AutoTokenizer.from_pretrained("klebster/g2p_multilingual_byT5_small_onnx")
15
16inputs = tokenizer("<eng-us>: hello", padding=True, add_special_tokens=False, return_tensors="pt")
17preds = model.generate(**inputs, num_beams=1, max_length=50)
18print(tokenizer.decode(preds[0], skip_special_tokens=True))
19# Output: ˈhɛɫoʊ<language_code>: word (e.g. <fra>: bonjour, <ger>: Straße). See CharsiuG2P for all 100 language codes.| Configuration | ms/word | vs PyTorch CPU |
|---|---|---|
| ONNX INT8 + threads=1 | ~135 | 2.78x faster |
| ONNX INT8 + threads=8 | ~196 | 1.92x faster |
| ONNX FP32 + threads=8 | ~392 | 0.96x |
| PyTorch CPU (baseline) | ~375 | 1.00x |
spa and spa-me are identicalspa (European Spanish) and spa-me (Mexican Spanish) dictionaries are identical in the
upstream CharsiuG2P repository. They should differ in the /s/–/θ/ distinction (ceceo/seseo).
See CharsiuG2P issue #15.1@misc{zhu2022byt5modelmassivelymultilingual,
2 title={ByT5 model for massively multilingual grapheme-to-phoneme conversion},
3 author={Jian Zhu and Cong Zhang and David Jurgens},
4 year={2022},
5 eprint={2204.03067},
6 archivePrefix={arXiv},
7 primaryClass={cs.CL},
8 url={https://arxiv.org/abs/2204.03067},
9}
10
11@misc{noel2026g2pmultilingualbyT5smallonnx,
12 title={Multilingual G2P ByT5 Small — ONNX export},
13 author={Kleber Noel},
14 year={2026},
15 month={apr},
16 url={https://huggingface.co/klebster/g2p_multilingual_byT5_small_onnx},
17}