Views
No views yet
⚠️ These files are NOT a plug-and-play model. They are the intermediate ONNX consumed by Edge-LLM's engine builder. The Talker/CodePredictor graphs contain Edge-LLM's customAttentionPluginop and a runtime-boundlm_head, so they will not run in stock ONNX Runtime or generic TensorRT. You must use TensorRT-Edge-LLM: build engines on your device, then run itsqwen3_tts_inference. See Usage.
llm/ # Talker (TalkerCausalLM) — model.onnx + model.onnx.data (FP16, ~2.83 GB) + sidecars
code_predictor/ # CodePredictor (residual RVQ codebooks) — model.onnx + .data + lm_heads / codec_embeddings / ...
code2wav/ # Code2Wav vocoder — model.onnx + .datamodel.onnx next to its model.onnx.data (external weights) and the sidecar
*.safetensors / tokenizer.json in the same directory.1# Build TensorRT-Edge-LLM v0.7.1, then point at its plugin:
2export EDGELLM_PLUGIN_PATH=$PWD/build/libNvInfer_edgellm_plugin.so
3
4# Build the 3 engines from this ONNX (per-GPU; ~5 min):
5./build/examples/llm/llm_build --onnxDir llm --engineDir engines/talker --maxInputLen 4096 --maxKVCacheCapacity 4096 --maxBatchSize 1
6./build/examples/llm/llm_build --onnxDir code_predictor --engineDir engines/code_predictor --maxInputLen 4096 --maxKVCacheCapacity 4096 --maxBatchSize 1
7./build/examples/multimodal/audio_build --onnxDir code2wav --engineDir engines/code2wav
8
9# Run inference (input.json: speaker + messages; see Edge-LLM TTS docs):
10./build/examples/omni/qwen3_tts_inference \
11 --talkerEngineDir engines/talker --code2wavEngineDir engines/code2wav/code2wav \
12 --tokenizerDir llm --inputFile input.json --outputAudioDir out
13# -> out/audio_req0.wav (24 kHz)ryan, serena, aiden, vivian, dylan, eric, uncle_fu, ono_anna, soheepython -m llm_loader.export_all_cli Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice <out_dir> (Edge-LLM export tools).