Qwen3-ASR-1.7B ONNX deployment build
This directory contains a locally exported deployment build of
Qwen/Qwen3-ASR-1.7B.
Format
- ONNX opset 18
- native FP16 audio encoder with FP32 inputs and outputs
- INT4 RTN decoder, block size 64, ONNX Runtime accuracy level 4
- FP16 token embedding table
- shared external decoder weights
- model directory size: about 3.5 GB
This is a split native ONNX Runtime pipeline. It is not a sherpa-onnx model and
cannot be dropped into a sherpa-onnx runner without an adapter.
Provenance
- Base model:
Qwen/Qwen3-ASR-1.7B, Apache-2.0
- Exporter:
andrewleech/qwen3-asr-onnx
- Exporter commit:
4aee2ae0c355c7bc33bd5c09befec9e9eabd4b50
- Python 3.13
- PyTorch 2.11.0+cu128
- ONNX Runtime GPU 1.23.2
The official model was downloaded and verified locally before export. The
encoder, decoder-init, and decoder-step graphs all passed ONNX Runtime loading
checks. CUDA validation rejects silent CPU fallback.
Local validation
Run from the exporter repository:
1$env:PYTHONUTF8 = "1"
2uv sync
3uv run python validate.py `
4 --model models/Qwen3-ASR-1.7B `
5 --onnx-dir release/qwen3-asr-1.7b-onnx-int4 `
6 --audio C:\path\to\test.wav `
7 --provider cuda `
8 --max-tokens 1024
The validation command also loads the original PyTorch model for comparison,
so it is slower and requires the original weights. Production inference needs
only the ONNX model files, tokenizer, audio preprocessing, and ONNX Runtime.
Measured results
Measured on an RTX 5070 Ti 16 GB:
| Audio | Duration | ONNX encoder | ONNX decoder | Total | RTF |
|---|
| Japanese news/narration | 108.9 s | 0.54 s | 51.29 s | 51.83 s | 0.48 |
| Japanese dialogue | 104.7 s | 0.53 s | 47.42 s | 47.95 s | 0.46 |
Normalized character error rate against the two supplied transcripts was
3.21% and 5.14%, or 4.20% aggregate. Punctuation, spaces, and symbol
differences were excluded. A company RTX 4090 should be faster, but it must be
benchmarked on that machine before setting concurrency limits.
Reproduction outline
1uv run python export.py `
2 --model models/Qwen3-ASR-1.7B `
3 --output output/qwen3-asr-1.7b-fp32 `
4 --device cpu `
5 --dtype fp32 `
6 --opset 18 `
7 --no-share-weights
8
9uv run python share_weights.py output/qwen3-asr-1.7b-fp32 --verify
10
11uv run python quantize_nbits.py `
12 --input output/qwen3-asr-1.7b-fp32 `
13 --output output/qwen3-asr-1.7b-fp32 `
14 --bits 4 `
15 --block-size 64 `
16 --accuracy-level 4 `
17 --algo rtn
18
19uv run python share_weights.py `
20 output/qwen3-asr-1.7b-fp32 `
21 --suffix int4 `
22 --verify
23
24uv run python export_encoder_native_fp16.py `
25 --model models/Qwen3-ASR-1.7B `
26 --output output/qwen3-asr-1.7b-fp32/encoder.int4.onnx `
27 --opset 18 `
28 --verify
29
30uv run python convert_embed_fp16.py `
31 --model-dir output/qwen3-asr-1.7b-fp32
The release directory was assembled from the native FP16 encoder, shared INT4
decoder pair, shared INT4 external weights, FP16 embeddings, tokenizer, and
configuration files.
