Pre-compiled QAIRT context binary of the
NVIDIA Parakeet TDT 0.6B v3
encoder for Qualcomm Hexagon NPU (HTP). Targets Snapdragon X Elite;
inputs are pinned to a static 16-second audio window.
This is the encoder only. The TDT decoder, mel preprocessor, vocab and
config remain dynamic-shape and are sourced from the upstream ONNX export
at istupakov/parakeet-tdt-0.6b-v3-onnx.
Files
File
Size
Purpose
encoder-model.bin
631 MB
QAIRT context binary (compiled by Qualcomm AI Hub)
encoder-model.onnx
408 B
EPContext wrapper — points ORT's QNN EP at the .bin
How it was built
Static-shape FP32 encoder. Started from istupakov's FP32 ONNX export
of nvidia/parakeet-tdt-0.6b-v3. The encoder uses a dynamic-shape
attention mask (Shape → Gather → Range → Expand) that the HTP backend
cannot evaluate. The mask was constant-folded against the fixed 8 s
window and the input shapes were frozen to [1, 128, 1601] /
[1]. See scripts/build_npu_encoder.py
in OpenWritr.
Quantize. Qualcomm AI Hub submit_quantize_job with 32 FLEURS
calibration samples (weights_dtype=INT8, activations_dtype=INT16).
The INT8/INT16 mix is the standard HTP recipe for transformer encoders.
Compile. Qualcomm AI Hub submit_compile_job, target
Snapdragon X Elite CRD, options
--target_runtime qnn_context_binary --truncate_64bit_io.
How to use
Python (reference)
python
1import os, numpy as np, onnxruntime as ort, onnxruntime_qnn as qep
2os.add_dll_directory(qep.LIB_DIR_FULL_PATH)3ort.register_execution_provider_library("QNNExecutionProvider", qep.get_library_path())4npu_devs =[d for d in ort.get_ep_devices()5if d.device.type== ort.OrtHardwareDeviceType.NPU
6and d.ep_name =="QNNExecutionProvider"]7so = ort.SessionOptions()8so.add_provider_for_devices(npu_devs,{"backend_type":"htp","htp_performance_mode":"burst"})9sess = ort.InferenceSession("encoder-model.onnx", sess_options=so)1011# audio_signal: float32 [1, 128, 1601] (mel features, 8 s of 16 kHz audio)12# length: int32 [1] (= 801)13out0, out1 = sess.run(None,{"audio_signal": mel,"length": np.array([1601], np.int32)})14# out0: float32 [1, 1024, 201] encoder features15# out1: int32 [1] encoded length
Rust (ort 2.0)
ort 2.0-rc.12's session builders crash inside QnnHtp when consuming an
EPContext-wrapper ONNX. See OpenWritr's
src/asr/qnn_ffi.rs
for a direct ort_sys FFI workaround that loads + runs cleanly in
native Rust apps.
Required helper DLLs
The QNN backend (onnxruntime_providers_qnn.dll) loads several sibling
DLLs by name from os.add_dll_directory-registered paths:
Without libQnnHtpV73Skel.so + libqnnhtpv73.cat, the V73 stub fails
LoadLibrary with ERROR_MOD_NOT_FOUND (126) and the HTP session
creation later aborts with STATUS_STACK_BUFFER_OVERRUN (0xC0000409).
Easy to miss when bundling for a fresh machine; you'll want all of them
alongside the EP DLL.
Performance (measured on Snapdragon X Elite, X1E80100)
Audio length
Decode (Preproc + NPU encode + TDT decoder)
× Realtime
3 s
128 ms (single window)
23×
5.8 s
221 ms
26×
16.4 s
375 ms (chunked, 3 windows w/ 1 s overlap)
44×
23.0 s
626 ms (chunked, 4 windows)
37×
For comparison, CPU INT8 on the same chip runs the full pipeline at
~25× realtime average.
License & attribution
Weights: CC-BY-4.0 — NVIDIA parakeet-tdt-0.6b-v3. Original
attribution preserved; please credit NVIDIA in derivative work.
Compile output (.bin): Produced by Qualcomm AI Hub. Redistribution
for use on Qualcomm Snapdragon devices is permitted per AI Hub's
terms. Device-gated to Snapdragon X Elite; will not run on other
Qualcomm chipsets without recompilation.
