PhoWhisper-small (VinAI, BSD-3-Clause) is the strongest open Vietnamese speech recognizer — 6.33% WER on VIVOS and 11.08% on CommonVoice-vi (paper), roughly half the error rate of stock Whisper-small on Vietnamese. No compiled Snapdragon-NPU build of it was published anywhere as of this upload; this package provides one for Snapdragon 8 Gen 3 phones.

Measured on device (Qualcomm AI Hub hosted Snapdragon 8 Gen 3)

Component	Inference	Peak memory	Profile job
Encoder (30 s window)	26.4 ms	133.6 MB	`jpey22z15`
Decoder (per token)	6.2 ms	169.7 MB	`j5w4xxl6g`

≈ 150 ms for a typical 20-token Vietnamese utterance — far faster than real-time on a mid-range 2024 flagship chip.

Quantization notes (why full INT8)

W8A16 on the encoder works (measured 269 ms — 10× slower than INT8) but W8A16 on the decoder fails QNN context-binary conversion (exit code 14) — reproduced across chip generations. Blind 16-bit-activation quantization and KV-cache decoder graphs don't mix in this path; official Qualcomm quantized models avoid it via AIMET calibration.
Full INT8 (--quantize_full_type int8 --quantize_io, applied server-side at compile — no local AIMET needed) compiles and runs fast for both components.
Accuracy caveat — MEASURED, important: calibration-free INT8 significantly degrades this encoder. On a FLEURS-vi sample, on-device INT8 outputs diverge from FP32 (cosine similarity mean ≈ 0.52, min 0.17, across the 24 cross-attention K/V tensors; benign quantization would be ≥ 0.98). Treat these binaries as latency / deployment-path references, not production ASR. For usable accuracy: run an AI Hub hosted quantize job with Vietnamese calibration audio, then compile identically — same pipeline, still no local AIMET.

Reproduce (~20 lines, any HF Whisper fine-tune)

python

import qai_hub as hub
from qai_hub_models.models._shared.hf_whisper.model import HfWhisperEncoder, HfWhisperDecoder

device = hub.Device(attributes="chipset:qualcomm-snapdragon-8gen3")
for name, cls in (("encoder", HfWhisperEncoder), ("decoder", HfWhisperDecoder)):
    component = cls.from_pretrained(hf_whisper_version="vinai/PhoWhisper-small").eval()
    spec = component.get_input_spec()
    hub.submit_compile_job(
        model=component.convert_to_torchscript(spec), device=device, input_specs=spec,
        options="--target_runtime qnn_context_binary --quantize_full_type int8 --quantize_io",
        name=f"phowhisper-small-{name}-int8")

Requires Python ≥ 3.10, pip install "qai-hub-models[whisper-small]", and an aihub.qualcomm.com API token. I/O is quantized per --quantize_io; exact tensor specs are on the job pages. Companion QCS6490/IoT builds of Whisper-base/small live in this org; the same recipe with the QCS6490 target produces a PhoWhisper build for Dragonwing boards.

Attribution & license

Base model: vinai/PhoWhisper-small (BSD-3-Clause) — please cite the PhoWhisper paper. Export wrappers: qualcomm/ai-hub-models (BSD-3-Clause); related community validation: PR #322. Prepared by Phạm Văn Ngoan. Not an official Qualcomm or VinAI release; community-contributed, moderated for structure only.

qualcomm-ai-hub-community/phowhisper-small-vi-snapdragon-8gen3-qnn-int8

PhoWhisper-Small (Vietnamese ASR) — Snapdragon 8 Gen 3 · QNN context binary · full INT8

Measured on device (Qualcomm AI Hub hosted Snapdragon 8 Gen 3)

Quantization notes (why full INT8)

Reproduce (~20 lines, any HF Whisper fine-tune)

Attribution & license

qualcomm-ai-hub-community/
phowhisper-small-vi-snapdragon-8gen3-qnn-int8