Nemotron-3.5-ASR — LiteRT q4-mix, zh-TW fine-tuned (v2)
On-device (Android) LiteRT build of
Luigi/nemotron-3.5-asr-streaming-0.6b-zhtw
— the zh-TW
fine-tuned Nemotron (3.2× better Taiwan Mandarin than base).
q4-mix: INT4 encoder + fp16 decoder/joint + fp32 prompt-fusion. Bundle
663 MB.
Supersedes v1 (which packaged the zero-training warm-start).
Accuracy
| build | CV zh-TW CER (n=120) |
|---|
| fp32 fine-tuned model (reference) | 11.33 |
| this q4-mix LiteRT | 13.20 (+1.9 INT4 cost) |
| v1 q4-mix (base warm-start) | ~38 |
Per language-prompt slot on this build — all within ~0.7 CER, pick either:
| slot | q4-mix CER | fp32 CER |
|---|
zh-CN (4) | 13.20 | 11.57 |
auto (101) | 13.79 | 11.33 |
zh-TW (5) | 13.90 | 12.03 |
auto was the worst slot on the base model (50.58 CER) and is repaired by the
fine-tune — training used prompt_mode: unified, which trains the auto path
alongside the explicit language ID. Use auto when the speaker may switch
languages, or an explicit slot when you know it.
Other languages (fp32 FT, vs base): ko/de/ja/hi/en all improve, ar flat,
fr/es +1.5 — see the
source model card.
Files
| file | role | precision | size |
|---|
nemotron_encoder_q4.tflite | FastConformer encoder | INT4 FC (blockwise-128) + fp32 convs | 596 MB |
nemotron_prompt_fuse_fp32.tflite | 128-slot language-prompt fusion | fp32 | 18 MB |
nemotron_decoder_fp16.tflite | RNN-T prediction net | fp16 | 31 MB |
nemotron_joint_fp16.tflite | RNN-T joint (→ vocab 13088) | fp16 | 18 MB |
tokenizer.json / processor_config.json / config.json | runtime | — | — |
Run
1python -m nemotron.runner --models ./ --wav clip.wav --lang zh-CN --s2t --itn
2# --lang auto also works well (see slot table); --s2t gives Traditional output
Output is Simplified Chinese — the tokenizer cannot represent many common
Traditional characters, so apply OpenCC s2t (--s2t) for Traditional.
The INT4 encoder needs the LiteRT-Next CompiledModel runtime (Android NNAPI /
XNNPACK-QD8); the classic Interpreter cannot allocate dynamic-range INT4.
Set the CPU thread count explicitly (big-core count on big.LITTLE).
Provenance
Base: nvidia/nemotron-3.5-asr-streaming-0.6b (NVIDIA Open Model License),
fine-tuned for zh-TW (see source model card), then exported with
litert_torch 0.9.1 + ai_edge_quantizer 0.7.0 (INT4 weights pre-baked to the
export grid before quantization).
