Cache-aware
streaming multilingual speech recognition. A 0.6 B
FastConformer-RNNT encoder with a 128-slot
language prompt, exported to
LiteRT (.tflite) with channelwise
weight-only INT8 encoder weights.
Activations and compute remain FP32, making the compact encoder compatible with
the standard LiteRT CPU runtime without requiring an NNAPI, GPU or NPU
delegate. For best quality across all languages, use the
FP16 build.
The encoder uses LiteRT's hybrid path: weights are stored channelwise INT8
while activations, outputs and compute remain FP32. The original compact graph
incorrectly quantized one shared non-weight constant, selecting an invalid
integer FULLY_CONNECTED path. This release restores that constant to FP32 and
allows allocate_tensors() on the plain LiteRT CPU interpreter. A hardware
delegate is optional.
1from ai_edge_litert.interpreter import Interpreter
2
3enc = Interpreter(model_path="nemotron-multilingual-encoder.tflite")
4enc.allocate_tensors()
5# io_map.json describes the 22 ports: audio/mel input, language-prompt slot,
6# carried encoder caches (attention / conv / pre-cache), and emitted features.
Production streaming, cache management and RNN-T greedy decoding are handled by
the
speech-android SDK.
Converted from
nvidia/nemotron-3.5-asr-streaming-0.6b
(NVIDIA NeMo) via ai-edge-torch. Licensed under the
NVIDIA Open Model License.
