Moonshine Streaming Small — merged-decoder int8 ONNX
int8-quantized ONNX export of
Moonshine,
the fast English speech-recognition model from
Useful Sensors
(
paper: Moonshine: Speech Recognition for Live Transcription and Voice Commands,
arXiv:2410.15608). All model weights, architecture,
and accuracy are Useful Sensors' work — please cite and credit them as the source of
this model.
This repo is only a packaging change: the decoder is shipped as a single
use_cache_branch-gated graph instead of the usual two-file split decoder. Nothing
about the math, weights, or quantization changes.
What's different from a standard streaming ONNX export
Streaming-Moonshine ONNX exports normally ship the decoder as two files —
decoder_model_quantized.onnx (step 0, no KV cache) and
decoder_with_past_model_quantized.onnx (steps 1..N, with cache) — whose weights are
~90% identical, so both sit resident in RAM at once. This repo merges that pair into
one decoder_model_merged_quantized.onnx switched by a use_cache_branch input —
the same single-graph shape the official onnx-community/moonshine-{tiny,base} exports
already use.
| file | size |
|---|
encoder_model_quantized.onnx | ~72 MB |
decoder_model_merged_quantized.onnx | ~144 MB |
tokenizer.json | ~3.6 MB |
Result: ~131 MB smaller on disk and roughly half the resident decoder RAM, with no
accuracy change — the merge operates on the already-int8 graphs (no re-quantization),
and the merged decoder is bit-for-bit functionally identical to the split pair (same
token sequence; logits agree to ~1e-5, pure graph-optimization reordering).
Lineage / attribution
- Useful Sensors — Moonshine
— the original model, weights, and architecture. The model is their work.
- Mazino0/moonshine-streaming-small-onnx
— the int8 split-decoder ONNX export this repack is built from.
- This repo — the split decoder merged into one
use_cache_branch graph (encoder
and tokenizer copied verbatim).
License
MIT, inherited from the upstream Moonshine release.
Usage
Built for
my-voice, an offline
English push-to-talk voice-typing daemon, but it's a standard
onnxruntime-loadable encoder + merged-decoder +
tokenizer.json layout.
