You can deploy this model and start earning money today!

foryoung365/
Fun-ASR-Nano-2512-int4-onnx

Name: Fun-ASR-Nano-2512-int4-onnx
Availability: InStock

sherpa-onnx

onnx

automatic-speech-recognition

funasr

int4

quantized

FunAudioLLM/Fun-ASR-Nano-2512

quantized

Views

No views yet

Fun-ASR-Nano-2512 INT4 ONNX for sherpa-onnx

This repository contains a locally quantized INT4 ONNX variant of FunAudioLLM/Fun-ASR-Nano-2512, prepared for sherpa-onnx offline inference.

Important Notes

This is not an official release from FunAudioLLM, ModelScope, or k2-fsa.
The INT4 weights were generated locally from the fp32 ONNX package distributed in the k2-fsa/sherpa-onnx ASR model release assets.
The original upstream model card currently does not declare clear Hugging Face YAML license metadata. Please verify the upstream usage terms before any redistribution or commercial use.

Source and Lineage

Upstream model: FunAudioLLM/Fun-ASR-Nano-2512
- https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512
ONNX export lineage referenced by upstream sherpa-onnx package:
- https://www.modelscope.cn/models/zengshuishui/FunASR-nano-onnx/files
- https://github.com/Wasser1462/FunASR-nano-onnx
fp32 package used as quantization source:
- sherpa-onnx-funasr-nano-2025-12-30.tar.bz2
- from https://github.com/k2-fsa/sherpa-onnx/releases/tag/asr-models

Files

encoder_adaptor.int4.onnx
embedding.int4.onnx
llm.int4.onnx
Qwen3-0.6B/
- tokenizer.json
- merges.txt
- vocab.json

Quantization Method

Quantizer: onnxruntime.quantization.matmul_nbits_quantizer.MatMulNBitsQuantizer
Quantization type: weight-only INT4
Scope: MatMul weights
Output format: single-file ONNX artifacts compatible with local sherpa-onnx loading

Local Validation

This INT4 variant was validated locally on Windows with CUDA-enabled sherpa-onnx.

Reference smoke test result on rag_math.wav:

fp32: 对微分形式的积分是微分几何中的基本概念。
int8: 对微分形式的积分是微分几何中的基本概念。
int4: 对微分形式的积分是微分几何中的基本概念。

Example Usage

python

import sherpa_onnx

recognizer = sherpa_onnx.OfflineRecognizer.from_funasr_nano(
    encoder_adaptor="encoder_adaptor.int4.onnx",
    embedding="embedding.int4.onnx",
    llm="llm.int4.onnx",
    tokenizer="Qwen3-0.6B",
    provider="cuda",
    num_threads=1,
)

Compatibility

Intended for sherpa-onnx offline inference
Tested locally with sherpa-onnx==1.12.39+cuda12.cudnn9
Tested locally with onnxruntime-gpu==1.24.4

Repository Purpose

This repository is intended as a convenient packaged INT4 deployment artifact for local or private inference workflows.

foryoung365/Fun-ASR-Nano-2512-int4-onnx