SenseVoice.cpp Jetson Nano Binaries
SenseVoice.cpp is a high-performance, open-source C++ speech-to-text implementation aimed at edge devices. It leverages the
GGML inference framework and supports multiple backends, including CUDA for GPU acceleration.
This repository hosts prebuilt binaries optimized for NVIDIA Jetson Nano , so you can skip the build step and start transcribing right away.
✨ Key Features
Multi-language ASR : Supports Chinese (Mandarin), Cantonese, English, Japanese, and Korean.
Low latency : Efficient inference with optional flash-attn .
Quantization : Q3, Q4, Q5, Q6, Q8 quantized models to reduce memory footprint.
Flexible backends :
CPU (all platforms)
CUDA (NVIDIA GPUs)
BLAS, Metal, Vulkan (upstream)
Voice Activity Detection (VAD) : Built-in silence-based VAD parameters.
Inverse Text Normalization (ITN) : Optionally output punctuation and formatted text.
For full feature details (streaming mode, extra backends), see the upstream documentation . 📁 Deliverable Directory Structure
1 project-root/
2 ├── bin/ # Executables
3 │ ├── sense-voice-main # Main ASR program
4 │ ├── sense-voice-quantize # Model quantization utility
5 │ └── sense-voice-zcr-main # Zero-Crossing Rate detection example
6 └── lib/ # Libraries
7 ├── libcommon.a # Common static library
8 ├── libggml-base.so # GGML base operations
9 ├── libggml-cpu.so # GGML CPU support
10 ├── libggml-cuda.so # GGML CUDA support
11 ├── libggml.so # GGML core
12 └── libsense-voice-core.a # SenseVoice core
bin/ : Standalone executables for Jetson Nano.lib/ : Static (.a) and shared (.so) libraries required at runtime.
🚀 Quick Deployment
Follow these steps to deploy and run on Ubuntu-based distributions (e.g., JetPack 4.5.1 on Jetson Nano):
1. Clone the Repo with Git LFS Support
If you haven’t installed Git LFS yet, do so and initialize:
1 # Install Git LFS
2 curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
3 sudo apt-get install git-lfs
4 # Initialize in your repo
5 git lfs install
Clone the repository:
1 git clone https://huggingface.co/ < YOUR_USERNAME > /sensevoice-jetson-nano.git
2 cd sensevoice-jetson-nano
3 git lfs pull
2. Track Large Binary Files with Git LFS
Ensure large files (shared libraries) use LFS to avoid push errors:
1 git lfs track "lib/*.so"
2 git add .gitattributes
3. Uploading New Binaries
When you update or add new .so files in lib/, commit and push as usual:
1 git add lib/*.so
2 git commit -m "Add updated shared libraries via LFS"
3 git push
4. Make Binaries Executable
5. Install Shared Libraries System-wide
1 sudo mkdir -p /usr/local/lib/sensevoice
2 sudo cp lib/*.so /usr/local/lib/sensevoice/
3 echo "/usr/local/lib/sensevoice" | sudo tee /etc/ld.so.conf.d/sensevoice.conf
4 sudo ldconfig
Alternatively, set LD_LIBRARY_PATH locally:
export LD_LIBRARY_PATH="$PWD/lib:$LD_LIBRARY_PATH"6. Model Setup
Download or convert a GGUF model (e.g., sense-voice-small-q4_k.gguf):
1 # From Hugging Face
2 git clone https://huggingface.co/lovemefan/sense-voice-gguf.git models
7. Run Examples
Speech-to-Text (non-streaming)
1 bin/sense-voice-main \
2 -m models/sense-voice-small-q4_k.gguf \
3 -f input.wav \
4 -t 4 \
5 -l zh \
6 --use-itn \
7 --flash-attn
Options :
-t N / --threads N: Number of decode threads (default: 4)-l LANG / --language LANG: auto, zh, en, yue, ja, ko--min_speech_duration_ms, --max_speech_duration_ms: VAD thresholds--no-gpu (-ng): Disable GPU--use-itn (-itn): Enable inverse text normalization--flash-attn (-fa): Enable Flash Attention decoder
Quantization Utility
1 bin/sense-voice-quantize \
2 --input models/sense-voice-small.bin \
3 --output models/sense-voice-small-q4_k.gguf \
4 --type q4_k
Supported quant types: q3, q4_k, q4_0, q5_0, q6_k, q8.
Zero-Crossing Rate Demo
bin/sense-voice-zcr-main input.wavFollow these steps to deploy and run on Ubuntu-based distributions (e.g., JetPack 4.5.1 on Jetson Nano):
1. Clone the Repo
1 git lfs install
2 git clone https://huggingface.co/ < YOUR_USERNAME > /sensevoice-jetson-nano.git
3 cd sensevoice-jetson-nano
4 git pull
2. Make Binaries Executable
3. Install Shared Libraries System-wide
1 sudo mkdir -p /usr/local/lib/sensevoice
2 sudo cp lib/*.so /usr/local/lib/sensevoice/
3 echo "/usr/local/lib/sensevoice" | sudo tee /etc/ld.so.conf.d/sensevoice.conf
4 sudo ldconfig
Alternatively, set LD_LIBRARY_PATH locally:
export LD_LIBRARY_PATH="$PWD/lib:$LD_LIBRARY_PATH"4. Model Setup
Download or convert a GGUF model (e.g., sense-voice-small-q4_k.gguf):
1 # From Hugging Face
2 git clone https://huggingface.co/lovemefan/sense-voice-gguf.git models
5. Run Examples
Speech-to-Text (non-streaming)
1 bin/sense-voice-main \
2 -m models/sense-voice-small-q4_k.gguf \
3 -f input.wav \
4 -t 4 \
5 -l zh \
6 --use-itn \
7 --flash-attn
Options :
-t N / --threads N: Number of decode threads (default: 4)-l LANG / --language LANG: auto, zh, en, yue, ja, ko--min_speech_duration_ms, --max_speech_duration_ms: VAD thresholds--no-gpu (-ng): Disable GPU--use-itn (-itn): Enable inverse text normalization--flash-attn (-fa): Enable Flash Attention decoder
Quantization Utility
1 bin/sense-voice-quantize \
2 --input models/sense-voice-small.bin \
3 --output models/sense-voice-small-q4_k.gguf \
4 --type q4_k
Supported quant types: q3, q4_k, q4_0, q5_0, q6_k, q8.
Zero-Crossing Rate Demo
bin/sense-voice-zcr-main input.wavFor streaming ASR or advanced examples, please refer to upstream's sense-voice-stream in the original repo.
🛠 Compatibility
Hardware : NVIDIA Jetson NanoOS : Ubuntu 18.04 / JetPack 4.5.1CUDA : 10.2C++ : C++17
📜 License
MIT License — see
LICENSE for details.
For comprehensive build instructions, extra examples, and advanced backend support, visit the
official SenseVoice.cpp documentation . Happy prototyping! 🎙️💕
