SenseVoiceSmall-RKNN2

• Inference speed (RKNN2): About 20x real-time on a single NPU core of RK3588 (processing 20 seconds of audio per second), approximately 6 times faster than the official whisper model provided in the rknn-model-zoo.
• Memory usage (RKNN2): About 1.1GB

Usage

1. Clone the project to your local machine
2. Install dependencies

1. Copy librknnt.so to /usr/lib/

1. Run

RKNN Model Conversion

1. Download or convert the ONNX model

1. Convert to RKNN model

Known Issues

• When using fp16 inference with RKNN2, overflow may occur, resulting in inf values. You can try modifying the scaling ratio of the input data to resolve this.
  Set SPEECH_SCALE to a smaller value in sensevoice_rknn.py.

References

• FunAudioLLM/SenseVoiceSmall
• lovemefan/SenseVoice-python

FastAPI Transcription Server

Running the Server

1. Ensure all dependencies for sensevoice_rknn.py and the server are installed. This includes fastapi and uvicorn:
   pip install fastapi uvicorn
2. Place the required model files (*.rknn, *.onnx, spm.model) in the same directory as server.py.
3. Run the server:
   python server.py
   The server will start on http://0.0.0.0:8000 by default.

API Endpoint: /transcribe

• Method: POST
• Description: Transcribes the audio file specified in the request.
• Request Body: JSON object with the following fields:
  • audio_file_path (string, required): The absolute path to the WAV audio file on the server's filesystem.
  • language (string, optional, default: "en"): The language code for transcription. Supported codes depend on the model (e.g., "en", "zh", "ja", "ko").
  • use_itn (boolean, optional, default: false): Whether to apply Inverse Text Normalization to the transcription output.
• Example Request (curl):
  bash

  1curl -X POST -H "Content-Type: application/json" \ 2-d '{"audio_file_path": "/path/to/your/audio.wav", "language": "en", "use_itn": false}' \ 3http://0.0.0.0:8000/transcribe
• Response Body: JSON object with the following fields:
  • full_transcription (string): The complete transcribed text, including any special tokens from the model.
  • segments (list of objects): A list where each object represents a transcribed audio segment and contains:
    • start_time_s (float): Start time of the segment in seconds.
    • end_time_s (float): End time of the segment in seconds.
    • text (string): Transcribed text for the segment.
• Example Response:
  json

  1{ 2 "full_transcription": "<|en|><|HAPPY|><|Speech|><|woitn|>the stale smell of old beer lingers <|en|><|NEUTRAL|><|Speech|><|woitn|>it takes heat to bring out the odor but <|en|><|HAPPY|><|Speech|><|woitn|>a cold dip restores health and zest a salt pickle tastes fine with ham tacos al pastor are my favorite <|en|><|EMO_UNKNOWN|><|Speech|><|woitn|>a zestful food is the hot cross bun", 3 "segments": [ 4 { 5 "start_time_s": 1.01, 6 "end_time_s": 3.93, 7 "text": "<|en|><|HAPPY|><|Speech|><|woitn|>the stale smell of old beer lingers" 8 }, 9 { 10 "start_time_s": 4.21, 11 "end_time_s": 6.59, 12 "text": "<|en|><|NEUTRAL|><|Speech|><|woitn|>it takes heat to bring out the odor but" 13 }, 14 { 15 "start_time_s": 6.87, 16 "end_time_s": 14.68, 17 "text": "<|en|><|HAPPY|><|Speech|><|woitn|>a cold dip restores health and zest a salt pickle tastes fine with ham tacos al pastor are my favorite" 18 }, 19 { 20 "start_time_s": 14.96, 21 "end_time_s": 18.34, 22 "text": "<|en|><|EMO_UNKNOWN|><|Speech|><|woitn|>a zestful food is the hot cross bun" 23 } 24 ] 25}