This repository contains GGUF weights for Voxtral Realtime 4B, a high-performance speech recognition (STT) model optimized for low-latency, real-time inference.
These weights are converted from the original
mistralai/Voxtral-Mini-4B-Realtime-2602 model.
Voxtral is designed to process streaming audio with minimal delay, making it ideal for live transcription, voice assistants, and interactive applications.
For the fastest inference performance on CPU and GPU, use the voxtral.cpp (
https://github.com/andrijdavid/voxtral.cpp) repository. It provides a lightweight C++ implementation based on ggml.
1 git clone https://github.com/andrijdavid/voxtral.cpp
2 cd voxtral.cpp
3 cmake -B build -DCMAKE_BUILD_TYPE=Release
4 cmake --build build -j
-
Run Transcription
Prepare a 16kHz mono WAV file and run inference:
1 ./build/voxtral \
2 --model models/voxtral/Q4_0.gguf \
3 --audio input.wav \
4 --threads 8
For more advanced usage, including streaming examples and conversion scripts, please visit the voxtral.cpp GitHub repository (
https://github.com/andrijdavid/voxtral.cpp).
