Voxtral Mini is Mistral's 3B-parameter speech-LLM — an enhancement of Ministral 3B with state-of-the-art audio input capabilities while retaining best-in-class text performance. It excels at speech transcription, translation, and audio understanding.
8 languages (English, French, German, Spanish, Italian, Portuguese, Dutch, Hindi) with automatic language detection
Built-in audio Q&A and summarization — ask questions about audio content directly
Function calling from voice — trigger backend functions based on spoken intents
Long-form context — up to 30 minutes of audio for transcription, 40 minutes for understanding
Natively multilingual with state-of-the-art WER across the world's most widely used languages
Highly capable at text — retains the text understanding capabilities of its Ministral 3B backbone
Apache-2.0 licence
Files
File
Size
Notes
voxtral-mini-3b-2507-q4_k.gguf
2.5 GB
Q4_K — recommended default
voxtral-mini-3b-2507-q8_0.gguf
5.0 GB
Q8_0, near-lossless
Both quantisations produce the correct transcript on samples/jfk.wav:
And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.
The mel filterbank from WhisperFeatureExtractor and the Tekken tokenizer vocab are baked into the GGUF, so the C++ runtime computes everything natively — no Python/torch/librosa at inference time.
Measured on samples/jfk.wav (11 seconds), 4-core CPU:
Variant
Mel
Encoder
Prefill
Decode/tok
Total
F16 (8.8 GB)
264 ms
48.7 s
78.4 s
1134 ms
157 s
Q4_K (2.5 GB)
246 ms
32.7 s
30.8 s
242 ms
70 s
Q4_K gives a 2.2× speedup over F16 while producing identical transcripts. The 3B model is larger than the Qwen3-ASR 0.6B — for fastest CPU inference on short clips, Qwen3-ASR Q4_K (6.6s for 11s audio) is faster; Voxtral's advantage is the richer capabilities (audio understanding, function calling, text Q&A) and superior multilingual WER.
Architecture
Voxtral-Mini-3B is a three-module speech-LLM:
Component
Details
Audio encoder
32-layer Whisper-large-v3 encoder: d=1280, 20 heads, head_dim=64, FFN=5120, 128 mels, learned absolute positional embedding (1500, 1280). Conv1d front-end: conv1(128→1280, k=3, stride=1, pad=1) + GELU → conv2(1280→1280, k=3, stride=2, pad=1) + GELU. Note: conv1 stride is 1 (not 2 like standard Whisper), so only conv2 does temporal downsampling (2×). 3000 mel frames → 3000 → 1500 encoder frames.
Conv1 stride is 1 (Whisper uses stride 2). This means the conv front-end only does 2× temporal reduction (just conv2), not 4×. 3000 mel frames → 1500 encoder frames (vs Whisper's 750).
K-proj has no bias in the encoder's self-attention (Whisper quirk preserved from the Whisper-large-v3 weights).
The encoder output is not consumed by a Whisper decoder — it's fed through a 4-frame-stack projector into a general-purpose Llama 3 LLM that generates the transcript (or any other text response) autoregressively.
Implementation notes
The C++ runtime is verified against the PyTorch reference (bf16) at every architectural boundary:
Stage
Diff metric
Result
LLM forward (30 layers, text-only)
cosine sim at last position
0.999973, top-5 5/5 match
Audio encoder + projector (32 layers + stack-4)
per-row cosine sim vs proj2_out.npy
mean 0.998, min 0.870 (bf16 ref precision)
End-to-end transcription on jfk.wav
generated token sequence
Correct transcript
The 0.87 min cosine sim on the encoder is from the bf16 reference precision (7-bit mantissa) vs F16 GGUF weights (10-bit) with F32 compute in C++. An F32 reference would give tighter numbers — the end-to-end transcript is the real correctness test and it passes.
Bugs found during the port
ggml_conv_1d output layout: returns (OL, OC, N) not (OC, OL). Bias needs (1, OC, 1) reshape to broadcast over time+batch.
Post-conv transpose: ggml_conv_1d puts time on ne[0], but LayerNorm needs feature dim on ne[0]. Fixed by reshape+transpose to (d, T_enc).
Tekken vocab blob storage: gguf-py's add_array with Python int lists stores as INT32, corrupting the uint8 byte stream. Fixed by storing as a 1D F32 tensor.
How this was made
HF safetensors converted to GGUF F16 by models/convert-voxtral-to-gguf.py. All 765 tensors (762 model + mel_filters + mel_window + Tekken vocab blob) map cleanly.
Quantised variants produced by cohere-quantize with the Q4_0 fallback for 1280-wide audio encoder tensors (1280 % 256 ≠ 0 for Q4_K, same situation as Qwen3-ASR).
Inference implemented in src/voxtral.{h,cpp} (~1300 LOC): encoder and LLM each run as one ggml graph, with a persistent F16 KV cache (head_dim, max_ctx, n_kv_heads, n_layers) shared between prefill and per-token decode steps. Flash attention (ggml_flash_attn_ext) used on both prefill (F16 causal mask) and decode (no mask) paths.
Upstream licence:apache-2.0. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
What was done here: format conversion and/or quantisation only (GGUF/GGML). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository. No training-content summary was found on the upstream model card at the time of writing; that documentation gap is upstream's and is not filled here.
Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.
