The offline Voxtral model is a Whisper-style audio encoder + Mistral-Small-24B decoder that transcribes by generating text tokens. Higher quality than the 3B Mini, at a larger size.

Quants

File	Approx size	Notes
`Q6_K`	~20 GB	highest quality here
`Q5_K`	~17 GB	high quality
`Q4_K_M`	~15 GB	recommended — best size/quality trade-off
`Q4_0`	~14 GB	smaller
`Q3_K`	~11 GB	smallest, some quality loss

Usage

bash

# build voxtral.cpp, then:
./build/voxtral --model Q4_K_M.gguf --audio clip.wav --gpu metal

16 kHz mono WAV; long audio is transcribed in 30 s windows. GPU auto-detected (--gpu metal|cuda|vulkan|none). Needs a GPU/RAM budget of ~15 GB+ for Q4_K_M.

License

Apache 2.0, inherited from the base model.

andrijdavid/Voxtral-Small-24B-2507-GGUF

Voxtral-Small-24B-2507-GGUF

Quants

Usage

License

andrijdavid/
Voxtral-Small-24B-2507-GGUF