Views
No views yet
.pte files
for Voxtral-Mini-4B-Realtime-2602,
Mistral's ~4B parameter streaming speech-to-text model.| File | Description |
|---|---|
model.pte | Exported model (audio encoder + text decoder + token embedding) |
aoti_cuda_blob.ptd | AOTInductor CUDA delegate data (compiled kernels + weights) |
preprocessor.pte | Mel spectrogram preprocessor for audio-to-mel conversion |
Backend: cuda (AOTInductor)
Dtype: bf16
Mode: streaming
Encoder quant: 4w (group_size=32, tile_packed_to_4d)
Decoder quant: 4w (group_size=32, tile_packed_to_4d)
Embedding quant: 8w
Max seq len: 4096
Delay tokens: 6 (480ms)tekken.json tokenizer from the
original modelmake voxtral_realtime-cuda # Linux1cmake --workflow --preset llm-release-cuda
2Push-Location examples/models/voxtral_realtime
3cmake --workflow --preset voxtral-realtime-cuda
4Pop-Location1cmake-out/examples/models/voxtral_realtime/voxtral_realtime_runner \
2 --model_path model.pte \
3 --data_path aoti_cuda_blob.ptd \
4 --tokenizer_path tekken.json \
5 --preprocessor_path preprocessor.pte \
6 --audio_path input.wav \
7 --streaming1.\cmake-out\examples\models\voxtral_realtime\Release\voxtral_realtime_runner.exe `
2 --model_path model.pte `
3 --data_path aoti_cuda_blob.ptd `
4 --tokenizer_path tekken.json `
5 --preprocessor_path preprocessor.pte `
6 --audio_path input.wav `
7 --streaming1ffmpeg -f alsa -i default -ar 16000 -ac 1 -f f32le -nostats -loglevel error pipe:1 | \
2 cmake-out/examples/models/voxtral_realtime/voxtral_realtime_runner \
3 --model_path model.pte \
4 --data_path aoti_cuda_blob.ptd \
5 --tokenizer_path tekken.json \
6 --preprocessor_path preprocessor.pte \
7 --micffmpeg -i input.mp3 -ar 16000 -ac 1 output.wav--data_path flag pointing to aoti_cuda_blob.ptd is required for CUDA backends.--delay-tokens 6. Other values may degrade accuracy.