VibeVoice-ASR is Microsoft's 7B-parameter speech-LLM capable of transcribing up to 60 minutes of audio in a single pass. Unlike most ASR models it outputs structured JSON containing who (speaker diarization), when (timestamps), and what (content) simultaneously — with support for customised hotwords and 50+ languages.
60-minute long-form audio in a single forward pass (streaming segmentation for >60 s)
Built-in speaker diarization — Speaker IDs in every output segment
Word-level timestamps — Start time / End time per segment
Hotword / context injection — pass terms or metadata via --context
50+ languages with automatic language detection
MIT licence
Files
File
Size
Notes
vibevoice-asr-q4_k.gguf
~5 GB
Q4_K_M — smallest
vibevoice-asr-q8_0.gguf
9.45 GB
Q8_0 — near-F16 quality, recommended if Q4_K sounds off
<|im_start|>system
You are a helpful assistant that transcribes audio input into text output in JSON format.
<|im_end|>
<|im_start|>user
<|speech_start|><|speech_pad|>×N<|speech_end|>
This is a X.XX seconds audio, please transcribe it with these keys: Start time, End time, Speaker ID, Content
<|im_end|>
<|im_start|>assistant
Where N = ceil(samples / 3200).
Audio preprocessing
Resample to 24 kHz mono
Normalise to −25 dBFS RMS: audio *= 10^(−25/20) / (rms + 1e-6), then clip if |audio| > 1
What's baked into the GGUF
All acoustic encoder weights (F16 for conv weights except depthwise which stays F32)
Both SpeechConnectors
Full Qwen2-7B LLM weights
Qwen2.5 tokenizer vocabulary (embedded as tokenizer.ggml.tokens)
Architecture hyperparameters as GGUF metadata keys
Implementation notes
The C++ runtime validates against the PyTorch reference at every pipeline boundary using tools/dump_reference.py --backend vibevoice:
Stage
Key
Notes
Audio normalisation
audio_norm
−25 dBFS RMS, clip guard
Acoustic encoder mean
at_enc_mean
(T', 64) — deterministic (mean, no noise)
Semantic encoder mean
st_enc_mean
(T', 128) — always deterministic
Acoustic connector
at_conn_out
(T', 3584)
Semantic connector
st_conn_out
(T', 3584)
Combined features
speech_features
elementwise sum of both connectors
Generated tokens
llm_argmax
greedy decode
Note on acoustic sampling: the Python model uses std_dist_type='gaussian' (adds per-batch noise during training/inference for robustness). The C++ runtime uses the VAE mean directly (deterministic), which is equivalent to dist_type='none' and gives reproducible, comparable output for diff-testing.
How this was made
microsoft/VibeVoice-ASR safetensors converted to GGUF F16 by models/convert-vibevoice-to-gguf.py. Tensors loaded in native dtype (BF16) to avoid OOM on the large embedding table; converted to F16/F32 per-tensor at write time.
Quantised variants produced by the CrispASR quantize binary (Q4_K_M).
Full pipeline implemented in src/vibevoice.{h,cpp}: two encoder graphs + two connector graphs + Qwen2 autoregressive decoder with F16 KV cache.
Upstream licence:mit. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
What was done here: format conversion and/or quantisation only (GGUF/GGML). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository. No training-content summary was found on the upstream model card at the time of writing; that documentation gap is upstream's and is not filled here.
Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.
