You can deploy this model and start earning money today!

dbrains/
Qwen3-TTS-Tokenizer-12Hz-Q8mixed-GGUF

Name: Qwen3-TTS-Tokenizer-12Hz-Q8mixed-GGUF
Availability: InStock

khimaros/Qwen3-TTS-Tokenizer-12Hz-GGUF

quantized

apache-2.0

Views

No views yet

Qwen3-TTS-Tokenizer-12Hz (Q8-mixed GGUF)

A pre-quantized variant of the Qwen3-TTS vocoder GGUF for use with qwen3-tts.cpp.

Identical to the F16 source for all conv kernels, biases, norms, and codebooks. The pre-transformer attention + FFN weights, the upsample point-wise 1×1 conv weights, and the pre-transformer/VQ projection mat-muls are pre-quantized to Q8_0 — exactly the same subset the qwen3-tts.cpp runtime would otherwise quantize on every cold start.

Shipping the conversion offline cuts ~180 ms off cold-start vocoder load (the F16 → F32 → Q8_0 CPU step) at no change to what gets uploaded to the GPU.

Numbers (RTX 3060, qwen3-tts.cpp, ~32 s paragraph)

	F16 source + runtime quant	This GGUF
Vocoder load	298 ms	98 ms
Cold time-to-first-audio (end-to-end)	1694 ms	1492 ms
File size	325 MiB	283 MiB
GPU VRAM upload	175.9 MiB Q8	175.9 MiB Q8 (identical)

Audio quality matches the runtime-quant case to within ±0.02 dB on every band metric across a broad prompt set (voice clone + voice design + edge cases, multiple seeds). Sample-level WAV bytes differ because Python gguf.quants.quantize and C++ ggml_quantize_chunk round slightly differently; the resulting diff sits at −47 dB SNR vs signal — perceptually invisible.

Parent model

Derived from khimaros/Qwen3-TTS-Tokenizer-12Hz-GGUF (F16). All weights are reused as-is; only the dtype of the subset listed above changes.

How it was produced

python scripts/convert_vocoder_to_q8_mixed.py \
    Qwen3-TTS-Tokenizer-12Hz-F16.gguf \
    Qwen3-TTS-Tokenizer-12Hz-Q8mixed-Q8_0.gguf

Script at qwen3-tts.cpp/scripts/convert_vocoder_to_q8_mixed.py. The quantization selector mirrors src/audio_tokenizer_decoder.cpp::should_quantize_q8 including the ne[0] % 32 == 0 gate, so the four vq_first / vq_rest input/output projections (whose inner dim is 1) stay F16 to match runtime behaviour.

62 of 448 tensors are quantized; the remaining 386 pass through unchanged.

Usage

With the qwen3-tts.cpp server:

qwen3-tts-server \
  --hf-repo  khimaros/Qwen3-TTS-12Hz-1.7B-VoiceDesign-GGUF:Q8_0 \
  --hf-repo-v dbrains/Qwen3-TTS-Tokenizer-12Hz-Q8mixed-GGUF:Q8_0 \
  ...

Or with a local path:

qwen3-tts-server \
  -v /path/to/Qwen3-TTS-Tokenizer-12Hz-Q8mixed-Q8_0.gguf \
  ...

License

Inherits from the parent (Apache-2.0).