Qwen3-Omni-30B-A3B-bf16
Speech-to-speech fine-tune of Qwen3-Omni-30B-A3B, merged, in bf16.
Fine-tuned from
Qwen/Qwen3-Omni-30B-A3B-Instruct on spoken dialogue, then merged back into the full model at full precision.
What was actually trained
Qwen3-Omni ships the codec decoder (code2wav) but no codec encoder or quantizer,
so target speech cannot be turned back into the discrete Talker labels a speech-out
loss would need. This run therefore trains the Thinker -- the half that listens and
decides what to say -- inside the full speech-in/speech-out pipeline, with the Talker
and code2wav frozen and bit-identical to the base checkpoint. That is also the only
half DynQuant quantizes here: all 96 expert banks are Thinker banks.
| |
|---|
| Trained stage | thinker (31.72 B params) |
| Frozen stages | talker, code2wav (3.54 B params, bit-identical to base) |
| Method | QLoRA, rank 16, on attention and dense-MLP projections |
| Steps | 500 optimizer steps at effective batch 16 (2 x GPU) |
| Audio seen | 10.0 hours |
| Wall clock | 30 min |
Size
| Artifact | On disk |
|---|
| QLoRA adapter | 0.10 GiB (112,207,855 B) |
| bf16 merge | 65.69 GiB (70,537,517,111 B) |
| DynQuant 4-bit | 21.38 GiB (22,953,262,961 B) |
| DynQuant 3-bit | 17.68 GiB (18,988,492,249 B) |
Honest limits
There is no accuracy metric. No benchmark, no A/B against the base checkpoint, no
listening study -- that was accepted going in. What did run is a smoke probe:
4 held-out clips per arm, seed pinned, each answered end to end from audio
in to audio out, with the loaded module tree counted afterwards so that a directory
which silently loaded unquantized could not pass as a working quantized model.
| Arm | Packed modules | Clips | Silent | Median new tokens | Speech emitted |
|---|
| bf16 merge | none (bf16) | 4 | 0 | 39 | 45.4 s |
| DynQuant 4-bit | 602 | 4 | 0 | 21 | 37.6 s |
| DynQuant 3-bit | 602 | 4 | 0 | 28 | 30.5 s |
That checks for the destroyed-model failure -- silence, fluent nonsense, a load path
that quietly fell back -- and all three arms pass it. It is not a measure of quality.
A handful of replies read by eye cannot separate "as good as bf16" from "noticeably
worse but still coherent", and nothing in this campaign does.
This is the unquantized side, so it carries only the limits of the fine-tune
itself: one epoch over spoken dialogue, and a Talker that was never trained and so
still speaks in the base checkpoint's manner.
Use
1import torch, transformers
2
3model = transformers.Qwen3OmniMoeForConditionalGeneration.from_pretrained(
4 "VikramPal/Qwen3-Omni-30B-A3B-bf16", dtype=torch.bfloat16, device_map="auto"
5)
6processor = transformers.AutoProcessor.from_pretrained("VikramPal/Qwen3-Omni-30B-A3B-bf16")
7
8conversation = [
9 {"role": "system", "content": [{"type": "text", "text": "You are a helpful voice assistant. Reply naturally and briefly, as if speaking aloud."}]},
10 {"role": "user", "content": [{"type": "audio", "audio": wave_16khz}]},
11]
12inputs = processor.apply_chat_template(
13 conversation, add_generation_prompt=True, tokenize=True,
14 return_dict=True, return_tensors="pt", sampling_rate=16000,
15).to(model.device).to(model.dtype)
16
17text_ids, audio = model.generate(**inputs, speaker="Ethan", return_audio=True)
18# audio comes back at 24 kHz
Provenance
Produced with
DynQuant v0.5.0. The
allocation is the signal map the fine-tune itself recorded -- per-module gradient
variance and activation RMS -- consumed by a role-aware ROI allocator. It was computed
over the Thinker and applied to the full model, which is why the other two stages are
still bf16.
