GGUF conversions of the official OmniVoice checkpoint for synthesize.cpp.

Ported from k2-fsa/OmniVoice revision 468e927ba3716cd8dd86421148dfb3046e9f9d7b and validated on 2026-08-07 against the pinned upstream PyTorch implementation.

A self-contained OmniVoice inference package converted from the flagship k2-fsa/OmniVoice checkpoint. It is a non-autoregressive mask-predict diffusion language model: a bidirectional Qwen3-0.6B backbone refines a fixed-length canvas of 8 acoustic codebooks x 1025-entry vocabulary over 32 parallel denoising steps with classifier-free guidance, and the Higgs Audio V2 codec's DAC-style decoder turns the finished token grid into 24 kHz audio. Voice arrives through Reference Audio cloning, Description Text voice design, or an unnamed auto-voice default with no speaker conditioning, whose speaker is emergent rather than pinned by the seed alone. The package carries its own byte-level BPE text frontend, so raw UTF-8 text goes in and audio comes out.

One hazard to know before your first request, and nothing in the runtime warns about it. The canvas length is estimated from the text and then divided by the speaking rate. Below roughly 37-40 frames (about 1.5 seconds) there are too few positions to place the text, and the output degenerates into a near-DC sub-50 Hz rumble instead of speech. Two ordinary requests reach that: a very short text (--text "Hi." resolves to a 22-frame canvas), and a high --rate on a short one -- the declared speaking_rate_range goes to 2.0 and is accepted silently, but upstream's own demo UI caps speed at 1.5, and the pinned upstream model is already degenerate at rate 1.50 on a 32-character sentence. This is upstream's behavior, faithfully reproduced, not a defect in this port: on the affected validation case this package matches the reference implementation's waveform at Pearson r = 0.999670, the tightest agreement of any case measured, and the reference's own audio for it is equally unintelligible. Rate 2.0 on a long text is fine; the variable is the resolved canvas length, not the rate.

All profiles use the same OmniVoice architecture and public synthesize.cpp API. The profile name describes a versioned storage policy, not the language or Execution Backend.

A profile name here says which half of the model it quantizes. This family has two: a Qwen3-based generator that paints the token canvas, and the Higgs Audio V2 codec that turns the canvas into audio. F16 and Q8 quantize the generator half and leave all 486 codec tensors bit-identical to F32, which is why all three reproduce the Reference Audio cloning path's token grid exactly — and why a quantized generator re-draws the other grid, the greedy decode canvas, on purpose. A profile that quantized the codec half instead would carry a _CODEC qualifier in its name; none is published, because every one produced so far breaks the cloning grid and fails that exact-token gate.

No profile here is faster than F32, and none claims to be. Measured against F32 on this project's CUDA reference host: Q8 -0.4%, F16 +2.7%; on CPU, Q8 +1.5% and F16 -1.2%, where a positive number is the slower one. The generator is compute-bound at roughly 205 multiply-accumulates per weight byte, so a narrower weight does not relieve the bottleneck. Two of those four figures land marginally under F32, by about one part in a hundred; neither is a reason to pick a profile. What a smaller profile buys is disk, accelerator memory and load time.

These files are not GGUF tensor storage; they are declared resources that must travel with the package. Verify each digest before relying on its contents.

8 graph stages were replayed for 20 cases on DGX Spark CPU, NVIDIA GB10 CUDA 13.3. Duration structure was exact in every case. Both halves now run on CUDA, and only one of them did through Plan 4. The codec's decode graph (the RVQ dequantizer, the acoustic decoder, and the final projection -- 152 tensors) moved first: swept over all twenty golden cases, every one of its 8,440 nodes left the CPU while every one of the generator's 880,032 nodes stayed on it, held there by docs/backends.md's discrete-outputs rule. On 2026-08-08 the generator earned that rule's one narrow exception -- its per-step token choice picks the content of a canvas whose shape deterministic host arithmetic fixed before the first forward runs, and grid size was then measured to match the CPU oracle on all seventeen greedy cases -- so it received its own accelerator-resident weight twin and now moves too. 2,421.04 of the model's 3,042.2 MiB are mirrored, and the suite's longest case falls from 122.61 s to 4.84 s, 0.168x real time. (Corrected 2026-08-10: this read 5.481 s / 0.1906x, which was measured before the two host-side optimizations that followed it and so understated the shipped package. Re-measured on the published F32 GGUF, five runs, 4.844-4.866 s.) Committed token content consequently diverges from the CPU baseline on most cases: a different valid realization inside an unchanged canvas, audited by ear before the move was accepted, not a regression. The cloning path's own RVQ token grids stay byte-exact across the move.

Waveform cosine is measured for completeness; it is not this family's real guarantee and is not enforced as a tolerance the way it is for Kokoro or VITS. What the Port Validation Suite reports instead is exact token identity, and it is two claims with two different scopes. First: the greedy decode loop's full 8 x T unmasking grid matches the oracle's byte-for-byte on 17 of 17 golden cases for F32 on CPU -- the reference configuration, and the only one that certifies it. A quantized generator re-draws that grid by design (Q8 commits a different token at 95.83% of positions), and so does the CUDA backend, whose TF32 arithmetic moves the per-step argmax; both change what the canvas contains and never how large it is, both were accepted by listening rather than by a token gate, and their flip counts are recorded as data, not gated. Second, and this is the claim that is gated unconditionally: the Reference Audio cloning path's own RVQ encode grid matches the oracle on both clone cases -- 8 codebooks x 351 frames, 2,808 of 2,808 tokens exact -- and it holds on every profile listed above, because F16 and Q8 quantize the generator half only and leave every one of the 486 codec.* tensors bit-identical to the F32 package. A profile that flips even one of those 2,808 tokens is not shipped, and that rule is what decides which profiles are listed above. Every codec-half profile produced so far fails it and none is listed: F16_CODEC mismatched 103 of 2,808 tokens (3.7%), and Q8_CODEC_MIXED mismatched 1,023 (36.4%) while it still block-quantized convolution kernels, 98 (3.49%) since the conv-exempt codec policy of 2026-08-09 stopped it doing so. None is a knife-edge margin call, so none ships and no perceptual claim substitutes for the gate. The two waveform-cosine columns are measured against the PyTorch oracle and are reported for F32 alone. They are not withheld for the quantized profiles; they do not apply to them. Because a quantized generator re-draws the token grid rather than perturbing F32's, its output is a different valid realization of the same request, and a waveform distance to the oracle would score that difference as error. The last column is the measurement that does separate them: long-term average spectrum distance from F32's own CPU render, the speaker-identity proxy this family uses, worst case over the seventeen greedy validation cases. Roughly 3 dB is the line a listener has consistently called "same person", calibrated on fourteen answers across two audits with no exceptions. Every deep generator probe gates on cosine rather than a sample-wise bound because pre-norm outlier channels reach a max-abs of 0.08 while cosine still holds at eight nines; a sample-wise threshold there would reject a provably correct port.

Quality evaluation has not been run. These results establish that the port, the unconditioned default-Voice request path, deterministic request path, and CPU/CUDA execution work. They do not claim perceptual equivalence, naturalness, intelligibility, or speaker similarity.

A listening audit found no obvious regression. One maintainer compared a small set against the reference and reported nothing audible. That is release evidence, not a measurement: no rated comparison, no panel, no score, and it does not change the validation level. It says a defect large enough to hear was not found in what was heard.

This package exposes no preset speaker catalog. Its package default has no fixed identity and no speaker conditioning: with no Voice Profile supplied, which speaker a request produces is emergent, and the synthesis seed alone does not pin it. Reproducing a speaker needs the same seed and the same Execution Backend, package, and decode settings; switching between the CPU and CUDA backends can produce a different speaker for an otherwise identical request. Measured on this package's own validation cases, one case in seventeen changes speaker between the CPU and CUDA backends, and a listener judged that pair to be two different speakers of equal audio quality. Callers who want a stable identity should instead build a Voice Profile from Reference Audio (voice cloning) or Description Text (voice design) and pass it to the request. It produces 24000 Hz mono F32 audio.

This package accepts raw UTF-8 text through the built-in synthesize.qwen_bpe frontend. The frontend tokenizes text directly: no grapheme-to-phoneme conversion happens or is needed, and the runtime does not silently invoke eSpeak or download a frontend.

Build synthesize.cpp and synthesize a deterministic request:

The same local GGUF can be loaded through the public C ABI and wrapped by C++, Rust, or Python. Model loading never contacts Hugging Face.

This package carries three separate upstream grants, and only two of them touch anything shipped here. The generator (LM) weights are CC-BY-NC, with no version stated by upstream -- the upstream model card carries no license: frontmatter key at all; the only statement is this prose, quoted verbatim from the pinned upstream model page:

Upstream names no CC-BY-NC version anywhere in that statement or elsewhere on the card, so this card says exactly that rather than inventing one (never cc-by-nc-4.0: no version is asserted upstream and none is asserted here). These weights are not licensed for commercial use. The restriction traces to the training data the statement itself names -- Emilia, a large-scale multilingual speech corpus distributed under its own non-commercial terms -- so it is a property of what the generator was trained on, not a choice this project made.

The codec (Higgs Audio V2) weights carry a second, separate license: the Boson Higgs Audio 2 Community License. The weights repository bundles it as audio_tokenizer/LICENSE, and this package republishes it beside the weights as LICENSE-higgs-audio-2.txt (listed below).

Two licence files travel with these weights, not one. The Boson agreement defines its own name to include Meta's -- "Agreement" means the terms and conditions ... set forth herein and the Meta License Agreement -- and section 1.b.i(A) then requires "a copy of this Agreement and the ... Meta License's Llama 3 agreement" to accompany any redistribution of the Higgs Materials. So LICENSE-meta-llama-3.txt ships here too. Corrected 2026-08-10: this package originally shipped only the Boson text while this card said it did so "per that agreement's own redistribution terms", which was a claim of compliance it did not meet.

You must download both together with the GGUF -- the usage command below does; the runtime does not enforce the pairing, and nothing in the GGUF's metadata declares either file, so a download of the weights alone leaves you holding the Higgs Materials with none of the agreement that governs them. The Boson agreement is itself derived from the Meta Llama 3 Community License and requires dual attribution in any redistribution, which this card carries forward verbatim from the sidecar text:

That agreement also requires a redistributor to display the following notice, which it specifies verbatim and which this card carries for that purpose:

Two further obligations, stated in the agreement's own terms rather than paraphrased. Corrected 2026-08-08: this card previously said "caps commercial use at 100,000 monthly active users", which was wrong twice over and wrong in the reader's favour -- the agreement says annual, and it does not cap anything.

Above 100,000 annual active users in the preceding calendar year, section 2 does not impose a ceiling: it withdraws authorisation. You "must request an expanded license from Boson AI, which Boson AI may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under this Agreement unless or until Boson AI otherwise expressly grants you such rights."

Section 1.b.i(v) forbids using the Higgs Materials "or any output or results of the Higgs Materials to improve any other large language model (excluding Boson Higgs Audio 2 or derivative works thereof)". Note the scope: other large language models, with Boson's own carved out -- not, as this card previously said, all models.

Both are independent of, and in addition to, the generator's own CC-BY-NC restriction above.

Apache-2.0 covers only the upstream GitHub source code (k2-fsa/OmniVoice) and nothing produced by this project: no weight file, converted artifact, or GGUF in this package may be described as apache-2.0.

Finally, the upstream card carries a use disclaimer this package repeats verbatim rather than paraphrases:

This is a Restricted Model Package (ADR 0018), not a Published Model Package, and must never be described as one: it carries the full upstream terms above rather than an unrestricted embeddable-in-other-programs grant. Both upstream licenses permit non-commercial redistribution with attribution, and upstream itself distributes the weights openly on Hugging Face -- the basis for republishing this converted GGUF under exactly those terms, not a broadening of them. Publication itself remains a separate act requiring its own per-act confirmation.

Only en, zh, and ja are declared as this family's Language Capability Catalog. The checkpoint claims 600+ languages through its training data and prompt format, and an untested language tag may well produce recognizable speech, but no language beyond these three has its own validation case, so none beyond these three is advertised.

 

 

 

 

 

OmniVoice is a massively multilingual zero-shot text-to-speech (TTS) model supporting over 600 languages. Built on a novel diffusion language model-style architecture, it delivers high-quality speech with superior inference speed, supporting voice cloning and voice design.

To get started, install the omnivoice library:

Step 1: Install PyTorch

Step 2: Install OmniVoice

You can use OmniVoice for zero-shot voice cloning as follows:

For more generation modes (e.g., voice design), functions (e.g., non-verbal symbols, pronunciation correction) and comprehensive usage instructions, see our GitHub Repository.

You can directly discuss on GitHub Issues.

You can also scan the QR code to join our wechat group or follow our wechat official account.

Our code is released under the Apache 2.0 License. The pre-trained model is licensed under the CC-BY-NC due to constraints from its training data (e.g., Emilia).

Users are strictly prohibited from using this model for unauthorized voice cloning, voice impersonation, fraud, scams, or any other illegal or unethical activities. All users shall ensure full compliance with applicable local laws, regulations, and ethical standards. The developers assume no liability for any misuse of this model and advocate for responsible AI development and use, encouraging the community to uphold safety and ethical principles in AI research and applications.