[!IMPORTANT] This repository is a community MLX conversion of the official k2-fsa/OmniVoice Hugging Face release.

The official model and original model card remain the authoritative source for model capability, license, training, and citation details. This repository only adds Apple Silicon / MLX packaging and runtime notes.

MLX runtime and conversion scripts are maintained here:

GitHub: https://github.com/ailuntx/OmniVoice-MLX

Hugging Face target repo: https://huggingface.co/mlx-community/OmniVoice-4bit

Current local variant: int4 row-wise 4-bit. The OmniVoice MLX loader reads mlx_manifest.json and dequantizes weights at load time.

MLX Usage

bash

pip install "git+https://github.com/ailuntx/OmniVoice-MLX.git"
pip install mlx

omnivoice-infer-mlx \
  --model . \
  --text "Hello." \
  --instruct "female, british accent" \
  --output out.wav

The audio_tokenizer/ directory is included in this repository so the model can be loaded from a single local directory.

Use with OpenVox

OpenVox provides a graphical desktop interface for running compatible AI voice models locally on Apple Silicon without requiring command-line setup.

With OpenVox, OmniVoice can be used for:

Generating speech from text
Creating custom voices through natural-language voice descriptions
Using reference audio for supported voice-cloning workflows
Adjusting generation settings through a desktop interface
Creating and exporting long-form audio
Running compatible model workflows locally after the required files are downloaded
Exposing local voice-generation capabilities through API and MCP integrations

OpenVox is useful for users who want an integrated desktop workflow while retaining the option to use the underlying OmniVoice MLX runtime directly.

OpenVox is a third-party application and is not affiliated with or endorsed by the OmniVoice authors or k2-fsa.

OmniVoice 🌍

GitHub Code

Open In Colab

OmniVoice is a massively multilingual zero-shot text-to-speech (TTS) model supporting over 600 languages. Built on a novel diffusion language model-style architecture, it delivers high-quality speech with superior inference speed, supporting voice cloning and voice design.

Paper: OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models
Repository: GitHub
Demo: Hugging Face Space
Colab: Google Colab Notebook

Key Features

600+ Languages Supported: The broadest language coverage among zero-shot TTS models.
Voice Cloning: State-of-the-art voice cloning quality from a short reference audio.
Voice Design: Control voices via assigned speaker attributes (gender, age, pitch, dialect/accent, whisper, etc.).
Fine-grained Control: Non-verbal symbols (e.g., [laughter]) and pronunciation correction via pinyin or phonemes.
Fast Inference: RTF as low as 0.025 (40x faster than real-time).
Diffusion Language Model-style Architecture: A clean, streamlined, and scalable design that delivers both quality and speed.

Usage

To get started, install the omnivoice library:

We recommend using a fresh virtual environment (e.g., conda, venv, etc.) to avoid conflicts.

Step 1: Install PyTorch

NVIDIA GPU

bash

# Install pytorch with your CUDA version, e.g.
pip install torch==2.8.0+cu128 torchaudio==2.8.0+cu128 --extra-index-url https://download.pytorch.org/whl/cu128

See PyTorch official site for other versions installation.

Apple Silicon

pip install torch==2.8.0 torchaudio==2.8.0

Step 2: Install OmniVoice

pip install omnivoice

Python API

You can use OmniVoice for zero-shot voice cloning as follows:

python

from omnivoice import OmniVoice
import soundfile as sf
import torch

# Load the model
model = OmniVoice.from_pretrained(
    "k2-fsa/OmniVoice",
    device_map="cuda:0",
    dtype=torch.float16
)

# Generate audio
audio = model.generate(
    text="Hello, this is a test of zero-shot voice cloning.",
    ref_audio="ref.wav",
    ref_text="Transcription of the reference audio.",
) # audio is a list of `np.ndarray` with shape (T,) at 24 kHz.

sf.write("out.wav", audio[0], 24000)

For more generation modes (e.g., voice design), functions (e.g., non-verbal symbols, pronunciation correction) and comprehensive usage instructions, see our GitHub Repository.

Discussion & Communication

You can directly discuss on GitHub Issues.

You can also scan the QR code to join our wechat group or follow our wechat official account.

Wechat Group	Wechat Official Account
wechat	wechat

Citation

bibtex

@article{zhu2026omnivoice,
      title={OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models},
      author={Zhu, Han and Ye, Lingxuan and Kang, Wei and Yao, Zengwei and Guo, Liyong and Kuang, Fangjun and Han, Zhifeng and Zhuang, Weiji and Lin, Long and Povey, Daniel},
      journal={arXiv preprint arXiv:2604.00688},
      year={2026}
}

Disclaimer

Users are strictly prohibited from using this model for unauthorized voice cloning, voice impersonation, fraud, scams, or any other illegal or unethical activities. All users shall ensure full compliance with applicable local laws, regulations, and ethical standards. The developers assume no liability for any misuse of this model and advocate for responsible AI development and use, encouraging the community to uphold safety and ethical principles in AI research and applications.

theoracleguy/OmniVoice-4bit