Views
No views yet
mlx-audio for Apple Silicon. Quality-conservative variant —
use this if lightsofapollo/omnivoice-mlx-q4-g64
sounds off on your prompts.| Backbone | Total | |
|---|---|---|
| original (bf16) | ~1.2 GB | ~1.6 GB |
| this repo (int8 g=64 backbone, bf16 tokenizer) | ~620 MB | ~1.0 GB |
| Prompt | RTF (bf16) | RTF (this) |
|---|---|---|
| "Voice synthesis on Apple Silicon has come a long way. We can now generate full sentences in real time." | 3.68× | 4.60× (+25%) |
1import json
2import mlx.core as mx
3import mlx.nn as nn
4from huggingface_hub import snapshot_download
5from mlx_audio.tts.models.omnivoice.config import OmniVoiceConfig
6from mlx_audio.tts.models.omnivoice.omnivoice import Model
7
8path = snapshot_download("lightsofapollo/omnivoice-mlx-q8-g64")
9cfg_dict = json.load(open(f"{path}/config.json"))
10model = Model(OmniVoiceConfig(**{k: v for k, v in cfg_dict.items() if k in OmniVoiceConfig.__dataclass_fields__}))
11
12q = cfg_dict["quantization"]
13nn.quantize(model, group_size=q["group_size"], bits=q["bits"], mode=q.get("mode", "affine"),
14 class_predicate=lambda _p, m: hasattr(m, "to_quantized"))
15
16raw = dict(mx.load(f"{path}/model.safetensors"))
17model.load_weights(list(model.sanitize(raw).items()))
18mx.eval(model.parameters())1python -m mlx_audio.tts.models.omnivoice.convert \
2 --model k2-fsa/OmniVoice --output omnivoice-bf16 --dtype bfloat16
3
4python -m mlx_audio.convert \
5 --hf-path omnivoice-bf16 --mlx-path omnivoice-q8-g64 \
6 --quantize --q-bits 8 --q-group-size 64k2-fsa/OmniVoice).