The upstream MLX weights (mlx-community/parakeet-tdt-0.6b-v2) are full precision: 2,472 MB to download, then quantized on each machine at load time. These weights are the result of that same quantization (mlx.nn.quantize, bits=8, group_size=64), saved once:

	upstream	this repo
download	2,472 MB	933 MB
model load	~1.7 s	~0.05 s
transcription output	reference	identical

Verified byte-identical transcripts against the quantize-at-load path on real speech clips (English, 5 to 45 seconds).

Usage

parakeet_mlx.from_pretrained() does NOT load quantized weights: it builds a full-precision model and loads the tensors into it. Build the quantized structure first, then load:

python

import json
import mlx.nn as nn
from huggingface_hub import hf_hub_download
from parakeet_mlx.utils import from_config

repo = "cosmicfusionlabs/parakeet-tdt-0.6b-v2-mlx-8bit"
config = json.load(open(hf_hub_download(repo, "config.json")))
quant = config.pop("quantization")            # {"bits": 8, "group_size": 64}

model = from_config(config)
nn.quantize(model, bits=quant["bits"], group_size=quant["group_size"])
model.load_weights(hf_hub_download(repo, "model.safetensors"))
model.eval()

result = model.transcribe("audio.wav")
print(result.text)

Accuracy

Same weights as quantizing the upstream model at load time, so accuracy is unchanged from the 8-bit path. In our own benchmark on real (non-synthetic) English speech, 8-bit Parakeet TDT 0.6B v2 measured 11.6% mean WER, between whisper-small.en (13.5%) and whisper-medium.en (8.2%), while running 2 to 4x faster than medium.en on an M1 Pro. 4-bit was measured too and degraded accuracy (13.3% WER), which is why this repo ships 8-bit.

Credits and license

Original model: nvidia/parakeet-tdt-0.6b-v2 (CC-BY-4.0)
MLX conversion: mlx-community/parakeet-tdt-0.6b-v2
Runtime: parakeet-mlx

Released under CC-BY-4.0, the same license as the original NVIDIA model.