A 4-bit MLX quantization of
CohereLabs/cohere-transcribe-03-2026, hosted by
Spoke for on-device speech-to-text on Apple Silicon.
Derived from the native mlx-speech conversion (
mlx-community/cohere-transcribe-03-2026-mlx-8bit, maintained by
appautomaton) by dequantizing the int8 weights and re-quantizing to 4-bit (affine, group size 64) with mlx-speech's own quantizer. Loads via the
mlx-speech library (Python 3.13+), not stock mlx-audio.
On Spoke's internal LibriSpeech test-clean benchmark (250 samples), this 4-bit build measured 1.28% WER at 0.05x real-time factor with a 1.89 GB peak memory footprint, beating the previously shipped 4-bit Whisper turbo (2.66% WER) on accuracy and speed while staying under a 2 GB memory budget.
1import numpy as np, soundfile as sf
2from mlx_speech.generation import CohereAsrModel
3
4audio, sr = sf.read("input.wav", dtype="float32", always_2d=False)
5if audio.ndim > 1:
6 audio = audio.mean(axis=1)
7# resample to 16 kHz first if sr != 16000
8
9model = CohereAsrModel.from_path("path/to/this/repo")
10print(model.transcribe(audio, sample_rate=16000, language="en").text)
