MLX-compatible 4-bit quantization of Meta's Omnilingual ASR CTC-300M model,
targeting on-device inference on Apple Silicon (M1/M2/M3/M4).
Omnilingual ASR is a wav2vec 2.0–style encoder-only model with a linear CTC
head, trained by Meta for speech recognition across 1,600+ languages. The
CTC variant is language-agnostic at inference time (no language hint needed).
Model
Parameters
326 M
Format
MLX safetensors (quantized linear layers + fp16 features)
Raw audio [1, samples]
→ Wav2Vec2FeatureExtractor (7-layer 1D conv, stride 320×)
→ Linear 512 → 1024
→ Wav2Vec2PositionEncoder (weight-normalized conv, kernel 128, groups 16)
→ 24 × StandardTransformerEncoderLayer (pre-norm, dim 1024, heads 16, ffn 4096)
→ LayerNorm
→ Linear 1024 → 10288 (CTC head)
→ logits [1, T/320, 10288]
CTC greedy decoding with duplicate collapsing over the argmax path.
Performance
FLEURS test set, CTC-300M fp32 on CPU (Apple M-series), 30 utterances/language,
aggregate WER via exact-edit-distance scorer (no external text normalization):
Language
WER
Audio
Inference
RTF
English (en_us)
20.0%
289 s
16.3 s
0.056
French (fr_fr)
23.2%
334 s
19.5 s
0.059
German (de_de)
16.5%
361 s
20.8 s
0.058
Arabic (ar_eg)
19.5%
331 s
17.0 s
0.051
Hindi (hi_in)
22.5%
364 s
18.2 s
0.050
Aggregate CPU RTF ≈ 0.05; on M-series GPU via MLX, expect RTF < 0.02.
(4-bit quantization typically adds <1% absolute WER on wav2vec2-class
models; treat these as close upper bounds for the quantized variant.)
Usage
python
1import mlx.core as mx
2from mlx.utils import tree_unflatten
3from safetensors import safe_open
45weights ={}6with safe_open("model.safetensors", framework="mlx")as f:7for k in f.keys():8 weights[k]= f.get_tensor(k)910# Your MLX wav2vec2 + CTC implementation consumes these keys.11# Expected input : float32 audio [1, samples] at 16 kHz, zero-mean unit-var12# Expected output: logits [1, T, 10288] then CTC greedy decode via the13# tokenizer in tokenizer.model
Swift inference is provided by speech-swift
(see Sources/OmnilingualASR/).
