You can deploy this model and start earning money today!

divydeep/
granite-speech-4.1-2b-mlx-8bit

Name: granite-speech-4.1-2b-mlx-8bit
Availability: InStock

Views

No views yet

Granite Speech 4.1 2B MLX 8-bit

Quality-oriented MLX conversion of ibm-granite/granite-speech-4.1-2b for Apple Silicon. This is the autoregressive Granite Speech model, not the NAR variant.

Quantization

This conversion uses post-training weight quantization without training, calibration data, or an importance matrix.

Component	Precision
16-layer Conformer speech encoder	BF16
2-layer Q-Former speech projector	BF16
Eligible internal language-model linear layers	MLX affine 8-bit, group size 64
Token embedding and language-model output head	BF16
Norms, biases, and unsupported tensors	BF16

Source revision: de575db64086f84fdc79da4932d1076e965bc546
Effective average reported by MLX: 11.377 bits per weight
model.safetensors: approximately 3.14 GB

Usage

bash

pip install -U mlx-audio

python -m mlx_audio.stt.generate \
  --model /path/to/granite-speech-4.1-2b-mlx-8bit \
  --audio audio.wav \
  --output-path transcript \
  --format txt \
  --prompt "transcribe the speech with proper punctuation and capitalization."

Validation

The checkpoint was strictly loaded by mlx-audio and run with greedy decoding on IBM's bundled multilingual_sample.wav. Its English and French transcript, including punctuation, capitalization, accents, and hyphenation, exactly matched the BF16 reference. This is a smoke test, not a complete WER benchmark.

Quantization may still affect names, rare words, translation, keyword biasing, and difficult or noisy audio.

License

Apache-2.0, matching the original model.

divydeep/granite-speech-4.1-2b-mlx-8bit