FireRedLID 887M — MLX INT8
Native MLX conversion of
FireRedTeam/FireRedLID,
including its relative-position Conformer encoder, autoregressive LID decoder,
and original beam-3, two-token hierarchical decoding. This is not a
single-step classifier approximation.
This release applies MLX native 8-bit affine weight-only quantization with group size 64 to all eligible encoder and decoder linear matrices. Activations are not quantized, and convolution, normalization, embedding, and positional tensors retain the mixed-fp16-encoder-fp32-decoder release's validated precision. INT8 is numerically lossy; token parity and logit comparisons are recorded in validation.json. The audio frontend reproduces the
official 80-bin Kaldi fbank and global CMVN path, including the Int16 PCM
amplitude convention.
Usage
1pip install mlx==0.32.0 numpy==1.26.4 scipy==1.14.1 \
2 soundfile==0.13.1 kaldiio==2.18.1 kaldi-native-fbank==1.22.3
1from pathlib import Path
2
3import mlx.core as mx
4
5from frontend import FireRedFrontend
6from mlx_model import FireRedLIDModel
7
8model_dir = Path(".")
9frontend = FireRedFrontend(model_dir / "cmvn.ark")
10features = frontend.extract_file("speech.wav")
11model = FireRedLIDModel.from_directory(model_dir)
12result = model.identify_features(mx.array(features[None]))
13print(result[0][0].text, result[0][0].confidence)
Architecture and provenance
- 16 kHz mono audio; 80-bin Kaldi fbank; 25 ms window / 10 ms shift;
snip_edges=true; zero dither; supplied global CMVN.
- Two stride-2 Conv2d layers (4x time subsampling).
- 16-layer, 1,280-dimensional relative-position Conformer encoder.
- 20 attention heads (64 dimensions per head) in both encoder and decoder.
- Six-layer autoregressive Transformer LID decoder over 120 tokens.
- Beam size 3, maximum output length 2, smoothing 1.25, GNMT length penalty
0.6, and EOS penalty 1.0.
- Source model revision:
1bb4d285c8456429385d9c0810300df4297bc11b.
- Source checkpoint SHA-256:
7dee2a280e9b11d5241a0e3d4fa60ee1520a036a2e8385f17960371cfea10093.
- Official source revision:
4e7d9aaf4482a47cec1724807026b9b151926eb5.
The official paper reports 97.18% on an 82-language FLEURS intersection,
92.07% on a 74-language Common Voice intersection, and 88.47% on its combined
Chinese dialect/accent evaluation. Those intersections differ from other LID
papers and should not be treated as directly comparable full-dataset scores.
Conversion validation
validation.json records independent frontend comparison, Conv2d subsampling
parity, all 16 encoder-layer comparisons, both autoregressive decoder steps,
decoded tokens, confidence, latency, and peak MLX memory. Validation runs the
official PyTorch model in a separate process before loading MLX so the two
large runtimes do not overlap in memory.
benchmark.json records fresh-process first-shape and repeated model-inference
latency after separately timed frontend extraction, plus peak memory.
Filesystem pages may already be cached, so its first inference timing is not
represented as a cold-disk download/load benchmark.
The 1,134,584,042-byte artifact is 46.06% smaller than the validated
2,103,273,736-byte mixed-precision release. On four parity fixtures spanning
1.5 to 30 seconds, all decoded tokens match the pinned official PyTorch model,
including the hierarchical zh mandarin result. Minimum cosine similarity is
0.9999585914 across all encoder layers and 0.9999605973 across both decoder
steps; the maximum confidence difference is 0.00014848.
On Apple Silicon, the median of three fresh processes (seven warm repetitions
each) reduced peak process RSS from 2.263 GB to 1.299 GB (42.6%). Model
inference after frontend extraction improved from 89.3 to 53.7 ms first and
51.1 to 31.8 ms warm for a 1.5-second clip. For a 30-second clip, it improved
from 260.2 to 251.9 ms first and from 255.6 to 242.4 ms warm. Frontend
extraction is separately timed at roughly 1.1 ms and 18 ms and is unchanged by
weight quantization. Long-input peak MLX memory fell from 3.904 GB to 2.753 GB
(29.5%). See benchmark_comparison.json for raw medians and methodology.
These fixture measurements do not establish lossless quantization or unchanged
corpus-level accuracy. The independent multilingual evaluation below broadens
the release gate; the mixed-precision model remains the closest parity
reference.
Independent FLEURS quantization evaluation
fleurs82_evaluation.json records a deterministic stratified sample from the
FLEURS test set at revision
70bb2e84b976b7e960aa89f1c648e09c59f894dd: three clips for each of 82 mapped
languages, evaluated clean and with 8 kHz telephony, 10 dB noise, and an
energy-bearing 1.5-second crop.
evaluate_fleurs.py reproduces the evaluation.
| Condition | Clips | Dense accuracy | INT8 accuracy | Token agreement |
|---|
| Clean | 246 | 95.93% | 95.93% | 100% |
| 8 kHz telephony | 246 | 97.15% | 97.15% | 100% |
| 10 dB noise | 246 | 92.68% | 92.68% | 100% |
| Highest-energy 1.5 s | 246 | 36.59% | 36.99% | 99.59% |
| All conditions | 984 | 80.59% | 80.69% | 99.90% |
There are zero INT8 regressions and one INT8 recovery. Confidence absolute
difference has median 0.00000191, mean 0.001441, p95 0.008888, p99 0.017291,
and maximum 0.034719. The maximum occurs on a short low-margin input where both
models return the same label. Applications using a confidence threshold should
still calibrate that threshold directly on this INT8 release.
The upstream paper reports a full-test result over 82 languages but does not
publish its language list or mapping. This release defines its intersection as
79 direct dictionary/FLEURS code matches, Mandarin and Cantonese hierarchical
mappings, and the checkpoint's legacy iw/jw Hebrew and Javanese codes. The
sampled result is not a reproduction of the paper's 97.18% headline and should
not be compared as if it were the full test set.
Limitations
This is a closed-vocabulary decoder. It does not provide calibrated unknown-
language detection and can return a confident known token for silence, music,
unsupported speech, or code-switching. The upstream training range is 1.5 to
35 seconds. Production systems should use VAD, target-domain calibration, and
an explicit reject policy. Language and regional-accent labels are model
predictions, not reliable inferences of nationality or ethnicity.
See the
official paper and
source code for the training and
evaluation methodology.
