An 8-bit, group-size-64 MLX quantization of Xiaomi's
MiLMMT-46-1B-v1.0, for on-device
translation on Apple Silicon. 46 languages.
Unlike the 4B, this checkpoint is a plain Gemma3ForCausalLM (model_type: gemma3_text) with no
vision tower, so the conversion is a straight re-quantization — nothing is dropped.
The small download is tempting. Measured against the 4B, it is not a close call.
A 156-item hand-authored translation set (46-language spread; en→X, X→en and non-English pairs,
with items targeting negation scope, numbers/dates, idiom, register and script) was run through
three on-device systems and judged blind — per-item shuffled labels, two independent judge
panels, 95.7% inter-judge agreement. Restricted to the 153 items whose target language MiLMMT
actually supports:
system
score
items with a confirmed meaning-changing error
MiLMMT-46 4B (5-bit)
86.5%
5
Hunyuan-MT2 1.8B (5-bit)
77.5%
13
MiLMMT-46 1B (8-bit, this repo)
73.9%
19
Three of the 1B's errors were polarity inversions — fluent, confident output meaning the
opposite of the source:
ko→ja: 늦지 않을 것 같아요 ("I won't be late") → 間に合わないと思います ("I won't make it")
es→fr: No me acordaba ("I didn't remember") → Je n'avais pas oublié ("I hadn't forgotten")
en→ja: "was rejected" → 採用された ("was hired")
It also read "half past ten" as halb zehn (9:30) and left 野餐 — a Chinese word, not Japanese —
in Japanese output. A reader cannot detect any of these without knowing the target language.
This is not quantization damage. The measurements above are of the 8-bit build; the same
family's 4-bit and 5-bit builds at 4B show no cliff at all.
Where the 1B is genuinely usable
Its weakness is generation, not comprehension:
direction
score
X→English
90%
English→X
68%
non-English → non-English
69%
At 90% into English it is a reasonable choice for reading foreign text on a small device. For
producing text in another language, prefer the
4B.
Caveat on the numbers: LLM judges, and the same author wrote the source sentences. The direction
of a 12-point gap is trustworthy; the exact figures are not a substitute for COMET/XCOMET.
Usage
python
1from mlx_lm import load, generate
2from mlx_lm.sample_utils import make_sampler
34model, tokenizer = load("translate-studio/MiLMMT-46-1B-v1.0-8bit-MLX")56src, tgt ="Japanese","English"7text ="先週の会議で決まったことを、来週までにまとめておいてください。"8prompt =f"Translate this from {src} to {tgt}:\n{src}: {text}\n{tgt}:"910print(generate(model, tokenizer, prompt=prompt, max_tokens=512,11 sampler=make_sampler(temp=0.0), verbose=False))
Prompt format
The model card's plain block, with no BOS token and no chat template — the checkpoint's
chat_template.jinja just concatenates message contents, so there are no turn markers:
Translate this from <source language name> to <target language name>:
<source language name>: <source sentence>
<target language name>:
Language names must be written as the base model card lists them (e.g. Chinese (Simplified)).
Note on stop tokens
config.json carries only eos_token_id: 1, while generation_config.json lists [1, 106]
(106 is <end_of_turn>). Loaders reading the stop id from config.json alone can miss 106
and run on to max_tokens. Honour both.
Prompts targeting Telugu, Marathi or Gujarati — which are not in that list — return fluent
Hindi/Devanagari rather than an error. Filter to the supported set; do not rely on the model to
refuse.
License & attribution
Derived from Gemma and distributed under the Gemma Terms of Use.
Credit: Xiaomi Inc. (MiLMMT-46) and Google (Gemma 3 base architecture). This repository
only provides an MLX-quantized copy of the released weights, redistributed under the same terms.
bibtex
1@misc{han2026referencefreeposttrainingopenlarge,
2 title={Reference-Free Post-Training of Open Large Language Models for Multilingual Machine Translation},
3 author={Chris Han and Pengzhi Gao and Pei Fu and Jian Luan},
4 year={2026},
5 eprint={2608.10812},
6 archivePrefix={arXiv},
7 primaryClass={cs.CL},
8 url={https://arxiv.org/abs/2608.10812}
9}
