Quantization: Block floating-point MX FP4 (microscaling)
Precision: ~4 bits/weight
Group size: 32
Disk size: 917 MB
Quantized by: sahilchachra

About this variant

Microscaling (MX) block floating-point quantization at FP4 precision. Uses a shared floating-point exponent per block of 32 weights instead of integer affine scaling — different numerical properties vs affine int4.

Benchmark results

Evaluated on Apple M5 Pro with MLX. Model loaded once; performance and quality measured in a single pass.

Performance

	This model	FP16 baseline
Prefill (tok/s)	1475.39	1269.81
Decode (tok/s)	227.95	77.12
Peak memory (GB)	1.224	3.72
Disk size (MB)	917	3897

Translation quality (FLORES-200 devtest)

Reported as chrF++ (higher is better). Sample-size noted per pair.

Direction	This model	FP16 baseline	n
eng_Latn→fra_Latn	64.09	63.81	20
eng_Latn→deu_Latn	57.0	57.66	20
eng_Latn→zho_Hans	26.22	29.09	20
eng_Latn→jpn_Jpan	31.15	34.19	20
eng_Latn→spa_Latn	56.15	56.5	20
fra_Latn→eng_Latn	63.38	64.58	20
zho_Hans→eng_Latn	52.78	55.17	20
jpn_Jpan→eng_Latn	51.53	55.29	20

Avg chrF++: 55.46 vs FP16 56.95
Avg BLEU: 29.62 vs FP16 30.71

Context scaling (decode tok/s)

Context length	Decode tok/s
~128 tokens	223.0
~256 tokens	220.3
~512 tokens	220.8
~1024 tokens	217.3

Usage

Install

pip install mlx-lm

Translate

python

from mlx_lm import load, generate

model, tokenizer = load("sahilchachra/hy-mt2-1.8b-mxfp4-mlx")

prompt = (
    "Translate the following text from English to French.\n"
    "English: The early bird catches the worm.\n"
    "French:"
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=128, verbose=True))

Stream

python

from mlx_lm import load, stream_generate

model, tokenizer = load("sahilchachra/hy-mt2-1.8b-mxfp4-mlx")
for chunk in stream_generate(model, tokenizer, prompt="Translate \"Hello world\" to Japanese:", max_tokens=64):
    print(chunk.text, end="", flush=True)

All variants in this collection

Model	Method
sahilchachra/hy-mt2-1.8b-4bit-mlx	Affine int4 (group 64)
sahilchachra/hy-mt2-1.8b-8bit-mlx	Affine int8 (group 64)
sahilchachra/hy-mt2-1.8b-mxfp4-mlx	Block float MX FP4 ← this model
sahilchachra/hy-mt2-1.8b-mxfp8-mlx	Block float MX FP8

Notes

Requires Apple Silicon (M1 or later) with MLX
Benchmarks run on Apple M5 Pro, 24 GB unified memory
FLORES-200 sample sizes are small — treat chrF/BLEU figures as indicative, not definitive
License: see tencent/Hy-MT2-1.8B for the original model's license terms

Original model

See tencent/Hy-MT2-1.8B for full model details, supported languages, and intended use.