MLX MXFP8 (8-bit microscaling float) quantization of
LiquidAI/LFM2.5-VL-3B, Liquid
AI's efficient vision-language model: a hybrid convolution/attention LFM2.5
text backbone (≈2.6B) paired with a SigLIP2 NaFlex vision encoder (≈400M). Runs
on Apple Silicon via mlx-vlm. Stays
image-text-to-text — the vision tower and multimodal projector are kept in
bf16, so only the text backbone is quantized.
≈3.6 GB weights + KV cache (tested on a 24 GB M-series Mac)
This is the higher-fidelity build. For the smallest footprint, the
MXFP4 build (2.3 GB) is
also available.
Verification
Quantized with mlx_lm.quantize_model (mode mxfp8, group 32), keeping the
vision path in bf16. LFM2.5-VL is an efficient instruct model (not a heavy
reasoner), so the reference is the model's own bf16 behaviour, generated on
the same machine with deterministic greedy decoding.
Text — vs bf16 reference (greedy):
Prompt
bf16
MXFP8
17 + 28 (number only)
45
45
✅
256 / 4 (number only)
64
64
✅
capital of France
Paris
Paris
✅
largest planet
Jupiter
Jupiter
✅
three primary colors
Red, Blue, Yellow
Red, Blue, Yellow
✅
describe a sunrise (open-ended)
"…horizon in shades of gold and pink."
"…sky in vibrant shades of orange and pink."
≈
5/6 exact match to bf16 on the deterministic prompts; the only divergence is
the open-ended creative sentence, which stays coherent and on-theme (expected
for free-form generation). Mean character-level similarity to bf16: 0.971.
Vision — image discrimination (bf16 vision path, through the quantized backbone):
Image
Question
Answer
solid red
main color?
Red
✅
solid blue
main color?
Blue
✅
green circle
shape and color?
"a green circle"
✅
Colour and shape are read correctly — the vision path is live, not ignored.
The multi-tile + thumbnail split path was also exercised (a 1024×640 two-region
image read as "the left half is red, and the right half is green"); the vision
tower + projector are the identical bf16 weights in both the MXFP4 and MXFP8
builds.
1from mlx_vlm import load, generate
2from mlx_vlm.prompt_utils import apply_chat_template
34model, processor = load("sahilchachra/LFM2.5-VL-3B-MXFP8")5config = model.config
67# text8prompt = apply_chat_template(processor, config,"What is the capital of France?")9print(generate(model, processor, prompt, max_tokens=64, verbose=True))1011# image12prompt = apply_chat_template(processor, config,"Describe this image.", num_images=1)13print(generate(model, processor, prompt, image="photo.jpg", max_tokens=128, verbose=True))
Recommended sampling (from the base model card): temperature=0.2,
top_k=50, repetition_penalty=1.0.
Run in LM Studio
Loads and runs in LM Studio (tested on 0.4.20, mlx-llm runtime) with both
text and image working — the lfm2_vl architecture is recognized, the model
indexes cleanly (format: mlx metadata is present), and the ChatML template runs
as-is. Verified: text prompts return correct answers, and image prompts sent via
the OpenAI-compatible API are read correctly (solid-red → "Red", solid-blue →
"Blue"). It indexes and runs as shipped from this repo — no additional
changes needed on top of what the repo already contains.
Notes & limitations
Compatibility fix baked into this repo. On current mlx-vlm 0.6.12 the
lfm2_vl config defaults are set for the earlier (smaller) LFM2-VL sizes, so
the base LFM2.5-VL-3B config cannot load as-is: block_ff_dim falls back to
6656 and the model fails with a feed-forward shape error, and image_token_index
falls back to 396 so every image request raises "Image features and image
tokens do not match". This repo's config.json adds both fields explicitly
(block_ff_dim=10752, image_token_index=124907) — redundant, correct aliases
of the base config's intermediate_size / image_token_id that change nothing
about the weights. If you hit those errors loading the base repo, this is the
fixed one.
tie_word_embeddings=true: the token embedding is shared with the output
projection and is quantized here at 8-bit (ample headroom).
Inherits all capabilities and limitations of the base model. See the
original model card.
