microsoft/Fara-7B converted to MLX and quantized to 4-bit, for inference on Apple Silicon.

Fara-7B is a computer-use agent model built on Qwen2.5-VL — it reads screenshots and acts on interfaces. The vision path is preserved in this conversion, which for this model class is the point.

See also Fara-7B-8bit for the higher-fidelity variant, and Fara1.5-9B-8bit for the newer generation of the same family.

Effective bits exceed the requested value because mlx-vlm quantizes only the language model and leaves the vision tower in bf16 by design — 390 vision tensors, none of them quantized. The vision encoder is a small share of the weights but disproportionately sensitive to quantization error.

language_model : 198 tensors quantized (4-bit, group size 64)
vision_tower   : 390 tensors, bf16    <- unquantized

Measured against the bf16 source, tensor by tensor, over all 198 quantized tensors (7,615,283,200 parameters). No prompts or sampling involved — this is a direct measurement of how much numerical information the quantization discarded, and it is exact and hardware-independent.

Highest-error tensors at 4-bit — v_proj and early-layer down_proj are consistently the most sensitive:

rel_l2=0.10771  snr= 19.36 dB  language_model.model.layers.1.mlp.down_proj
rel_l2=0.10725  snr= 19.39 dB  language_model.model.layers.23.self_attn.v_proj
rel_l2=0.10547  snr= 19.54 dB  language_model.model.layers.22.self_attn.v_proj
rel_l2=0.10419  snr= 19.64 dB  language_model.model.layers.25.self_attn.v_proj
rel_l2=0.10355  snr= 19.70 dB  language_model.model.layers.27.self_attn.v_proj

Going from 4-bit to 8-bit improves the signal-to-quantization-noise ratio by 22 dB — a factor of roughly 160 in noise power. If your machine has the memory, prefer the 8-bit variant; use this one when 5.3 GB versus 8.8 GB is the deciding constraint.

Measured on an M2 Pro / 32 GB, 64 generated tokens, greedy.

This variant decodes 1.9x faster at 1.7x less memory. Numbers do not transfer across chips.

Other conversions in this series report perplexity ratio, top-1 agreement and KL divergence against the bf16 source — see Fara1.5-9B-8bit, which reaches top-1 agreement of 1.000 that way. That protocol does not work for Fara-7B, and the reason is worth stating rather than quietly omitting.

Fara-7B is a computer-use model: it expects a screenshot plus an action space, not prose. Scored on plain text it is out of distribution before any quantization — the unquantized bf16 source itself has a perplexity of 12.30 on the same passages where Fara1.5-9B scores 3.23. With a distribution that flat, the metric stops discriminating: measured that way this 4-bit variant scored a better perplexity ratio (1.2312) and KL (0.2768) than the 8-bit one (1.4897 / 0.4809), which is impossible — a 4-bit quantization cannot be more faithful than an 8-bit one of the same model.

The weight-level numbers above give the correct ordering, so the anomaly is in the measurement, not in the weights. Those behavioural figures are therefore excluded rather than reported.

A meaningful behavioural benchmark for this model would need screenshots and a verifiable action space — a computer-use harness, which was not available here.

No standard benchmarks: no ScreenSpot, WebArena, OSWorld, or any agentic evaluation. No judged quality. The vision path was verified to load and run, not scored on a dataset. If your use case is the full computer-use loop, evaluate on your own tasks.

Text-only works too — pass num_images=0 and omit image.

Note that stock mlx-lm loads the text path only; use mlx-vlm for image input.

All credit for the model belongs to Microsoft. This is a format conversion and quantization; no training or fine-tuning was performed. Licensed MIT, as the original. See the original card for intended use and limitations.