HY-Embodied-0.5 — 5-bit MLX
5-bit affine quantization (group size 64) of
tencent/HY-Embodied-0.5,
a 3.79B mixture-of-transformers embodied VLM, running natively on Apple
Silicon. These weights require the from-scratch MLX implementation of the
hunyuan_vl_mot architecture published alongside them:
hy-embodied-mlx. No other public runtime supports this
architecture; the implementation validates token-for-token against the
reference before quantization.
What was modified
The original BF16 safetensors were converted to MLX layout, and the 448
decoder linear layers (both the text path and the _v vision-routed
path) plus the tied embedding table were quantized to 5-bit, group
size 64. The vision tower, merger, and all norms remain BF16. The
quantization_manifest.json file in this repo records the decision for
every module. Nothing else was changed.
Measured
| weights | decode tok/s | Where2Place no-think | Where2Place think |
|---|
| this repo (5-bit) | 2.98 GiB | 136.9 | 0.700 [0.605, 0.781] | 0.677 [0.580, 0.761] |
| bf16 reference | 7.05 GiB | 66.8 | 0.696 [0.600, 0.778] | 0.690 [0.593, 0.772] |
Text-only probe (50 scripted prompts): 0.660 [0.522, 0.776] no-think, 0.840 [0.715, 0.917] think
(bf16: 0.700 [0.562, 0.809] / 0.900 [0.786, 0.957]).
Brackets are Wilson 95% intervals (n=100 pointing, n=50 probe). Within the intervals of bf16 on both metrics; the text-probe dip (0.66 vs 0.70) is two questions out of fifty and not significant.
Measured on an M3 Max (36 GB), greedy decoding; per-question CSVs,
statistical addendum, and one-command reproduction live in the
implementation repo. All comparisons are within-harness: Tencent's
published Where2Place 68.0 comes from an unpublished harness and is cited
for range only.
The "Use this model" snippet Hugging Face auto-generates for MLX repos
(mlx-vlm) does not support this architecture; the Usage section below is
the working path.
Usage
1from PIL import Image
2from transformers import AutoTokenizer
3from hy_embodied_mlx.model import load, generate
4from hy_embodied_mlx.pointing import FORMAT_INSTRUCTION
5from hy_embodied_mlx.processor import Processor
6
7model_dir = "HY-Embodied-0.5-5bit-mlx"
8tok = AutoTokenizer.from_pretrained(model_dir)
9model = load(model_dir)
10messages = [{"role": "user", "content": [
11 {"type": "image"},
12 {"type": "text", "text": f"Point to the red mug in the image. {FORMAT_INSTRUCTION}"},
13]}]
14inputs = Processor(tok)(messages, images=[Image.open("desk.jpg")])
15print(tok.decode(generate(model, inputs, max_tokens=128)))
Pointing needs the format instruction shown — a bare "point to X" gets a
prose location description. Emitted coordinates are integers in 0-1000,
normalized to the preprocessed canvas (Tencent's documentation does not
specify the frame; for images whose dimensions are multiples of 32 and
within the 2048x2048 pixel budget, the canvas is pixel-identical to the
input image). Thinking mode is controlled with
enable_thinking=True/False on the chat template.
License
These weights are a Model Derivative of Tencent HY, distributed under the
Tencent HY Community License (full text in the LICENSE file; NOTICE
included). This is not an open-source license. The obligations and
restrictions pass through to you:
- Territory: the license does not grant rights in the European Union, the
United Kingdom, or South Korea.
- The Section 5(a) acceptable-use restrictions and the Section 5(b)
restriction — including not using this model or its outputs to improve
any other AI model — apply to these weights and anything you build on
them.
- If you redistribute these weights or derivatives of them, include a copy
of the license agreement, the NOTICE file, and a prominent statement of
what you modified.
These Model Derivatives are distributed by the Hugging Face account
vimalnakrani. This repository is an independent quantization and is not
affiliated with, sponsored, or endorsed by Tencent.
