HY-Embodied-0.5 — bf16 MLX
BF16 MLX conversion of
tencent/HY-Embodied-0.5,
a 3.79B mixture-of-transformers embodied VLM, running natively on Apple
Silicon. This is the reference every measurement in the quantization
ladder was made against: it matches the pinned upstream implementation
token-for-token over the committed golden fixtures, in both thinking
modes. These weights require the from-scratch MLX implementation of the
hunyuan_vl_mot architecture published alongside them:
hy-embodied-mlx. No other public runtime supports this
architecture.
What was modified
BF16 safetensors converted to MLX layout; auto_map removed; nothing
quantized.
Measured
| weights | decode tok/s | Where2Place no-think | Where2Place think |
|---|
| this repo (bf16) | 7.05 GiB | 66.8 | 0.696 [0.600, 0.778] | 0.690 [0.593, 0.772] |
Text-only probe (50 scripted prompts): 0.700 [0.562, 0.809] no-think,
0.900 [0.786, 0.957] think.
Brackets are Wilson 95% intervals (n=100 pointing, n=50 probe). Measured
on an M3 Max (36 GB), greedy decoding; per-question CSVs, statistical
addendum, and one-command reproduction live in the implementation repo.
Quantized variants measured against this reference:
8-bit,
6-bit,
5-bit,
4-bit.
The "Use this model" snippet Hugging Face auto-generates for MLX repos
(mlx-vlm) does not support this architecture; the Usage section below is
the working path.
Usage
1from PIL import Image
2from transformers import AutoTokenizer
3from hy_embodied_mlx.model import load, generate
4from hy_embodied_mlx.pointing import FORMAT_INSTRUCTION
5from hy_embodied_mlx.processor import Processor
6
7model_dir = "HY-Embodied-0.5-bf16-mlx"
8tok = AutoTokenizer.from_pretrained(model_dir)
9model = load(model_dir)
10messages = [{"role": "user", "content": [
11 {"type": "image"},
12 {"type": "text", "text": f"Point to the red mug in the image. {FORMAT_INSTRUCTION}"},
13]}]
14inputs = Processor(tok)(messages, images=[Image.open("desk.jpg")])
15print(tok.decode(generate(model, inputs, max_tokens=128)))
Pointing needs the format instruction shown — a bare "point to X" gets a
prose location description. Emitted coordinates are integers in 0-1000,
normalized to the preprocessed canvas (Tencent's documentation does not
specify the frame; for images whose dimensions are multiples of 32 and
within the 2048x2048 pixel budget, the canvas is pixel-identical to the
input image). Thinking mode is controlled with
enable_thinking=True/False on the chat template.
License
These weights are a Model Derivative of Tencent HY, distributed under the
Tencent HY Community License (full text in the LICENSE file; NOTICE
included). This is not an open-source license. The obligations and
restrictions pass through to you:
- Territory: the license does not grant rights in the European Union, the
United Kingdom, or South Korea.
- The Section 5(a) acceptable-use restrictions and the Section 5(b)
restriction — including not using this model or its outputs to improve
any other AI model — apply to these weights and anything you build on
them.
- If you redistribute these weights or derivatives of them, include a copy
of the license agreement, the NOTICE file, and a prominent statement of
what you modified.
These Model Derivatives are distributed by the Hugging Face account
vimalnakrani. This repository is an independent conversion and is not
affiliated with, sponsored, or endorsed by Tencent.
