4-bit MLX quantization of the dense text embedding tower of Alibaba-NLP/UEmbed-2B (Qwen3.5 text backbone), packaged to run natively on Apple Silicon with mlx-lm. It produces L2-normalized dense text embeddings via the upstream chat-template contract and was gated against the MLX bf16 reference before release.
Reference parity (bf16 baseline vs upstream): MLX bf16 vs upstream torch (transformers 5.4.0, shipped Qwen35Embedder, fp32): cosine >= 0.99998 on all 6 fixed parity inputs; retrieval sims match (matched pairs 0.77/0.74, distractors ~0).
Fidelity note: at 4-bit the embeddings drift measurably from the bf16 reference (worst-case cosine 0.9369 on the gate inputs). Retrieval ordering is preserved with wide margins, but if you need maximum embedding fidelity, prefer the 8-bit pack above.
Usage
Requires: mlx-lm with qwen3_5 support (git main as of 2026-08-11; newer than the 0.31.3 release).
python
1from mlx_lm import load
2import mlx.core as mx
3model, tok = load("majentik/UEmbed-2B-MLX-4bit")4defembed(text, instruction="Represent the user's input."):5 ids = tok.apply_chat_template(6[{"role":"system","content": instruction},{"role":"user","content": text}],7 add_generation_prompt=True, enable_thinking=False)8 h = model.model(mx.array([ids]))[0,-1,:].astype(mx.float32)9return h / mx.sqrt((h * h).sum())
Embedding contract: messages=[system(instruction), user(text)]; tokenizer.apply_chat_template(add_generation_prompt=True, enable_thinking=False); dense embedding = final-norm hidden state at the LAST prompt token, L2-normalized. (Upstream appends 16 EOS and pools at last-16 — identical under causal attention.) Default instruction: "Represent the user's input."
DENSE TEXT embeddings v1: the pack contains the Qwen3.5 text tower only (the upstream repo's visual tower and SPLADE sparse heads are NOT included; use upstream for image/video/sparse). Checkpoint keys normalized from the upstream tripled prefix (model.language_model.language_model.language_model.*) to the standard text layout.
Reproduce
Prerequisite: the upstream checkpoint stores the text tower under a tripled key prefix (model.language_model.language_model.language_model.*); normalize the keys to the standard text layout (dropping the visual tower and SPLADE sparse heads) before conversion. Then:
CC-BY-4.0, inherited from upstream Alibaba-NLP/UEmbed-2B. Attribution: this repository redistributes a quantized derivative of UEmbed-2B by Alibaba-NLP, used under CC BY 4.0, with modifications (text-tower extraction, key normalization, 4-bit group-size-32 MLX quantization).
