Qwen3-Embedding is a decoder-only LLM-style text embedding model from the Qwen3 family, using last-token pooling to produce dense vector representations. It scores near the top of MMTEB multilingual benchmarks while retaining Apache-2.0 licensing.

Quantization

Method: MLX affine quantization (mlx_embeddings.convert), group_size=64
Bits per weight: 8
Output size: 7.5 GB (vs ~15 GB for bf16 source)

Quickstart

python

from mlx_embeddings import load

model, tokenizer = load("majentik/Qwen3-Embedding-8B-MLX-8bit")

inputs = tokenizer(
    ["What is the capital of France?", "Paris is the capital of France."],
    padding=True, truncation=True, return_tensors="mlx"
)
outputs = model(inputs["input_ids"], attention_mask=inputs["attention_mask"])
embeddings = outputs.text_embeds  # already L2-normalised, shape [batch, dim]

For sentence similarity:

python

import mlx.core as mx

e = embeddings
scores = (e[0] @ e[1:].T).tolist()
print(scores)

Model Specifications

Property	Value
Base Model	Qwen/Qwen3-Embedding-8B
Architecture	Decoder-only (Qwen3ForCausalLM) with last-token pooling
Parameters	8B (7.57B) (pre-quantization)
Context Length	32K
Embedding Dim	4096
BF16 Size	~15 GB
License	apache-2.0
Languages	100+ (multilingual)

License

Apache 2.0 — inherited from the upstream Qwen3-Embedding model. Free for research and commercial use.

majentik/Qwen3-Embedding-8B-MLX-8bit

Qwen3-Embedding-8B MLX 8-bit

What is this?

Quantization

Quickstart

Model Specifications

License

See also

majentik/
Qwen3-Embedding-8B-MLX-8bit