Apple Silicon (M1/M2/M3/M4) with TurboQuant structural pre-conditioning and MLX-native NVFP4 layout (E2M1 weights, per-16-element FP8 (NVIDIA Blackwell layout)). 4.503 bits/weight, ~18 GB on disk, sub-2-s load on M4 Max. Pick this over the affine MLX variants when you want NVFP4 format parity with hardware pipelines while running locally.

Quickstart

python

import mlx.core as mx
from mlx_lm import load, generate

model, tokenizer = load("majentik/Qwen3.6-35B-A3B-TurboQuant-MLX-NVFP4")

prompt = "Give me a short introduction to Mixture-of-Experts models."
response = generate(model, tokenizer, prompt=prompt, max_tokens=512)
print(response)

[!IMPORTANT] Text-only extraction. This repo contains only the quantized language tower of Qwen3.6-35B-A3B. The upstream vision tower (333 tensors) and MTP head are not included, so image/video input does not work and mlx_vlm.load(...) fails with a Missing ... parameters error (the vision tower it expects is absent from the checkpoint). Load it with mlx_lm (recent version with qwen3_5_moe support) as shown above. For image/video input, use the upstream BF16 model Qwen/Qwen3.6-35B-A3B on a runtime that supports it.

Hardware compatibility

Device	VRAM	Recommendation
Apple M4 Max 128 GB	~21 GB	recommended — headroom for long context
Apple M3 Max 64 GB	~21 GB	fits comfortably
Apple M2 Max 32 GB	~21 GB	tight — short context only

Reproduce

bash

# dequantize from the rotor/turbo MLX-8bit source, then re-quantize
python -c "from mlx_lm import convert; convert(hf_path=\"bf16\", mlx_path=\"out-nvfp4\", quantize=True, q_bits=4, q_group_size=16, q_mode=\"nvfp4\", trust_remote_code=True)"

Reproduced at commit 919836a.

Evaluation

Benchmark	Score
arc_easy_acc	0.4900
hellaswag_acc	0.4900

Family

bf16 — Qwen/Qwen3.6-35B-A3B
FP8 card — majentik/Qwen3.6-35B-A3B-FP8
TurboQuant MLX-4bit (affine) — majentik/Qwen3.6-35B-A3B-TurboQuant-MLX-4bit
plain MLX-NVFP4 (no rotor/turbo) — majentik/Qwen3.6-35B-A3B-MLX-NVFP4

Provenance

Calibration hash: none (nvfp4 is calibration-free; rotor/turbo conditioning inherited from source)
Uploaded: 2026-04-21T06:26:36.131649+00:00

Toolchain:

huggingface_hub: 1.11.0
mlx: 0.31.1
mlx-lm: 0.31.2

License

Released under apache-2.0. Upstream license of the base model applies.