Yes — VISION PRESERVED. Unlike most 4-bit quantization pipelines that degrade or strip the vision tower, this release keeps the full Qwen3.6-VL ViT and all projector weights in fp16. Use it as a real multimodal model at half the size of the 8-bit build.
MLX MXFP4 (OCP Microscaling FP4) quantization of a abliterated Qwen 3.6 27B v2 (the Jackrong Claude-Opus reasoning distill of Qwen 3.6 27B). Refusals reduced from 91/100 → 4/100 with KL drift of just 0.0176 (measured at BF16). Quantization adds only +0.87% perplexity with 92.4% greedy-token agreement vs the BF16 reference. By the Lemura Labs research team. Quantized with mlx-mtp — a vision-preserving MXFP4 quantizer.
Fastest absolute throughput of any Qwen3.6-27B MLX build — 27.7 tok/s vanilla on M4 Max. Apple Silicon is memory-bandwidth-bound: halving the model fits more weights in fast SRAM, which matters more than the precision drop.
TL;DR
Property
Value
Disk size
~14 GB
Effective BPW
4.25 (4-bit E2M1 data + E8M0 scale per group of 32)
All variants share the same abliterated base weights — pick by your runtime (Apple Silicon → MLX; CUDA/CPU/cross-platform → GGUF) and your RAM budget. MXFP4 is the fastest MLX build on M4 Max; use 8-bit if you want the MTP head for native speculative decoding without a drafter.
Lineage
Qwen/Qwen3.6-27B (Qwen Team — base multimodal pretrain)
│
▼
Jackrong/Qwopus3.6-27B-v2 (Jackrong — Claude-Opus reasoning distill)
│
▼
ablation abliteration (TPE-50) (Lemura Labs)
├── 25 random startup trials
├── 2 community priors (coder3101, wangzhang)
└── 23 TPE smart-sampling trials → best at trial 45
│
▼
lemuralabs/Qwen3.6-27B-V2-zerofuse (BF16 full-precision checkpoint)
│
▼
this repo — MLX MXFP4 quant (/mlx-mtp · Lemura Labs)
the ablation toolkit measures refusals on mlabonne/harmful_behaviors (100 hard red-team prompts) and KL divergence on mlabonne/harmless_alpaca.
Stage
Refusals (n=100) ↓
KL divergence ↓
Vanilla Jackrong/Qwopus3.6-27B-v2
91 / 100
— (reference)
Community prior: coder3101 (T27)
4 / 100
0.0359
Community prior: wangzhang (T28)
30 / 100
0.0259
TPE best (T45) — BF16 source
4 / 100
0.0176
MXFP4 quant (this model)
4 / 100
0.034 nats vs BF16
→ 96% reduction in refusals with capability preserved at BF16. MXFP4 quantization adds only +0.87% perplexity on top — the abliteration and quantization quality loss are both negligible.
Method
Abliteration (inherited from the BF16 source) — TPE-50 Optuna search on the ablation toolkit, M4 Max 128 GB. Full method in that card.
MXFP4 quantization (this repo):
Source — loaded lemuralabs/Qwen3.6-27B-V2-abliterated-uncensored (51 GB BF16, 3 shards) into MLX on M4 Max.
Layer audit — mlx_mtp.mxfp4_quantize catalogued 1199 tensors: identified all LM linear projections (attention Q/K/V/O + MLP gate/up/down + lm_head) as MXFP4 candidates; flagged 196 vision-tower + projector tensors and 48 SSM-sensitive params (a_log, dt_bias, conv1d) for fp16 preservation.
Quantization — LM linears → MXFP4 (OCP MX E2M1, group_size=32) using mlx.core.quantize. Vision + SSM tensors written verbatim at fp16. 15 MTP head weights absent from source — quantizer set mtp_num_hidden_layers: 0 in output config to prevent strict-load failure.
Output — 3 safetensor shards, 14 GB total. Embedded mlx_mtp config block records format, vision_fp16=True, mtp_preserved=False.
Verification — text generation correct; vision captioning correct (image → caption matches BF16 output); DFlash block-diffusion speculative decoding loads and runs; KL divergence measured end-to-end (see Quality section below).
Total wall-clock: ~2 h on M4 Max 128 GB (dominated by disk I/O).
Quantization quality vs BF16
Measured via teacher-forced prefill — 10 diverse prompts (code, math, reasoning, instruction-following), 244 token positions, full 248K-vocab distributions at every position.
Metric
Value
Interpretation
KL(bf16 ‖ mxfp4) mean
0.034 nats
Forward divergence from reference
JSD mean
0.009 nats
1.3% of [0, ln2] bound — essentially identical
Top-1 agreement
92.4%
Both pick same greedy token at 9 in 10 positions
PPL increase
+0.87%
Perplexity vs BF16 reference tokens
Code prompts (KL)
0.010–0.012
Lowest — code is deterministic
JSD < 0.01 is the standard "essentially identical distributions" threshold. At 0.009 we are below it. The 7.6% of positions where models disagree on the top-1 token are near-tie argmax flips from quantization noise — not quality regressions.
Use it
Fastest inference via mlx-mtp (recommended — native DFlash support)
Vision tower: preserved verbatim in fp16 — 333 vision weights (ViT encoder, vision projector, image-token embeddings, deepstack indexes). Pixel/patch features degrade visibly at 4-bit; keeping them fp16 is free (vision = minority of total params).
SSM-sensitive params: preserved in fp16 — 48 params (a_log, dt_bias, conv1d). Qwen3.5's hybrid-SSM recurrent state is numerically sensitive; fp16 costs nothing at scale.
MTP head: disabled (mtp_num_hidden_layers: 0). The BF16 source had 0 MTP weights; quantizer detects this and clears the config to prevent strict-load failure. For native MTP speculative decoding, use the 8-bit build.
Quantizer: mlx_mtp.mxfp4_quantize.
Architecture notes
Qwen 3.6 27B uses a hybrid attention stack — 3 linear-attention (GatedDeltaNet / SSM) layers followed by 1 full-softmax-attention layer, repeated 16× for 64 total layers; hidden 5120, vocab 248320, context 262144. The hybrid is fully supported by mlx-vlm 0.5.0+. For inference latency at MXFP4, expect 27–30 tok/s on M4 Max 128 GB at batch size 1 — approximately 2× faster than the 8-bit build because Apple Silicon is memory-bandwidth-bound and the smaller model fits more weights in fast SRAM per cycle.
Behavior caveats
Uncensored. Refusal directions were surgically removed; this model will answer prompts the parent would refuse. Use responsibly and within applicable law. The release is provided for safety research, red-teaming, and creative/educational use cases.
Identity preserved. The model still self-identifies as Qwen (developed by Alibaba's Tongyi Lab) — abliteration does not rewrite factual self-knowledge.
Heavy chain-of-thought. Qwen 3.6 inherits Claude-Opus's verbose reasoning style. For terse answers, use a system prompt like "Be brief and direct. Skip your reasoning.".
Credits
MXFP4 quantization
/mlx-mtp — vision-preserving MXFP4 + oQ8 quantizer + native MTP / DFlash speculative decode engine for Apple Silicon VLMs.
Apache-2.0, inherited from the foundation (Qwen3.6-27B) and the distill (Qwen 3.6 27B-v2) upstream.
Need a hosted endpoint, custom quant, or larger-scale inference? Lemura Labs — multi-provider LLM routing for the Indian developer ecosystem.
Faster decoding with DFlash (MLX, block-diffusion speculative)
This MLX build supports block-diffusion speculative decoding via DFlash — no requantization, no model changes. Because MXFP4 runs at ~27.7 tok/s baseline (already fast), DFlash gains are modest vs the 8-bit build; use block size 8 for the best result.
Why the modest speedup? DFlash's drafter overhead becomes a larger fraction of total time as baseline throughput rises. The 8-bit build sees ~3× gains because it starts slower; mxfp4 is already bandwidth-limited in a faster regime. If raw throughput is the priority, mxfp4 vanilla at 27.7 tok/s already beats oQ8 + DFlash (best = 23 tok/s).
Via mlx-mtp (recommended — correct block-size selection):
python
1from mlx_mtp.engine import load_model
2from mlx_mtp.dflash import load_dflash_drafter, dflash_generate
34model, processor, config = load_model("lemuralabs/Qwen3.6-27B-v2.1-uncensored-mxfp4")5drafter, _ = load_dflash_drafter("z-lab/Qwen3.6-27B-DFlash")67out = dflash_generate(model, processor, config, drafter,8 text="Write a merge function for two sorted lists in Python.",9 max_tokens=256, draft_block_size=8)10print(out["text"])
Via mlx-vlm CLI (standard interface):
bash
1python3 -m mlx_vlm generate \2 --model lemuralabs/Qwen3.6-27B-v2.1-uncensored-mxfp4 \3 --draft-model z-lab/Qwen3.6-27B-DFlash --draft-kind dflash \4 --prompt "Write a merge function for two sorted lists in Python." --max-tokens 256
