Yes — VISION PRESERVED — even at 3.7 BPW. Most OptiQ releases strip the vision tower; we keep it. The LM portion is OptiQ static-mixed (3/4/8-bit per tensor), and the ViT is spliced back in at BF16 from our 8-bit MLX build, giving you a full multimodal model that fits in 14 GB.
OptiQ static mixed ~3.7 BPW MLX quantization of a abliterated Qwen 3.6 27B v2 (the Jackrong Claude-Opus reasoning distill of Qwen 3.6 27B). Refusals reduced from 91/100 → 4/100 with KL drift of just 0.0176. By the Lemura Labs research team.
Smallest variant in our line. The LM uses mlx-optiq Frobenius-stream sensitivity scans to assign 3-bit to layers that tolerate it and 4–8-bit to the layers that don't, averaging 3.697 bits per weight. The vision tower (333 ViT weights, ~880 MB at BF16) is spliced back in unmodified from the abliterated parent — the cleanest combination of language compression and visual fidelity.
TL;DR
Property
Value
Disk size
~14 GB
LM BPW (achieved)
3.697 (target 3.7) · 4.147 effective with scale/bias
Abliteration tool: the ablation toolkit by Lemura Labs
Quantization tool: mlx-optiq by Thin Signal, on top of mlx-lm
Abliteration Results
the ablation toolkit measures refusals on mlabonne/harmful_behaviors (100 hard red-team prompts) and KL divergence on mlabonne/harmless_alpaca. These were measured at BF16 before quantization; the OptiQ-3.7bpw quant inherits them within calibration noise.
Stage
Refusals (n=100) ↓
KL divergence ↓
Vanilla Jackrong/Qwopus3.6-27B-v2
91 / 100
— (reference)
Community prior: coder3101 (T27)
4 / 100
0.0359
Community prior: wangzhang (T28)
30 / 100
0.0259
TPE best (T45) — shipped here
4 / 100
0.0176
TPE second-best (T37)
5 / 100
0.0210
→ 96% reduction in refusals with capability preserved (KL ≈ 0.018, well below the 0.3 healing threshold). No SFT / LoRA healing was required.
Method (TPE-50 with community priors → OptiQ-3.7 → vision merge)
Step 1. Abliteration (the ablation toolkit TPE-50)
Smoke — 3-trial dry run on the ablation toolkit / Python 3.11 / MPS validated batch size 4 on M4 Max 128 GB.
Random search — 25 startup trials over the ablation toolkit's parameter space (direction_index, attn.o_proj.*, mlp.down_proj.*). Best random-stage trial: T4 at direction_index=54.49 — capability-clean but still too refusing (41/100).
Community-prior enqueueing — published Qwen 27B the ablation toolkit runs (coder3101 on Qwen 3.5, wangzhang on Qwen 3.6) localized the refusal direction near layer 35–38 of 64. We injected both as study priors via study.enqueue_trial(skip_if_exists=True).
TPE smart-sampling — 23 Tree-structured Parzen Estimator trials refined around the community neighborhood. Trial 45 (direction_index=41.42) found the Pareto front: 4/100 refusals at KL=0.0176.
Auto-save — best Pareto trial merged into base weights via the ablation toolkit's LoRA-adapter merge path; saved as BF16 safetensors with vision tower fully intact.
Total the ablation toolkit wall-clock: ~13 h on M4 Max 128 GB.
Step 2. LM mixed-precision quantization (OptiQ)
Sensitivity analysis — optiq convert --target-bpw 3.7 --candidate-bits 2,3,4,8 --sensitivity stream mmaps each tensor and computes per-tensor Frobenius distance between BF16 reference and candidate quantized versions. For a 27B model on 128 GB unified memory this is the only mode that fits.
Multi-tier bit allocation — OptiQ's optimizer assigns each of the 582 quantizable tensors a bit-width from {3, 4, 8} such that the average lands at the 3.7 BPW target. Result: 38% × 8-bit (high-sensitivity), 36% × 4-bit (medium), 26% × 3-bit (low-sensitivity SSM projections and certain MLP gates). No tensor required 2-bit.
MLX serialization — quantized LM weights written via mlx-lm quantize hooks, with the per-tensor bit-width recorded in config.json["quantization"] for each module path.
Step 3. Vision splice-back (custom)
Why — OptiQ defaults to stripping vision weights and rewriting config.json to a text-only Qwen3_5ForCausalLM. We wanted a multimodal release. The --keep-unused-modalities flag preserves the architecture string but not the weights, so we built a manual splice.
What — we copied the 333 vision tensors verbatim from our companion 8-bit MLX build (where mlx_vlm.convert leaves the ViT at BF16) into a 4th safetensors shard, restored vision_config, image_token_id, video_token_id, vision_start/end_token_id, and the nested text_config block in config.json, and re-set architectures to Qwen3_5ForConditionalGeneration.
Verification — post-load norm weights at expected ~0.79 mean (no double-sanitize), text generation coherent, vision pipeline correctly reads test imagery.
Achieved LM BPW: 3.697 (target 3.7) — see optiq_metadata.json for the full per-tensor allocation table.
Effective LM bits/weight: 4.147 (with 16+16-bit scale+bias per group of 64 spread over the bit-width-weighted mean).
Group size: 64 throughout (LM).
Vision tower: spliced back at BF16 from the 8-bit MLX build (mlx_vlm.convert leaves ViTs unquantized by default; the 8-bit build's "8-bit" only refers to LM weights). 333 vision tensors, ~880 MB.
The full per-tensor allocation map lives in optiq_metadata.json. The vision tower (333 weights) does not appear in this table — it is unquantized BF16.
Architecture notes
Qwen 3.6 27B uses a hybrid attention stack — 3 linear-attention (GatedDeltaNet / SSM) layers followed by 1 full-softmax-attention layer, repeated 16× for 64 total layers; hidden 5120, vocab 248320, context 262144. The SSM kernels lack a VJP path in MLX, so backward-pass-based quant methods (DWQ, dynamic quant) cannot be applied to this architecture — OptiQ's forward-only Frobenius-stream sensitivity approach is the only calibration-aware option that works.
For inference latency at 3.7 BPW LM + BF16 ViT, expect ~12–15 tok/s on M4 Max 128 GB at batch size 1, or ~5–7 tok/s on M2/M3 16 GB at batch size 1.
Behavior caveats
Uncensored. Refusal directions were surgically removed; this model will answer prompts the parent would refuse. Use responsibly and within applicable law. The release is provided for safety research, red-teaming, and creative/educational use cases.
Multimodal preserved. Vision works — the 333 ViT weights are unmodified from the abliterated parent. The abliteration acts on the LM hidden states only.
Identity preserved. The model still self-identifies as Qwen (developed by Alibaba's Tongyi Lab) — abliteration does not rewrite factual self-knowledge.
Heavy chain-of-thought. Qwen 3.6 inherits Claude-Opus's verbose reasoning style. For terse answers, use a system prompt like "Be brief and direct. Skip your reasoning.".
Quantization noise at 3-bit. ~26% of LM tensors are at 3-bit. On long-generation tasks (>1000 tokens) you may notice slightly more drift than the 6/8-bit siblings. For best fidelity at this lineage, prefer 6-bit or 8-bit.
