An MLX INT8 build of huihui-ai/Huihui-Qwen3.8-27B-abliterated with its native Multi-Token Prediction (MTP) draft head kept intact and packed for self-speculative decoding on Apple Silicon.

On an M5 Max this decodes at 54.0 tok/s at draft depth 3 versus 17.1 tok/s autoregressive — a 3.17x speedup with no separate draft model, because the MTP head ships inside the model.

This model is uncensored. Its safety alignment has been removed. See Intended use and risks before you deploy it.

Provenance

This build is two transformations away from the original Qwen release, and neither of them trained anything:

Qwen/Qwen3.8-27B — the original release.
huihui-ai/Huihui-Qwen3.8-27B-abliterated (commit 739e3c5b89) — an abliteration of the above, produced with Sumandora/remove-refusals-with-transformers. Abliteration is a direct weight edit that identifies the model's refusal direction in activation space and projects it out. It is not fine-tuning and uses no training data. Only layers 18-51 of 64 were ablated; the MTP draft head and the vision tower were left unmodified.
This repo — an MTPLX forge of the above: INT8 body quantization plus a structurally quantized MTP sidecar. No calibration data, no training, no gradient ever computed.

Because huihui's edit never touched the MTP head, this build's draft head is bit-for-bit the head from the original Qwen release. That is verified, not assumed: see On the sidecar.

Build recipe

The trunk was forged with MTPLX using mtp_policy: keep_bf16:

json

{
  "body_bits": 8,
  "body_group_size": 64,
  "body_mode": "affine",
  "mtp_policy": "keep_bf16"
}

The MTP sidecar was then quantized post hoc as a separate structural step, matching how the official MTPLX reference build was made:


MTP quantization	INT8, group size 64, affine, policy `all`
Quantized matrices	8 — `fc`, attention `q/k/v/o_proj`, MLP `gate/up/down_proj`
Left in BF16	all head norms (`input_layernorm`, `post_attention_layernorm`, `q_norm`, `k_norm`, `mtp.norm`, `pre_fc_norm_embedding`, `pre_fc_norm_hidden`)
Sidecar size	849,400,403 B BF16 -> 451,270,903 B INT8
Calibration / training	none
Trunk weights	unchanged by this step

On the sidecar

The quantized sidecar in this repo is byte-identical (SHA-256 1e350715…) to the MTP sidecar shipped in the official Qwen3.8-27B-MTPLX-Optimized-Quality reference build.

That single fact confirms both halves of the provenance claim above: the huihui abliteration genuinely left the MTP head untouched, and this repack reproduces the official head quantization procedure exactly rather than approximating it.

Measured performance

Hardware: Apple M5 Max, 128 GB, macOS 26.5.2. Measured with mtplx forge verify (mtplx 2.9.1, suite long-code-uncapped, max_tokens 2048, profile sustained, fans pinned). Single run per depth.

Mode	tok/s	vs AR	Draft acceptance by position
AR (depth 0)	17.06	1.00x	—
Depth 1	31.49	1.85x	0.969
Depth 2	45.44	2.66x	0.952, 0.896
Depth 3 (default)	54.01	3.17x	0.973, 0.933, 0.870

Verdict mtp_depth_wins; nothing was flagged as quality-rejected or acceptance-collapsed. Depth 3 is the shipped default.

The INT8 sidecar was gated against the BF16 sidecar it replaces, on the same machine and the same harness. It did not regress — it measured better on every axis:

	BF16 sidecar	INT8 sidecar (shipped)
Depth-3 acceptance	0.951, 0.899, 0.838	0.973, 0.933, 0.870
Depth-3 multiplier	2.88x	3.17x
Depth-3 tok/s	48.89	54.01
Sidecar on disk	849 MB	451 MB

Both sidecars carry identical draft weights up to INT8 rounding, so treat the gap as single-run variance on a stochastic decode rather than as evidence that quantizing improved the head. The honest reading is flat or better, and 398 MB smaller.

What has not been measured

No capability, accuracy, or quality benchmark has been run on this model. The numbers above are decode throughput and draft-acceptance rates only. Neither the INT8 body quantization's effect on output quality nor the abliteration's effect on capability has been evaluated here. If quality matters for your use, benchmark it yourself.

Usage

Requires MTPLX on Apple Silicon; the MTP sidecar is what makes the speedup work, and a generic MLX loader will ignore it.

bash

mtplx setup --model mvid/Huihui-Qwen3.8-27B-abliterated-MTPLX-Q8 --download
mtplx quickstart --port 8000

Sampler shipped in mtplx_runtime.json: temperature 0.6, top_p 0.95, top_k 20; draft depth 3, profile sustained.

Roughly 28 GB on disk, so budget a 64 GB machine for comfort at long context.

Intended use and risks

This model is abliterated: the refusal direction has been projected out of its weights. It will not reliably decline harmful, illegal, unethical, or dangerous requests, and it will often comply fluently and confidently with requests that the original Qwen3.8-27B would have refused. The safety behavior is removed from the weights, so it cannot be recovered with a system prompt.

Consequences worth being explicit about:

Do not deploy this to untrusted or public users without your own external moderation layer.
Removing refusals does not add knowledge. Confidently-worded dangerous output can also simply be wrong, which makes it more dangerous, not less.
Abliteration is a blunt instrument. It can degrade instruction-following and reasoning in ways unrelated to safety, and as noted above, that has not been measured here.
You are responsible for what you generate with it and for complying with the Apache-2.0 terms and applicable law.

It is published for research on alignment, refusal mechanisms, and speculative decoding, and for adults who understand the above and want an unfiltered local assistant.

License

Apache-2.0, inherited from both parents: Qwen/Qwen3.8-27B and huihui-ai/Huihui-Qwen3.8-27B-abliterated. Neither the abliteration nor this quantization adds licensing terms.

Attribution

Thanks to the Qwen team for the base model and the MTP head, to huihui-ai for the abliteration, and to Sumandora for the refusal-removal tooling.