Qwen3.8-27B-AEON-Ultimate-Uncensored — Multimodal MLX 6-bit
A
6-bit MLX quantization of
AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-BF16 (revision
8f76e82), AEON's abliterated release of
Qwen/Qwen3.8-27B, for local inference on Apple Silicon.
The vision tower is fully preserved (333/333 tensors — AEON's release keeps it hash-identical to stock Qwen3.8), so image and video understanding work through mlx-vlm. The model's native MTP head is published separately as a drafter for lossless speculative decoding:
VisualInference/Qwen3.8-27B-AEON-Ultimate-Uncensored-MLX-MTP-Drafter — pairing it typically speeds decode 1.4–1.9× at identical output quality.
Requirements
- Apple Silicon Mac with MLX support;
pip install mlx-vlm (converted with mlx-vlm 0.6.13 / mlx 0.32.0).
- Unified memory: ~21 GB for weights, with an observed runtime peak of ~25 GiB at short context and ~30 GiB at 13k-token context. Comfortable on 48 GB+ machines, workable on 36 GB; not recommended below that.
- Because decode is memory-bandwidth-bound, tokens/sec scales roughly linearly with the chip's memory bandwidth (see measured numbers below).
Usage
1# text / vision
2python -m mlx_vlm generate \
3 --model VisualInference/Qwen3.8-27B-AEON-Ultimate-Uncensored-Multimodal-MLX-6bit \
4 --prompt "Describe this image." --image photo.jpg
5
6# faster decode with the MTP drafter (lossless speculative decoding)
7python -m mlx_vlm generate \
8 --model VisualInference/Qwen3.8-27B-AEON-Ultimate-Uncensored-Multimodal-MLX-6bit \
9 --draft-model VisualInference/Qwen3.8-27B-AEON-Ultimate-Uncensored-MLX-MTP-Drafter \
10 --draft-kind mtp --draft-block-size 3 \
11 --prompt "Write a short story about a lighthouse keeper."
Tips:
- Draft block size 3 is the all-round sweet spot; 4 edges ahead on code; ≥5 regresses.
- When serving (
mlx_vlm.server), --prefill-step-size 512 cuts peak prefill memory by ~6 GB at no measured speed cost.
- Recommended sampling (from the Qwen3.8 card): thinking
temp=1.0, top_p=0.95, top_k=20; non-thinking temp=0.7, top_p=0.8, presence_penalty=1.5.
Conversion recipe
1python -m mlx_vlm convert \
2 --hf-path AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-BF16 \
3 -q --q-bits 6 --q-group-size 64 \
4 --mlx-path Qwen3.8-27B-AEON-Ultimate-Uncensored-mlx-6bit
Affine mode, group size 64, no calibration (RTN is near-lossless at 6-bit). Language model and vision tower are both quantized at the global setting; mtp.* tensors are excluded by design — mlx-vlm loads the MTP drafter as a separate model (see the drafter repo for the split recipe).
Measured performance
Test machine: Mac mini M4 Pro, 48 GB unified memory (~273 GB/s). Higher-bandwidth chips (M-series Max/Ultra) scale decode proportionally — serial decode on this quantization measures at ~100% of the device's streaming-bandwidth roofline.
| Workload | Config | Decode |
|---|
| Coding (temp 0.2) | MTP block 4 | 21.5 tok/s |
| Document QA @ 13k ctx (temp 0.7) | MTP block 3 | 16.9 tok/s |
| Creative prose (temp 0.7) | MTP block 3 | 15.9 tok/s |
| Serial (no drafter) | — | 11.4 tok/s |
Prefill measured ~105–110 tok/s, flat with context length up to the 13k tested. Draft acceptance is workload-dependent: ~46% on open-ended prose, substantially higher on code and grounded QA.
Validation
- Text, vision, and long-context (13k) generation smoke-tested after conversion.
- Two-phase coding gate (spec research → implementation, executed against 28 hidden edge-case asserts across an SSE-parser task and a stack-VM task): 28/28.
- Speculative decoding verified lossless (drafter rejections fall back to the target model's own tokens; the sampling distribution is unchanged by construction).
Provenance & responsibility
Qwen/Qwen3.8-27B → AEON-7 SSM-conv1d repair + abliterix abliteration (BF16, vision and MTP untouched — see
their card for methodology and KL evidence) → this repo (6-bit MLX quantization, nothing else changed).
This is an abliterated, refusal-removed model. As the upstream card puts it: the model does not decide whether to comply — you do. Outputs are the responsibility of the operator; use within the law of your jurisdiction. Apache-2.0, inherited from base.
