An unofficial family of MLX derivatives of
Qwen/Qwen3.8-27B. The original
model is by Qwen; the MLX conversions, refusal-direction experiment, and
validation were performed by PocketAI Model Lab. PocketAiHub identifies the
publisher of these derivatives, not the creator of Qwen3.8.
This consolidated repository contains experimental 2-bit AWQ plus 4-bit,
6-bit, 8-bit, and BF16 variants. It is the canonical download location.
Important safety notice
These checkpoints have been modified to suppress learned refusal behavior. They
may produce harmful, illegal, offensive, deceptive, or dangerously incorrect
content more readily than the upstream instruction model. Abliteration is not
truthfulness training, capability improvement, or a safety guarantee. Use these
models only where you can independently evaluate and constrain their outputs.
All variants share the pinned upstream revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0. Quantized
variants use MLX affine quantization; 2-bit uses AWQ/group 32 and 4/6/8-bit use
group 64. Their vision towers remain in BF16.
Abliteration recipe
A projected harmful-minus-harmless direction was measured from 256
length-matched prompts per class at the assistant-generation boundary.
Each variant was evaluated deterministically on 100 harmful prompts and 100
benign controls with batch size 1, thinking disabled, and a 128-token ceiling.
Variant
Harmful explicit refusals
Benign explicit refusals
Final answers present
MLX 2-bit AWQ (experimental)
0/100
0/100
196/200
MLX 4-bit
0/100
0/100
200/200
MLX 6-bit
0/100
0/100
200/200
MLX 8-bit
0/100
0/100
200/200
MLX BF16
0/100
0/100
200/200
The scorer is a transparent phrase-based explicit-refusal detector. Most
generations reached the token ceiling, so this is an early-refusal screen rather
than a full-completion evaluation. It cannot establish universal compliance or
answer quality. Machine-readable results are in
benchmarks/validation-summary.json
and each variant's validation-summary.json.
Feature validation
The 4-bit, 6-bit, 8-bit, and BF16 variants passed:
12/12 deterministic quality checks
8/8 native tool-call checks
text and image smoke tests
temporal video understanding (red->blue)
4K-context retrieval (COBALT-7319)
The 2-bit AWQ build is experimental. It measured 9/12 quality checks,
0/8 tool-call checks, passed vision but not exact-text smoke, missed the
synthetic temporal-video check, and found the 4K needle but did not return it
exactly. These limitations are preserved in the machine-readable evidence.
4K MLX performance
These are single local runs on an Apple M5 Max with 128 GB unified memory,
using mlx==0.32.0, mlx-vlm==0.6.8, batch size 1, temperature 0, seed 0,
and thinking disabled. Each prompt contained 4,105 tokens.
Variant
Prompt prefill
Generation
End-to-end
Peak MLX memory
MLX 2-bit AWQ (experimental)
411.9 tok/s
25.2 tok/s
10.62 s
16.00 GB
MLX 4-bit
641.7 tok/s
33.2 tok/s
6.68 s
21.80 GB
MLX 6-bit
548.2 tok/s
24.7 tok/s
7.87 s
29.54 GB
MLX 8-bit
579.1 tok/s
18.7 tok/s
7.58 s
37.27 GB
MLX BF16
513.9 tok/s
9.0 tok/s
9.02 s
58.29 GB
These measurements are not cross-machine performance guarantees.
