Qwen-3.8-27B-Uncensored-8-Bit-MLX

What this file is not

• Not unique for vision-bf16-in-8bit. Other 8-bit MLX VLMs already leave the tower unquantized (including orcarouter/Qwen3.8-27B-Uncensored-MLX, mlx-community/Qwen3.8-27B-8bit, PocketAiHub Abliterated-MLX, avlp12 Alis-8bit). This card documents the same pattern. It does not claim first or only.
• Not an in-weight MTP 8-bit. mlx-vlm drops mtp.*. If you need native MTP, use the bf16 parent or the optional draft head (acceptance on hard prose is unmeasured).
• Not a speed SKU. This card publishes no Apple tok/s. Other 8-bit VLM cards have published their own numbers on their hardware (for example avlp12 Alis-8bit: 21.8 tok/s on an M3 Ultra; PocketAiHub Abliterated 8-bit: 18.7 tok/s on an M5 Max). Those are their measurements. Do not treat them as this repo.

Intended use: red teaming and defensive cybersecurity research

• Capability-ceiling measurement. Upper-bound what the weights can actually produce in a domain, independent of shipped refusals.
• Defensive-stack evaluation. Test input filters, output classifiers, prompt-injection defenses, and moderation APIs when the model itself contributes no refusals. That is how you find gaps in a defensive control plane.
• Attack-surface isolation. Automated red-team loops stall on unrelated refusals. A non-refusing target isolates the control under test (injection, tool abuse, data-exfil paths, policy bypass).
• Detection and classifier work. Generate labeled completions for training or benchmarking output-moderation and abuse-detection models.
• Interpretability of residual refusal. Abliteration is a specified rank-1 edit on a known layer span. The pair (base vs this) is a clean experimental control.

Family

Usage

LM Studio

Hardware

This checkpoint is MLX-only. It will not load with transformers, vLLM, or llama.cpp. For those, use the bf16 parent. Native MTP pairing (optional, unmeasured) uses qwen38-mtp-head-fc-bf16-4bit.

Attribution

Base model — Qwen

Abliteration — ZeroFuse

Quantization — MLX

What changed

Quantization applied on top

Method

1. Direction estimation. Run harmless and harmful prompt sets through the model and cache residual activations at every layer. The difference in means between the two populations, per layer, gives a candidate refusal direction.
2. Projected refinement. Subtract only the component of the refusal direction that is orthogonal to the harmless mean (project_out_harmless = true). This reduces collateral damage relative to naive difference-of-means.
3. Two-objective search. An Optuna study jointly minimizes refusal rate on a held-out harmful set and KL divergence from the base model on a held-out harmless set. The search space covers the source layer, the ablation strength, and the span of layers to edit.
4. Weight orthogonalization. For the winning trial, project the refusal direction out of every targeted weight matrix and serialize the result as a standard checkpoint.

Run configuration

Reproducing this build

• the 851 abliterated language-model tensors, exactly as ZeroFuse wrote them, and
• the 333 vision and 15 mtp.* tensors copied bit-for-bit from the base model, together with the original config.json, preprocessor_config.json, and video_preprocessor_config.json.

Results

These figures were measured on the bf16 parent, not on this quantized checkpoint. Quantization is a lossy numerical transform applied after the measurements above. It is expected to shift behavior only marginally at 8-bit, but the refusal rate and KL divergence reported here have not been re-measured post-quantization. If exact numbers matter for your work, re-run the evaluation against this checkpoint rather than inheriting the parent's.

Refusal behavior: this model vs. the original

Measured

Read this number carefully. The base model refused only 14 of 64 harmful prompts to begin with — a baseline refusal rate of 21.9%. "Zero refusals" therefore means every refusal the base model actually exhibited on this set was removed, measured against a baseline that was already fairly permissive. It does not mean the model has been tested against, and complies with, a broad or adversarial harmful-prompt distribution. During optimization the refusal objective was scored over those 14 base-refused prompts specifically, since the remainder carry no refusal signal to remove.

What this model still refuses

• Multi-turn re-assertion. Refusal can re-emerge over long conversations as context accumulates, even when the same request is answered in a single turn.
• System-prompt-driven refusal. A restrictive system prompt still steers behavior. Abliteration edits weights, not instruction-following — telling this model to decline things still works.
• Strongly-memorized refusal phrasings. Requests whose refusal was heavily reinforced during safety tuning can persist, particularly where the refusal is entangled with factual knowledge rather than expressed purely through the refusal direction.
• Soft refusals. Deflection, moralizing preambles, deliberate vagueness, and "I can discuss this in general terms" hedging are frequently not counted as refusals by automated scoring, and often survive when hard refusals do not.
• Vision-path refusals. The vision tower was not modified. Refusals triggered by image content route partly through unedited weights and are correspondingly less affected.

Note on the numbers below. The residual refusals above are described from the known behavior of the method, not from a category-by-category probe of this specific checkpoint. The only empirical claim in this section is the measured table. A per-category breakdown requires running a labeled probe against the finished weights; until that is published here, treat the categories as expectations to verify, not as measurements.

Comparison methodology

Red teaming and safety research

Why an abliterated variant is useful

• Dangerous-capability evaluation. Use this model to establish an upper bound on what the Qwen3.8 weights can actually produce in a given domain, independent of whether the shipped model would agree to. This is the standard argument for evaluating helpful-only variants alongside safety-tuned ones.
• Testing your own guardrails under worst case. If your deployment relies on input filters, output classifiers, or a moderation API, the base model's refusals hide gaps in that stack. Swapping in this model removes the mask and shows what your moderation layer catches when the model itself contributes nothing.
• Attack and jailbreak research. Automated red-team loops stall when the target refuses for reasons unrelated to the attack under test. A non-refusing target isolates the variable you are actually studying.
• Classifier training and evaluation. Generating harmful-completion corpora for training or benchmarking output moderation models is difficult with a refusing generator.
• Interpretability of refusal circuits. This is the strongest use. This checkpoint and the base model differ by a single, fully-specified rank-1 projection applied to a known span of layers (source layer 35, strength 1.2242, layers 9–56) — every other parameter is bit-identical. That makes the pair a clean experimental control for studying how refusal is represented, where it is written, and what remains when the primary direction is removed.
• Studying the residual. The refusals that survive abliteration are arguably more informative than the ones that don't: they mark the parts of refusal behavior that a single direction fails to explain.

Operating recommendations

• Run it in a contained environment. Do not expose it as a public endpoint without an independent moderation layer in front of it — it will not refuse on your behalf.
• Log prompts and completions for anything you intend to publish; automated refusal classifiers disagree with human labels often enough that spot-checking matters.
• Always report the base model alongside it. A number from this checkpoint alone is not interpretable; the delta against Qwen/Qwen3.8-27B is the actual result.
• Re-measure rather than trusting this card. The figures here come from a 100-trial optimization on one prompt set. Your domain, prompts, and scoring will differ.

Model details

Limitations and caveats

• Abliteration is statistical, not a guarantee. The refusal direction is an approximation. Some refusals survive; some phrasings will still trigger them. The measured refusal rate above is the rate on one specific held-out set, not a universal property.
• The evaluation is narrow. Scoring used the 64-prompt held-out split of mlabonne/harmful_behaviors, of which the base model refused 14. The optimizer's refusal objective was computed over only the 14 prompts the base model refused, so a single prompt flipping moves that metric by 7.1 percentage points. Treat the refusal figure as directional evidence from one prompt distribution, not a precise or general measurement.
• Removing refusals does not add knowledge. The model is no more accurate, and no less prone to hallucination, than the base model. It will now answer confidently in domains where it is simply wrong.
• Safety behavior was deliberately removed. This model will not decline requests the base model would have declined. It carries none of the guardrails Qwen shipped it with. Whoever deploys it owns the moderation layer.
• Small capability regressions are possible. KL was minimized, not driven to zero. Benchmark against the base model for your own workload rather than assuming parity.
• Not independently benchmarked. No MMLU/GSM8K/HumanEval or vision-benchmark comparison against the base model has been run. Capability preservation is inferred from the KL objective alone.
• Quantization loss is unmeasured. 8-bit quantization is generally close to lossless in perceived quality, but no perplexity or benchmark comparison against the bf16 parent has been run for this checkpoint. The abliteration metrics above are inherited from the parent, not re-measured.
• Quantization can interact with abliteration. The refusal edit is a small, precise perturbation (relative magnitude ~1.8% on the tensors it touches). Rounding weights to 8 bits perturbs those same tensors again. There is no evidence this restores refusals at 8-bit, but it has not been ruled out by measurement either — expect the behavior to be very close to the parent, and verify if it matters.
• MLX-only. Cannot be loaded by transformers, vLLM, TGI, or llama.cpp, and the missing mtp.* head means it cannot be converted back into a complete transformers checkpoint.

Intended use

License

Citations

Provenance