AWQ 4-bit (W4A16) quantization of llmfan46/Qwen3.6-35B-A3B-uncensored-heretic — an uncensored ("heretic"/abliterated) variant of Qwen3.6-35B-A3B (qwen3_5_moe: a hybrid GatedDeltaNet (linear-attention) + full-attention Mixture-of-Experts with 256 routed experts (8 active) + a shared expert, plus a vision tower and an MTP head).

Variant: AWQ W4A16 — 4-bit symmetric integer weights, group size 128, with activation-aware scaling. Activations stay BF16. Disk size: ~21.8 GB (vs ~70 GB BF16, ~3.2×) Quantized by: sahilchachra Tooling: llm-compressor 0.12 (AWQModifier + QuantizationModifier) → compressed-tensors pack-quantized

Why this source: the popular HauhauCS/Qwen3.6-35B-A3B-Uncensored-* repos ship only GGUF (no full-precision weights), which cannot be honestly AWQ-quantized. This checkpoint quantizes a genuine BF16 uncensored Qwen3.6-35B-A3B instead.

What is quantized

Quantized to int4 (30,880 modules):

routed experts mlp.experts.{0..255}.{gate,up,down}_proj on all 40 layers — 30,720
shared expert mlp.shared_expert.{gate,up,down}_proj on all 40 layers — 120
full-attention self_attn.{q,k,v,o}_proj on the 10 full-attention layers — 40

Kept in BF16 (not quantized): the GatedDeltaNet linear_attn (mamba) layers, the MoE router (mlp.gate) and shared_expert_gate, the vision tower, the MTP head, token embeddings, lm_head, and all norms.

Calibration

AWQ is calibration-based. Calibrated on 32 sequences × 512 tokens of general instruction-following chat from HuggingFaceH4/ultrachat_200k, rendered through the model's chat template. Because this is the base instruct model with refusals abliterated (not a task-specific fine-tune), general chat best matches its inference distribution. During calibration every one of the 256 experts sees all calibration tokens (moe_calibration_context), so per-expert AWQ scales are well-estimated despite sparse 8/256 routing.

Loading / serving

This checkpoint stores experts in the per-expert compressed-tensors (pack-quantized) layout that vLLM consumes for qwen3_5_moe. Serve with vLLM:

python

from vllm import LLM, SamplingParams
llm = LLM(model="sahilchachra/Qwen3.6-35B-A3B-uncensored-heretic-AWQ",
          trust_remote_code=True, dtype="bfloat16")
print(llm.generate(["Write a quicksort in Python."], SamplingParams(max_tokens=256))[0].outputs[0].text)

Notes

Text-only: the source repo nests its vision-tower weights under a non-standard key prefix (model.language_model.visual.* instead of model.visual.*), so the vision tower does not load functionally. This is a quantization of the language model; treat it as a text-generation model.

GatedDeltaNet kernels (causal-conv1d / flash-linear-attention) fall back to a pure-PyTorch path where unavailable (e.g. ARM/Jetson) — correct, just slower.

compressed-tensors == 0.17.1's eager decompress_model mis-reads group_size for pack-quantized checkpoints; load via vLLM (its own loader) or a fixed compressed-tensors. The weights/scales on disk are standard and correct.

Uncensored

This is an abliterated / uncensored model (refusal directions removed). It will follow instructions it would otherwise decline. Use responsibly and in compliance with applicable law and the Apache-2.0 / Qwen terms.

Original model

See llmfan46/Qwen3.6-35B-A3B-uncensored-heretic and base Qwen/Qwen3.6-35B-A3B for architecture, intended use, and license.