GGUF quantizations of Qwen/Qwen3.6-27B with refusal behavior removed via abliteration. For use with llama.cpp, Ollama, LM Studio, and other GGUF-compatible runtimes.

Blog post: Abliteration: Uncensoring LLMs via Weight Surgery

Standard Quantizations (No MTP)

Maximum compatibility — works with Ollama, LM Studio, KoboldCPP, llama.cpp out of the box.

File	Size	RAM Required	Notes
`...-F16.gguf`	53.8 GB	64+ GB	Full precision
`...-Q4_K_M.gguf`	26.9 GB	32+ GB	Recommended (F16 vision embed)
`...-Q4_K_M-Q8.gguf`	19.7 GB	24+ GB	Q4 language + Q8 vision embed

MTP Quantizations (Multi-Token Prediction)

Include MTP draft head for speculative decoding. Larger files but faster inference with compatible runtimes.

File	Size	RAM Required	Notes
`...-MTP-F16.gguf`	54.7 GB	64+ GB	Full precision + MTP
`...-MTP-Q8_0.gguf`	29 GB	36+ GB	Near-lossless + MTP
`...-MTP-Q6_K.gguf`	22.4 GB	32+ GB	Very high quality + MTP
`...-MTP-Q5_K.gguf`	19.5 GB	24+ GB	Recommended for 48GB + MTP
`...-MTP-Q4_K.gguf`	16.8 GB	20+ GB	Good quality + MTP
`...-MTP-Q3_K.gguf`	13.5 GB	16+ GB	16GB VRAM + MTP
`...-MTP-Q2_K.gguf`	10.9 GB	12+ GB	2-bit + MTP

Vision Encoder

File	Size	Notes
`...-mmproj-f16.gguf`	928 MB	Required for multimodal (image understanding)

MTP (Multi-Token Prediction)

MTP files include a draft head for speculative decoding — faster token generation:

bash

./llama-server -m Qwen3.6-27B-abliterated-MTP-Q5_K.gguf \
  --jinja --spec-type draft-mtp --spec-draft-n-max 6 -ngl 99

Runtime compatibility: MTP requires llama-server b9180+. Ollama does not support MTP yet. Use the standard (non-MTP) files for Ollama/LM Studio.

Multimodal (Vision)

The mmproj-f16.gguf file is the vision encoder for image understanding:

bash

./llama-mtmd-cli -m Qwen3.6-27B-abliterated-Q4_K_M.gguf \
  --mmproj Qwen3.6-27B-abliterated-mmproj-f16.gguf \
  -p "Describe this image" --image photo.jpg

Quickstart — Ollama

bash

# Recommended for Apple Silicon 48GB+ (M4 Pro, M4 Max)
ollama run hf.co/Bahushruth/Qwen3.6-27B-abliterated-GGUF:Q4_K_M

# For 24GB systems
ollama run hf.co/Bahushruth/Qwen3.6-27B-abliterated-GGUF:Q4_K_M-Q8

Usage — llama.cpp

bash

huggingface-cli download Bahushruth/Qwen3.6-27B-abliterated-GGUF \
  Qwen3.6-27B-abliterated-Q4_K_M.gguf --local-dir .

./llama-cli -m Qwen3.6-27B-abliterated-Q4_K_M.gguf \
  -p "You are a helpful assistant." \
  --chat-template chatml -cnv -c 262144

Architecture Notes

Qwen3.6-27B is a dense hybrid-attention multimodal model:

Hybrid attention: 16 blocks of (3x Gated DeltaNet + 1x Gated Attention) = 64 layers
Parameters: 27B dense (all active per token)
Hidden size: 5120
Context: 262K native, extensible to 1M+ via YaRN
Multimodal: Vision encoder for image understanding
MTP: Multi-token prediction draft head for speculative decoding

Disclaimer

This model has had safety guardrails removed. Released for research purposes. The creator assumes no responsibility for downstream use.