Qwen3.5-27B-Abliterated-IQ4_XS.gguf
Prior to abliterating this model, I was using my
Qwen2.5-14B-Instruct-Abliterated model.
The 14B Q8_0 quant was the only version of my abliterated models I could get to reliably call tools.
The 7B would output JSON to chat. The 32B stole my youth before calling tools because of how far I had to quantize it.
This model blows BOTH out of the water as far as agentic tasks. Reliable tool calls. Good understanding and reasoning. No refusals encountered as of yet.
I was very surprised by how resistant this model was to abliteration, I was expecting it to become incoherent before stopping refusals, but instead, it seems to have maintained
all coherence and capabilities through the process including agentic tasks and tool callng.
I can ask it to implement entire features in my apps and walk away, and when I've returned I have a new feature. I'm using Qwen-Agent SDK to build my app
Atomic Chat.
I recently moved from an Ollama server to llama.cpp server without the extra fluff as this model just about maxes out my 4080 Super. I'm running 56 of 64 layers on GPU, 8 on CPU.
IQ4_XS quantization of
sci4ai/Qwen3.5-27B-Abliterated, built with a self-generated importance matrix.
File
| Field | Value |
|---|
| Quantization | IQ4_XS |
| Average bpw | ~4.25 |
| File size | 14.05 GiB |
| Context length | 32768 (native) |
| Architecture | Qwen3.5 (hidden 5120, 64 layers, vocab 248320) |
Quantization method
Quantized with
llama.cpp using
llama-quantize + a self-generated importance matrix (imatrix) computed directly from the abliterated BF16 source. The imatrix is included in this repo as
Qwen3.5-27B.imatrix.
Important: using a community imatrix generated from the stock (non-abliterated) model would partially restore the refusal direction this model was trained to remove. Always quantize abliterated models with an imatrix built from their own weights.
Recommendations
Minimum VRAM for full GPU offload: ~14.5 GiB weights + KV cache. Fits a single 16 GiB GPU only when the display is not sharing the same card (e.g. integrated graphics output, headless server). For a desktop with compositor on the same GPU, budget partial layer offload or use q8_0 KV cache.
Recommended llama.cpp / llama-server flags:
1llama-server \
2 --model Qwen3.5-27B-Abliterated-IQ4_XS.gguf \
3 --jinja \
4 --reasoning off \
5 --flash-attn on \
6 --cache-type-k q8_0 \
7 --cache-type-v q8_0 \
8 -c 12288 \
9 -ngl 99
--ngl — For my 16GB VRAM I'm using --ngl 56 and haven't seen much of a slow down at all. Model calls tools pretty much immediately.--jinja — use the GGUF's embedded Jinja chat template (required for correct Qwen3 tool-call handling).--reasoning off — suppress Qwen3 <think> blocks for agent and chat use. Set to auto if you want reasoning traces.--cache-type-k/v q8_0 — halves KV cache size vs f16 with negligible perplexity impact; safe at IQ4_XS.
Abliteration source
See the source model card at
sci4ai/Qwen3.5-27B-Abliterated for abliteration methodology, ablated layer range, and calibration details. In brief: refusal direction removed via orthogonalization of
o_proj and
down_proj weights against per-layer refusal directions computed from harmful/harmless prompt contrast.
Disclaimer
This model is provided for research purposes. The removal of safety guardrails means it will comply with requests that the original model would refuse. Users are responsible for how they use this model.
