Using llama.cpp release b10068 for quantization. All K-quants and non-imatrix quants are converted directly from f16; all IQ-quants use the included imatrix.gguf, calibrated on 580 chunks of wikitext-2-raw-v1.

Original model: egibson/ThinkingCap-Qwen3.6-27B-heretic, a decensored (abliterated) version of bottlecapai/ThinkingCap-Qwen3.6-27B produced with Heretic.

Note: this model is exported with the multi-token-prediction (MTP) head excluded (--no-mtp). The base architecture's GGUF converter has an unresolved bug where the MTP head's metadata is written but its tensors are not, producing GGUF files that fail to load. Excluding it here produces a complete, correctly-loading model — MTP is a speculative-decoding speed optimization, not something the model needs to function.

Files

Filename	Quant type	File Size	Description
ThinkingCap-Qwen3.6-27B-heretic-f16.gguf	f16	50.1GB	Full 16-bit precision. Lossless conversion from the original weights; largest file, use as a source for further requantization or when you have the RAM/VRAM and want maximum fidelity.
ThinkingCap-Qwen3.6-27B-heretic-Q8_0.gguf	Q8_0	26.6GB	8.5 bpw. Essentially indistinguishable from f16 in practice; the safest choice if size isn't a major constraint.
ThinkingCap-Qwen3.6-27B-heretic-Q6_K.gguf	Q6_K	20.6GB	6.57 bpw. Very high quality, very close to f16, noticeably smaller than Q8_0.
ThinkingCap-Qwen3.6-27B-heretic-Q5_K_M.gguf	Q5_K_M	17.9GB	5.72 bpw. High quality, a good middle ground if Q6_K is too large.
ThinkingCap-Qwen3.6-27B-heretic-Q5_K_S.gguf	Q5_K_S	17.4GB	5.55 bpw. Slightly smaller than Q5_K_M with a small further quality trade-off.
ThinkingCap-Qwen3.6-27B-heretic-IQ4_XS.gguf	IQ4_XS	14.0GB	4.48 bpw, imatrix-calibrated. Usually beats Q4_K_S/Q4_0 at a similar or smaller size thanks to importance-matrix guidance — a strong pick if your loader supports IQ-quants.
ThinkingCap-Qwen3.6-27B-heretic-Q4_K_M.gguf	Q4_K_M	15.4GB	4.92 bpw. Default recommendation for most use cases — good balance of quality and size, widely compatible.
ThinkingCap-Qwen3.6-27B-heretic-Q4_K_S.gguf	Q4_K_S	14.5GB	4.63 bpw. Smaller than Q4_K_M with slightly lower quality; prefer IQ4_XS instead if your setup supports it.
ThinkingCap-Qwen3.6-27B-heretic-Q4_0.gguf	Q4_0	14.4GB	4.60 bpw. Legacy quant format (no k-quant mixing); mainly useful for older/ARM-optimized runtimes that specifically target Q4_0.
ThinkingCap-Qwen3.6-27B-heretic-Q3_K_L.gguf	Q3_K_L	13.4GB	4.26 bpw. The largest/highest-quality of the Q3_K family.
ThinkingCap-Qwen3.6-27B-heretic-IQ3_M.gguf	IQ3_M	11.7GB	3.74 bpw, imatrix-calibrated. Good quality-per-bit in the ~3-3.5bpw range.
ThinkingCap-Qwen3.6-27B-heretic-Q3_K_M.gguf	Q3_K_M	12.4GB	3.95 bpw. Noticeable quality loss begins here; usable for memory-constrained setups.
ThinkingCap-Qwen3.6-27B-heretic-IQ3_XS.gguf	IQ3_XS	11.1GB	3.56 bpw, imatrix-calibrated. Smaller than Q3_K_M at comparable or better quality.
ThinkingCap-Qwen3.6-27B-heretic-Q3_K_S.gguf	Q3_K_S	11.2GB	3.59 bpw. Not recommended unless you specifically need a non-IQ 3-bit quant; IQ3_XS is smaller and generally better.
ThinkingCap-Qwen3.6-27B-heretic-Q2_K.gguf	Q2_K	10.0GB	3.18 bpw. Significant quality loss; only for very constrained hardware.
ThinkingCap-Qwen3.6-27B-heretic-IQ2_M.gguf	IQ2_M	9.3GB	2.97 bpw, imatrix-calibrated. Relatively usable for a ~3bpw quant thanks to imatrix guidance, but still noticeably lossy — a 27B model at this size will generally beat a much smaller model at higher precision, but expect real degradation.
ThinkingCap-Qwen3.6-27B-heretic-IQ2_S.gguf	IQ2_S	8.7GB	2.78 bpw, imatrix-calibrated. Smallest quant offered here. Extreme compression — only worth it if nothing larger fits your hardware at all.
mmproj-ThinkingCap-Qwen3.6-27B-heretic-f16.gguf	f16 (vision projector)	0.9GB	Required alongside any text-model file above for image/video input.
imatrix.gguf	—	13MB	Importance matrix used for the IQ-quants above, calibrated on wikitext-2-raw-v1 (580 chunks). Provided so you can produce your own custom quants without recomputing it.

Usage

This is a multimodal model — pass the mmproj file alongside any text-model file to enable image/video input:

llama-server -m ThinkingCap-Qwen3.6-27B-heretic-Q4_K_M.gguf --mmproj mmproj-ThinkingCap-Qwen3.6-27B-heretic-f16.gguf

Text-only usage works without --mmproj.

Downloading using huggingface-cli

hf download egibson/ThinkingCap-Qwen3.6-27B-heretic-GGUF --include "ThinkingCap-Qwen3.6-27B-heretic-Q4_K_M.gguf" "mmproj-ThinkingCap-Qwen3.6-27B-heretic-f16.gguf" --local-dir ./

Which file should I choose?

If you're not sure, start with Q4_K_M — the best balance of quality and size for most hardware, and the most widely compatible format. If your runtime supports IQ-quants, IQ4_XS is a strong alternative at a similar size with imatrix-guided quality. Move up to Q5/Q6/Q8/f16 if you have the RAM/VRAM to spare, or down to the 3-bit/2-bit tiers only if nothing larger fits.