Vision-Stripped (Text-Only): All vision-related tensors (mm_proj, visual, etc.) were stripped from the original .safetensors prior to conversion. This makes these GGUF files significantly lighter, faster to load, and perfectly optimized for text-based inference and Roleplay.
Extreme Imatrix Calibration: The importance matrix (imatrix.dat) was calculated using a dedicated Roleplay dataset (calib_rp.txt). To ensure the absolute highest precision and retention of character reasoning and uncensored creativity, the calibration was run for an exhaustive ~8,000 chunks.
Custom "Large" Variants: Included are handcrafted Q4_K_L and Q5_K_L variants, created by forcing the most sensitive components (Embeddings and Output tensors) to 8-bit precision (q8_0), bridging the gap between medium quants and Q6_K.

Provided Quants

(Files are sorted by size. For roleplay, the IQ4_K_M or Q4_K_L are highly recommended for a great balance of speed and intelligence).

File Name	Size (GB)	Notes
Qwen3.6-27B-uncensored-heretic-v2-Q6_K.gguf	21.56	Practically identical to FP16. Highest quality.
Qwen3.6-27B-uncensored-heretic-v2-Q5_K_L.gguf	19.55	Custom large variant (Q8_0 embeddings/output).
Qwen3.6-27B-uncensored-heretic-v2-Q5_K_M.gguf	18.78	Very high fidelity, recommended.
Qwen3.6-27B-uncensored-heretic-v2-IQ5_K_S.gguf	18.24	Imatrix optimized 5-bit.
Qwen3.6-27B-uncensored-heretic-v2-Q4_K_L.gguf	17.08	Custom large variant (Q8_0 embeddings/output).
Qwen3.6-27B-uncensored-heretic-v2-IQ4_K_M.gguf	16.16	Recommended. Best size/speed/quality ratio.
Qwen3.6-27B-uncensored-heretic-v2-Q4_K_S.gguf	15.22	Fast and reliable standard 4-bit.
Qwen3.6-27B-uncensored-heretic-v2-IQ4_XS.gguf	14.73	Highly optimized lower-end 4-bit.
Qwen3.6-27B-uncensored-heretic-v2-Q3_K_M.gguf	12.99	Great for systems with limited RAM/VRAM.
Qwen3.6-27B-uncensored-heretic-v2-Q3_K_S.gguf	11.79	Lower quality 3-bit.
Qwen3.6-27B-uncensored-heretic-v2-Q2_K.gguf	10.46	Heavy compression, noticeable perplexity loss.
Qwen3.6-27B-uncensored-heretic-v2-Q2_K_S.gguf	10.01	Maximum space saving, for the desperate.

DIY: How to create your own High-Quality Quants

For fellow adventurers wanting to quantize their own models at home, here is a mini-guide on how this specific workflow was executed using llama.cpp on Windows:

1. Strip Vision Tensors (For Text-Only Inference) If the base model is multimodal, you save space by removing vision weights. Use a simple Python script with the safetensors library:

python

from safetensors.torch import load_file, save_file
# Load each .safetensors file, find keys containing "vision", "visual", "mm_proj", etc., delete them, and save.

2. Convert to FP16 Use the convert_hf_to_gguf.py script provided in the llama.cpp repository:

python llama.cpp/convert_hf_to_gguf.py ./base_model --outfile model_fp16.gguf --outtype f16

3. Calculate the Importance Matrix (Imatrix) Download a solid calibration dataset (like calib_rp.txt for roleplay). Limit the GPU layers (-ngl) and CPU threads (-t) to prevent your system from crashing during this heavy process:

llama-imatrix.exe -m model_fp16.gguf -f calib_rp.txt -o imatrix.dat -c 512 --gpu-layers 12 -t 8

4. Final Quantization Apply the imatrix to generate your desired formats:

bash

# Standard
llama-quantize.exe --imatrix imatrix.dat model_fp16.gguf result_IQ4_K_M.gguf IQ4_K_M

# Custom Large (Q8_0 embeddings)
llama-quantize.exe --imatrix imatrix.dat --token-embedding-type q8_0 --output-tensor-type q8_0 model_fp16.gguf result_Q4_K_L.gguf Q4_K_M

A Personal Note

This repository is a personal project. It was my very first time diving into the world of LLM quantization, compiling GGUFs, and calculating importance matrices. I've put a lot of effort, time (and a couple of system crashes!) into making sure the process was done right and that the quality of these files is as high as possible. While I am just starting out and there's always room to learn, I hope these models serve you well.

Happy Roleplaying!

Jolex98/Qwen3.6-27B-uncensored-heretic-v2-GGUF

About

Provided Quants

DIY: How to create your own High-Quality Quants

A Personal Note

Jolex98/
Qwen3.6-27B-uncensored-heretic-v2-GGUF