This is a GGUF conversion of the Qwen3-4B-Instruct-2507-heretic model, which is an abliterated (uncensored) version of Alibaba's Qwen2.5 4B Instruct model. The model has had its refusal mechanisms removed, making it more willing to engage with any prompt.

Original Model: p-e-w/Qwen3-4B-Instruct-2507-heretic Collection: The Bestiary by p-e-w

Quantization Formats

This repository contains 4 quantization levels:

File	Size	Description	Use Case
`qwen3-4b-instruct-2507-heretic-f16.gguf`	7.5GB	Full 16-bit precision	Best quality, highest memory usage
`qwen3-4b-instruct-2507-heretic-Q8_0.gguf`	4.0GB	8-bit quantization	High quality, good balance
`qwen3-4b-instruct-2507-heretic-Q5_K_M.gguf`	2.7GB	5-bit quantization	Balanced quality/size
`qwen3-4b-instruct-2507-heretic-Q4_K_M.gguf`	2.4GB	4-bit quantization	Smallest size, good quality

Recommended: Q4_K_M for most users (best balance of quality and size)

Usage

With Ollama

Download the GGUF file you want to use
Create a Modelfile:

FROM ./qwen3-4b-instruct-2507-heretic-Q4_K_M.gguf

TEMPLATE """{{ if .System }}<|im_start|>system<|im_sep|>{{ .System }}<|im_end|>{{ end }}{{ if .Prompt }}<|im_start|>user<|im_sep|>{{ .Prompt }}<|im_end|>{{ end }}<|im_start|>assistant<|im_sep|>{{ .Response }}<|im_end|>"""

PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER num_ctx 8192

Import to Ollama:

ollama create qwen3-4b-heretic:Q4_K_M -f Modelfile

Run:

ollama run qwen3-4b-heretic:Q4_K_M

With llama.cpp

./llama-cli -m qwen3-4b-instruct-2507-heretic-Q4_K_M.gguf -p "Your prompt here" -n 512

With Open WebUI

Once imported to Ollama, the model will automatically appear in the Open WebUI model dropdown.

Conversion Details

Converted using: llama.cpp (latest)
Conversion date: 2025-11-21
Base format: FP16 GGUF
Quantization method: llama-quantize

Important Note

This is an uncensored model with refusal mechanisms removed. Use responsibly and in accordance with applicable laws and regulations.

License

Inherits the Apache 2.0 license from the base Qwen model.

Credits

Original model: Alibaba (Qwen2.5)
Abliteration: p-e-w (The Bestiary)
GGUF conversion: cybrown

logos-flux/Qwen3-4B-Instruct-2507-heretic-GGUF

Qwen3-4B-Instruct-2507-heretic-GGUF

Model Description