The parent card carries what this model is, how it was made, what abliteration did to it, and the evaluation numbers. Read it before using these weights: this model does not refuse, which is the entire point of it and the thing to understand before downloading.

Files

File	Precision	Size	Use when
`llama-3.2-1b-f16.gguf`	F16	larger	You want the conversion with no quantisation loss, or you are making your own quants
`llama-3.2-1b-Q4_K_M.gguf`	Q4_K_M	~4x smaller	Almost always. The usual quality and size compromise

Both were converted from the parent's safetensors with convert_hf_to_gguf.py and quantised with llama-quantize. Each file was loaded and asked to generate before publication, because a GGUF that converts but does not run is exactly the kind of thing that ships broken.

Usage

llama.cpp

llama-server -m llama-3.2-1b-Q4_K_M.gguf -c 4096

Ollama

ollama run hf.co/ops-malware/llama-3.2-1b-abliterated-GGUF:Q4_K_M

Python, via huggingface_hub

python

from huggingface_hub import hf_hub_download

path = hf_hub_download(
    repo_id="ops-malware/llama-3.2-1b-abliterated-GGUF",
    filename="llama-3.2-1b-Q4_K_M.gguf",
)

Limitations

Everything on the parent card applies here unchanged, plus the usual quantisation caveat: Q4_K_M trades some quality for size, and small models have less quality to spare than large ones. If a result matters, check it against the F16.