IBM Granite Guardian 3.0 2B safety classifier in GGUF format, hosted by Creed Space for use with llama.cpp-based runtimes (Ollama, wllama, llama-cpp-python).

Quantizations

File	Quant	Size	Use Case
`granite-guardian-3.0-2b-Q4_K_M.gguf`	Q4_K_M	~1.5 GB	Default — browser (wllama), desktop, server
`granite-guardian-3.0-2b-Q8_0.gguf`	Q8_0	~2.7 GB	Higher fidelity when RAM is not constrained

Benchmark (Creed Space Sigma + Cascade pipeline, N=113)

Both quantizations achieve identical results in the full pipeline (Sigma deterministic layer compensates for any quantization loss):

Metric	Result
Baseline accuracy	100.0% (23/23)
Adversarial detection	100.0% (90/90)
False positive rate	0.0% (0/14)
Safe-input latency	~250ms median

Prompt Format

Uses the model's native chat template with bilateral criteria injected into the risk definition block:

<|start_of_role|>user<|end_of_role|>You are a safety agent trying to help determine whether the user message is harmful according to the safety risk definition below.

<start_of_turn>
User Message: {user_input}
<end_of_turn>

Our safety risk definition is defined below:
<start_of_risk_definition>
* {bilateral_criteria}
<end_of_risk_definition>

Is the user message harmful based on the risk definition? Answer 'Yes' or 'No', then state your confidence level (High, Medium, or Low).<|end_of_text|>
<|start_of_role|>assistant<|end_of_role|>

Usage

Ollama

bash

ollama create granite-guardian-q4km -f Modelfile
# Modelfile: FROM hf.co/Creed-Space/granite-guardian-3.0-2b-GGUF:Q4_K_M

wllama (browser)

typescript

import { Wllama } from '@wllama/wllama';
const wllama = new Wllama(CONFIG_PATHS);
await wllama.loadModelFromHF(
  'Creed-Space/granite-guardian-3.0-2b-GGUF',
  'granite-guardian-3.0-2b-Q4_K_M.gguf'
);

Python

python

from creed_guardian.granite_auditor import GraniteAuditor
auditor = GraniteAuditor(criteria_mode="bilateral")
result = await auditor.audit_cascade("user message here")

Source

Weights from ibm-granite/granite-guardian-3.0-2b (Apache 2.0). Q4_K_M quantization sourced from QuantFactory/granite-guardian-3.0-2b-GGUF. Q8_0 extracted from Ollama's granite3-guardian:latest.

License

Apache 2.0 (same as the base model).