GGUF quantizations of exploitintel/cve-cwe-gemma4-12b — a Gemma 4 12B fine-tune that maps a CVE description to its CWE ID(s) (MITRE View-1003).

📖 Write-up: From Essays to CWE-319 — how this fine-tune beats stock Gemma 4 at CWE classification

Input: a free-text vulnerability description.
Output: comma-separated CWE IDs, e.g. CWE-79 or CWE-89, CWE-352.
Text-only build (vision/audio projectors omitted).

Files

file	quant	size	notes
`cve-cwe-gemma4-12b-Q8_0.gguf`	Q8_0	11.8 GB	recommended — near-lossless
`cve-cwe-gemma4-12b-Q4_K_M.gguf`	Q4_K_M	6.87 GB	smaller/faster; some rare-CWE loss (see below)

Requires a current llama.cpp / Ollama build with Gemma 4 (gemma4_unified) support.

Quality vs the full model

Held-out test split (10,514 examples), description-only, greedy. The bf16 column is the unquantized transformers model; Q8/Q4 are measured through Ollama.

metric	bf16	Q8_0	Q4_K_M
exact-match	0.714	0.697	0.682
micro-F1	0.756	0.732	0.718
macro-F1	0.538	0.500	0.429
easy exact	0.805	0.808	0.793
hard exact	0.644	0.611	0.595

Choosing a quant: Q8_0 is effectively lossless. Q4_K_M costs only ~1.5 points on exact/micro but ~7 points on macro-F1 — i.e. it degrades the rare-CWE long tail specifically. Use Q8_0 if rare/long-tail CWEs matter; use Q4_K_M if size/speed dominate and you mostly see common CWEs.

Usage — Ollama

bash

# build the model from the GGUF (see the Modelfile below)
ollama create cve-cwe-gemma4 -f Modelfile

# IMPORTANT: gemma4 runs with "thinking" ON by default, which wastes ~100-900
# tokens per query for this single-label task. Disable it:
ollama run cve-cwe-gemma4
>>> /set nothink
>>> The update handler transmits user credentials over an unencrypted HTTP channel.
CWE-319

Via the API, pass "think": false (the response field is the clean answer):

bash

curl -s http://localhost:11434/api/generate -d '{
  "model": "cve-cwe-gemma4",
  "prompt": "SQL injection via the username parameter in the login form.",
  "stream": false,
  "think": false
}'
# -> {"response":"CWE-89", ...}

Modelfile

dockerfile

FROM ./cve-cwe-gemma4-12b-Q8_0.gguf

TEMPLATE """{{ if .System }}<|turn>system
{{ .System }}<turn|>
{{ end }}<|turn>user
{{ .Prompt }}<turn|>
<|turn>model
<|channel>thought
<channel|>{{ .Response }}<turn|>
"""

SYSTEM """You are a vulnerability analyst. Given a CVE description, reply with only the CWE ID(s) it maps to, comma-separated."""

PARAMETER temperature 0
PARAMETER stop "<turn|>"
PARAMETER stop "<|turn>"

Usage — llama.cpp

bash

llama-cli -m cve-cwe-gemma4-12b-Q8_0.gguf --jinja \
  -sys "You are a vulnerability analyst. Given a CVE description, reply with only the CWE ID(s) it maps to, comma-separated." \
  -p "A heap-based buffer overflow occurs when parsing a crafted font file."

Limitations

Description-only triage aid; outputs are suggestions, not authoritative classifications. Keep a human in the loop for security-relevant decisions. Scope is MITRE View-1003. See the base model card for training details.

License

Apache-2.0, inherited from the Gemma 4 base model.

exploitintel/cve-cwe-gemma4-12b-GGUF

cve-cwe-gemma4-12b-GGUF

Files

Quality vs the full model

Usage — Ollama

Modelfile

Usage — llama.cpp

Limitations

License

exploitintel/
cve-cwe-gemma4-12b-GGUF