Converted with convert_hf_to_gguf.py --no-mtp from llama.cpp build 9658. The fine-tune does not include MTP head weights (dropped during training), so MTP is not available in this GGUF.

Available Quantisations

File	Size	Type
OpenMythos-27B-F16.gguf	53.8 GB	F16
OpenMythos-27B-Q5_K.gguf	18.3 GB	Q5_K_M
OpenMythos-27B-Q4_K.gguf	15.4 GB	Q4_K_M
OpenMythos-27B-Q6_K.gguf	21.2 GB	Q6_K

Benchmark

Evaluated with SecEval (commit 7aef317) on 2189 multiple-choice security questions. Backend: llama.cpp OpenAI-compatible server, fully offloaded to GPU. No chain-of-thought / reasoning enabled (enable_thinking=false).

Prompt formatted with a system prompt requesting letter-only answers (no explanation).

Set	Model	Score
A	OpenMythos-27B-Q5_K	1703 / 2189 (77.8%)
B	VulnLLM-R-7B	1315 / 2189 (60.1%)

OpenMythos-27B-Q5_K test parameters

model: OpenMythos-27B-Q5_K.gguf
inference: temp=0.2, top_p=0.8, top_k=20, min_p=0.05, repeat_penalty=1.02
benchmark script: /mnt/storage/SecEval-tmp/run_bench.py
output: seceval-1781809723.json
prompt speed: 282 tok/s | generation speed: 68 tok/s

Per-topic scores

Topic	Score
PenTest	84.2%
MemorySafety	83.3%
WebSecurity	82.7%
Vulnerability	77.8%
NetworkSecurity	77.4%
SoftwareSecurity	75.0%
ApplicationSecurity	74.8%
SystemSecurity	73.6%
Cryptography	71.4%

VulnLLM-R-7B test parameters

model: VulnLLM-R-7B.Q6_K.gguf
inference: same settings as above
output: seceval-1781811525.json
prompt speed: 148 tok/s | generation speed: 39 tok/s

Per-topic scores

Topic	Score
PenTest	70.9%
WebSecurity	66.4%
Vulnerability	58.7%
NetworkSecurity	58.3%
SystemSecurity	56.4%
SoftwareSecurity	54.7%
ApplicationSecurity	54.7%
MemorySafety	54.2%
Cryptography	28.6%

Full detailed results are included in this repo: seceval-1781809723.json and seceval-1781811525.json.

Usage

llama-server (recommended)

ini

[OpenMythos-27B]
model = /mnt/storage/models/OpenMythos/OpenMythos-27B-Q5_K.gguf
chat-template-file = /mnt/storage/llama-server/chat_template-v15.jinja
ctx-size = 65536
cache-type-k = q8_0
cache-type-v = q8_0
cache-prompt = on
cache-reuse = 2048
batch-size = 4096
ubatch-size = 4096
kv-unified = on
parallel = 1
gpu-layers = all
temp = 0.2
top-p = 0.8
top-k = 20
min-p = 0.05
presence-penalty = 0.2
repeat-penalty = 1.02
spec-type = ngram-mod
spec-draft-n-max = 5
reasoning-format = deepseek
swa-checkpoints = 5

llama-cli

bash

/mnt/storage/llama.cpp/build/bin/llama-cli \
  -m /mnt/storage/models/OpenMythos/OpenMythos-27B-Q5_K.gguf \
  --chat-template-file /mnt/storage/llama-server/chat_template-v15.jinja \
  -c 65536 -b 4096 --ubatch-size 4096 \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --kv-unified -t 8 -fa \
  --temp 0.2 --top-p 0.8 --top-k 20 --min-p 0.05 \
  --presence-penalty 0.2 --repeat-penalty 1.02 \
  -ngl all \
  -p "Your prompt here"

educatedlucifer12/OpenMythos-GGUF

OpenMythos 27B - GGUF

Available Quantisations

Benchmark

OpenMythos-27B-Q5_K test parameters

Per-topic scores

VulnLLM-R-7B test parameters

Per-topic scores

Usage

llama-server (recommended)

llama-cli

educatedlucifer12/
OpenMythos-GGUF