🔧 Runtime: build the ROCmFPX fork below

Stock llama.cpp will not load this file. You need both the mellum architecture and the ROCmFP4 tensor types in one tree. Upstream charlie12345/ROCmFPX has the ROCmFP4 types but not mellum. Our fork has both:

kingjones30/ROCmFPX — a fork of charlie12345/ROCmFPX, branch main.

bash
1git clone https://github.com/kingjones30/ROCmFPX.git 2cd ROCmFPX 3cmake -B build -DGGML_HIP=ON -DGPU_TARGETS=gfx1151 -DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release 4cmake --build build --target llama-server llama-quantize -j$(nproc)

Verified 2026-08-27 on gfx1151: clean clone → 0 build errors → llama-server loads a mellum ROCmFP4 GGUF from this family and generates coherent text.

⚠️ STOCK llama.cpp WILL NOT LOAD THIS MODEL

The Mellum architecture is not merged upstream (PR #23966) and these are ROCmFPX quant types.

11.70 GiB · 75.72 tok/s on a Ryzen AI MAX+ 395.

Mellum2-12B-A2.5B-Thinking — ROCmFPX 8-bit GGUF

An 8-bit ROCmFPX quantization for AMD gfx1151 (Ryzen AI MAX+ 395 / Strix Halo), quantized from BF16 GGUF (24,311,969,856 B) — a lossless source, byte-verified against the upstream blob.


File	`Mellum2-12B-A2.5B-Thinking-Q8_0_ROCMFPX.gguf`
Size	11.70 GiB (12,560,021,568 B)
BPW	8.27
ftype	`Q8_0_ROCMFPX` (111)

⛔ Requires a llama.cpp with the ROCmFPX quant types

Q8_0_ROCMFPX (111) / Q8_0_ROCMFPX_AGENT (115) exist only in charlie12345/ROCmFPX. Stock llama.cpp reports invalid ggml type 103. The Mellum arch itself is also not merged upstream (PR #23966). Ignore the auto-generated "Use this model" commands above.

All quant variants

All measured on one box, one binary (Ryzen AI MAX+ 395, gfx1151, ROCm 7.2.4), median of 3, warm-up discarded — so these rows are directly comparable.

variant	ftype	size	bpw	decode (median)	range
4-bit COHERENT	102	6.49 GiB	4.59	104.99	104.96 – 105.73
8-bit AGENT	115	11.88 GiB	8.39	74.41	74.36 – 74.42
8-bit plain	111	11.70 GiB	8.27	75.72	75.72 – 75.73

Repos: 4-bit · 8-bit AGENT · 8-bit plain

⚠️ AGENT is SLIGHTLY SLOWER here — 74.41 vs 75.72. The opposite of the Instruct variant, which tells you the difference is model-specific and small. Neither 8-bit build approaches the 4-bit build's 104.99 tok/s.

On AGENT generally: it keeps more tensors at true Q8_0 instead of the packed 8-bit type. That raises MTP draft acceptance on models which have an MTP head (measured +6.2% on Qwen3.8-27B). Mellum2 has no MTP head, so there is nothing for the extra precision to feed and the two 8-bit builds differ only marginally — in either direction.

Correctness: 17×23 ⇒ ✅ 391 · capital of Japan ⇒ ✅ Tokyo · days in 2024 ⇒ ✅ 366 (checked with a 1024-token budget — a small budget makes these models look broken by spending it all inside <think> and returning empty content.)

What was NOT measured

No perplexity run, and no quality A/B against the source. The checks above are memorized-fact prompts — necessary but not sufficient.
No long-context testing. · No tool-calling evaluation.

Base model licence inherited; credit for the model goes to its authors.

kingjones777/Mellum2-12B-A2.5B-Thinking-ROCmFPX-Q8_0-GGUF

🔧 Runtime: build the ROCmFPX fork below

⚠️ STOCK llama.cpp WILL NOT LOAD THIS MODEL

Mellum2-12B-A2.5B-Thinking — ROCmFPX 8-bit GGUF

⛔ Requires a llama.cpp with the ROCmFPX quant types

All quant variants

What was NOT measured

kingjones777/
Mellum2-12B-A2.5B-Thinking-ROCmFPX-Q8_0-GGUF

⚠️ STOCK `llama.cpp` WILL NOT LOAD THIS MODEL