🔧 Runtime: build the ROCmFPX fork below

Stock llama.cpp will not load this file. You need both the muse-glimmer architecture and the ROCmFP4 tensor types in one tree. Upstream charlie12345/ROCmFPX has the ROCmFP4 types but not muse-glimmer. Our fork has both:

kingjones30/ROCmFPX — a fork of charlie12345/ROCmFPX, branch main.

bash
1git clone https://github.com/kingjones30/ROCmFPX.git 2cd ROCmFPX 3cmake -B build -DGGML_HIP=ON -DGPU_TARGETS=gfx1151 -DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release 4cmake --build build --target llama-server llama-quantize -j$(nproc)

Verified 2026-08-27 on gfx1151: clean clone → 0 build errors → llama-server loads a muse-glimmer ROCmFP4 GGUF from this family and generates coherent text.

Muse-Glimmer-30B — ROCmFP4 for AMD Strix Halo (gfx1151)

✅ the complete muse-glimmer port — text graph, vision projector and chat parser — ships as a single applyable patch in this repo, alongside 4 ROCmFP4 ftypes, the DFlash drafter and the vision projector

muse-glimmer is not an upstream llama.cpp architecture. Running it end-to-end takes three independent pieces of work; all three are in patches/muse-glimmer-complete.patch (20 files, 81,968 bytes, git apply --check clean). Variant count verified against Hugging Face repository metadata for the public ROCmFP4 builds of this base model — file facts only, no third-party build was benchmarked here.

Six quantisations of Muse-Glimmer-30B — four ROCmFP4 and two 8-bit ROCmFPX — built for AMD Ryzen AI Max+ 395 / Radeon 8060S / gfx1151, bundled with the DFlash speculative drafter and vision projector. ROCmFP4 is a runtime tensor format that exists only in the ROCmFPX fork of llama.cpp; Muse Glimmer support exists only in current upstream — this build ports the model forward into the ROCmFPX base so the two can meet.

Metric	Result
Quantization	ROCmFP4 / ROCmFPX (ggml types 100–115), 6 variants
Model size	13.80 – 27.23 GiB (6 quants)
Effective BPW	4.25 – 8.39 (see table)
Tested hardware	AMD Ryzen AI Max+ 395 (Strix Halo), 128 GB unified
GPU	Radeon 8060S, gfx1151
ROCm version	7.2.4
32K decode (FAST, real-world)	~17-45 tok/s depending on workload; ~30 typical
Prompt processing	not separately instrumented — see Not yet measured
DFlash decode	4.13 tokens accepted per target pass, n_max=15
Peak memory	19.99 GiB resident (model + drafter + projector + 32K KV)
Context validated	32768 only — see Not yet measured
Tool calling	6/7 on a 7-case suite (parity with upstream)
Reasoning	yes — `reasoning_content` / `content` split
Vision	yes — verified on spatial ground truth, requires `-fa off`

Why this build?

20.31 tok/s on the FAST variant vs 16.65 tok/s for Meta's fastest official GGUF (kquant-17gb) — same box, same flags, same drafter: 1.22×
13.80 GiB vs Meta's 15.61 GiB — 1.8 GiB smaller and faster
Four ftypes published so you can pick the size/speed/verbosity point you want
Text, vision and DFlash speculative decoding all work from one download
Chat-format parser ported, so no to=self<|message|> control tokens leak into output
Every published file validated before upload; nothing shipped unverified

Which file should I use?

Ryzen AI Max+ 395, ROCm 7.2.4, DFlash drafter at --spec-draft-n-max 15, -fa on, ctx 32768, batch 1, temperature 0. Warm medians of 9 generations; the first call after load is discarded.

Build	ftype	Size	BPW	TG 32K	Quality
`ROCmFP4-FAST`	103	13.80 GiB	4.25	20.31	3/3
`ROCmFP4-STRIX_LEAN`	106	14.00 GiB	4.38	18.72	3/3
`ROCmFP4-STRIX`	105	14.17 GiB	4.36	17.24	3/3
`ROCmFP4-BASE`	100	16.87 GiB	4.50	15.90	3/3
Meta `kquant-17gb` (reference)	—	15.61 GiB	—	16.65	3/3
Meta `kquant-17gb` on Vulkan	—	15.61 GiB	—	6.10	3/3

Why two different speed figures? The comparison table above is a controlled A/B: every build ran the same fixed prompt set, so the numbers are directly comparable to each other and to Meta's reference (that is where the 1.22x comes from). Real-world decode on the FAST build varies with workload by about 2.6x - measured on the live deployment at ~45 tok/s on code-transform/edit work, ~17 tok/s on freeform prose, and ~30 tok/s in typical mixed use. Quote the range, not a single number.

Start with FAST. BASE is both the slowest and the largest — it is published for completeness, not because anyone should choose it.

⚠ The faster files write shorter answers

Some of the speed comes from terser output, not only from faster decode. Median words per answer on identical prompts:

Build	Median words
`STRIX` (105)	377
`STRIX_LEAN` (106)	306
`BASE` (100)	301
`FAST` (103)	259

The quality check is substring-based and cannot distinguish "more concise" from "less thorough." If answer depth matters more than throughput, prefer STRIX. This is a real trade, not a free win.

Across the whole family — all six model quants

⚠️ Decode on this model is workload-dominated, not variant-dominated. DFlash proposes long runs on repetitive and code-like text and very little on freeform prose, so a single tok/s figure is misleading — mean accepted length moves 2.5 → 7.1 on the same binary and the same weights. Quote a range for this model, not a point.

Measured on one Ryzen AI MAX+ 395, median of 3, with the DFlash head (--spec-type draft-dflash --model-draft dflash-ROCmFP4-STRIX.gguf --spec-draft-ngl 99):

Variant	ftype	Size	prose	code-transform	accept len (code)
4-bit FAST	103	13.80 GiB	15.07	39.35	7.12
4-bit STRIX_LEAN	106	14.00 GiB	—	—	—
4-bit STRIX	105	14.17 GiB	14.96	37.55	6.80
4-bit BASE	100	16.87 GiB	—	—	—
8-bit plain	111	26.85 GiB	11.31	—	2.65
8-bit AGENT	115	27.23 GiB	11.27	—	2.51

Without a drafter the 8-bit builds measure 7.48 (111) and 7.22 (115) tok/s — medians of 3, raw runs 7.54 / 7.48 / 7.35 and 7.22 / 7.21 / 7.26.

⛔ Serve it with the draft head. Without --model-draft the 8-bit build drops 11.62 → 7.65 (−34%). The two 8-bit builds are within noise of each other: AGENT routing lifts draft acceptance on MTP models, and this model uses DFlash, so there is nothing for it to win here.

Quick start

hf download kingjones777/Muse-Glimmer-30B-ROCmFP4-Strix-Halo-DFlash-GGUF --local-dir muse

bash

llama-server \
  -m muse/muse-glimmer-30B-ROCmFP4-FAST.gguf \
  --mmproj muse/mmproj-kquant.gguf \
  --spec-type draft-dflash \
  --model-draft muse/dflash-kquant.gguf \
  --spec-draft-n-max 15 \
  --chat-template-kwargs '{"reasoning_strength":"medium"}' \
  -ngl 999 -fa off -dio --jinja \
  -c 32768 --host 127.0.0.1 --port 8080

Requires a llama.cpp built with ROCmFP4 support (ggml types 100–106). Stock llama.cpp rejects these tensor types.

Three flags that matter more than which file you pick

Flag	Why
`--chat-template-kwargs '{"reasoning_strength":"..."}'`	Template defaults to `high`. At `high` this model spent an entire 1200-token budget deliberating on a real refactor task and returned no visible answer at all. `medium` answered in 38.1s, `low` in 32.6s.
`-fa on` (text) / `-fa off` (vision)	`-fa off` costs +21% at ctx 32768 but is mandatory for images. Run separate endpoints if you serve both.
`--spec-draft-n-max 15`	DFlash block size is 16; one slot holds the previously accepted token.

⛔ --reasoning-budget does not work on this model. Values 256 and -1 produced byte-identical runs at temperature 0 — the flag is not enforced on peg-native format. Use reasoning_strength instead.

Verified hardware

Hardware	GPU	ROCm	Status	TG 32K	Notes
Ryzen AI Max+ 395 (Strix Halo)	Radeon 8060S / gfx1151	7.2.4	✅ Tested by KingJones	~30 (17-45)	128 GB unified
Any Vulkan backend	—	—	❌ Known incompatible	—	rejects ggml type 101 at parse time
gfx1201 / RDNA4	—	—	❓ Untested	—
NVIDIA / CUDA	—	—	❌ Known incompatible	—	ROCmFP4 is a ROCm-only tensor format

Vulkan is impossible, not merely slow. The backend rejects these files at parse time:

gguf_init_from_reader: tensor 'output.weight' has invalid ggml type 101. should be in [0, 43)

Vulkan's type table ends at 43; ROCmFP4 types are 100–106. No flag changes this. For reference, Meta's k-quant does run on Vulkan and measured 6.10 tok/s versus 16.65 on ROCm on this box, so Vulkan is not a useful path for this model in any case.

Speculative decoding (DFlash)

Muse Glimmer has no MTP tensors — zero in both the base checkpoint and the quants. It speculates using DFlash against a separate 5-layer drafter, which is what Meta's own recipe prescribes.

Metric	Value
Setting	`--spec-type draft-dflash --spec-draft-n-max 15`
Tokens accepted per target pass	4.13 (median, range 2.85–6.65)
Per-token acceptance	~21%
Drafter memory	1.52 GiB

⚠️ Per-token acceptance is a misleading statistic for a block drafter. DFlash proposes 15 tokens in one forward pass; ~21% acceptance means ~4.13 tokens land per target pass, which is healthy. Judge block drafters on tokens-per-pass.

n-gram speculation is not a substitute here. Measured on the same build: ngram-map-k reached 42.9% acceptance — double DFlash's — yet ran 45% slower on code-transform work (15.20 vs 27.49 tok/s), because it proposes far fewer tokens per pass.

A ROCmFP4 drafter is included but is not the default. It measured 1.008× against Meta's k-quant drafter on a quiet box — inside noise, acceptance unchanged. The drafter is ~1.5 GiB of a ~17 GiB working set, so shrinking it 8.5% moves total memory traffic by well under 1%. Shipped because it is valid, not because it is faster.

Tool calling

7-case suite, run against this build and against Meta's k-quant on the upstream binary as a reference:

Case	This build	Upstream reference
multi-arg (string/int/bool)	✅	✅
nested object argument	✅	✅
enum constraint	✅	✅
correctly declines (no spurious call)	✅	✅
multi-turn tool-result follow-up	✅	✅
streaming tool call	✅	✅
two parallel calls in one turn	❌	❌
Total	6/7	6/7

The parallel-call failure is the model's, not the quantisation's — Meta's own weights on upstream's own parser fail identically. Sequential agent loops are unaffected.

Raw example:

json

{"name": "book_flight",
 "arguments": {"passenger": {"name": "Alice Smith", "age": 34},
               "route": "LHR-JFK", "cabin": "business"}}

Agentic loop

A multi-step loop (list → move → observe → finish) over a directory of loose files, 3 runs at temperature 0.7: 3/3 completed the task correctly, 0 cases of claiming an action without emitting a tool call.

Vision

The vision path needs stage 2 of the port below — it registers PROJECTOR_TYPE_MUSE_GLIMMER in clip/mtmd. A build carrying only the 9-file text-graph patch reports unknown projector type: muse-glimmer when a projector is passed.

⚠️ Minimum useful image size is 28×28 px. The preprocessor snaps to patch 14 × merge 2, so anything smaller collapses to a single merge token and carries no spatial signal. Feed 256×256 or larger; llama-mtmd-cli behaves identically — this is preprocessing geometry, not the projector.

Works, and is verified for spatial correctness rather than plausible-sounding output: a four-quadrant colour image is scored on whether each colour lands in the right corner. A misapplied attention mask names colours confidently but places them wrongly, so this test distinguishes a working port from a fluent-but-broken one. 3/3.

Requires -fa off — ggml_flash_attn_ext aborts on Muse's per-layer sparse-window masks.

Quantization methodology

bash

# 1. convert BF16 safetensors -> GGUF (upstream tree; only it has the muse-glimmer converter)
python convert_hf_to_gguf.py <MODEL_DIR> --outtype bf16 --outfile muse-glimmer-30B-BF16.gguf

# 2. quantize with the ROCmFPX build (only it has ggml types 100-106)
llama-quantize muse-glimmer-30B-BF16.gguf muse-glimmer-30B-ROCmFP4-FAST.gguf 103

Source: meta-models/Muse-Glimmer-30B BF16 safetensors, 1436 tensors → 55.7 GB BF16 GGUF (731 text tensors) → ROCmFP4.

The model was ported forward into the ROCmFPX base in three stages:

Text graph, arch registration and converter. Three API gaps bridged: is_swa_impl → swa_layers, n_layer() from method to field, and the NVFP4-only output-scale argument (null on the ROCmFP4 path).
Vision tower — required teaching the older base's build_vit to accept per-layer attention masks at all; it previously took no mask parameter. Added as an overload so the ~32 other vision models calling it are untouched.
Chat-format parser, so harmony-style channel output is parsed rather than leaking to=self<|message|> into content.

Files

Ten files, three distinct networks. llama.cpp loads them via --model, --model-draft and --mmproj — there is no merged single-file format.

Model weights — all six quants live in this repo

File	ftype	Size	Bytes	BPW
`muse-glimmer-30B-ROCmFP4-FAST.gguf`	103	13.80 GiB	14,815,844,928	4.25
`muse-glimmer-30B-ROCmFP4-STRIX_LEAN.gguf`	106	14.00 GiB	15,031,512,640	4.38
`muse-glimmer-30B-ROCmFP4-STRIX.gguf`	105	14.17 GiB	15,210,123,008	4.36
`muse-glimmer-30B-ROCmFP4-BASE.gguf`	100	16.87 GiB	18,117,264,192	4.50
`muse-glimmer-30B-Q8_0_ROCMFPX.gguf`	111	26.85 GiB	28,826,594,688	8.28
`muse-glimmer-30B-Q8_0_ROCMFPX_AGENT.gguf`	115	27.23 GiB	29,235,965,312	8.39

Drafter, projector and patches

File	Size	Role
`dflash-kquant.gguf`	1.52 GiB	DFlash drafter (Muse's, unmodified) — use this
`dflash-ROCmFP4-STRIX.gguf`	1.39 GiB	ROCmFP4 drafter — works, 1.008× (a wash)
`mmproj-kquant.gguf`	1.30 GiB	vision projector
`patches/muse-glimmer-complete.patch`	81,968 B	20-file port — text graph + vision + chat parser
`patches/rocmfpx-3edc3d3-add-muse-glimmer.patch`	44,062 B	9-file text-graph-only port

Total 117.12 GiB. Download a single quant rather than the whole repo:

bash

hf download kingjones777/Muse-Glimmer-30B-ROCmFP4-Strix-Halo-DFlash-GGUF \
  --include "muse-glimmer-30B-ROCmFP4-FAST.gguf" --local-dir muse

⚠️ hf download silently ignores --include when given more than one pattern — issue one call per file.

Not yet measured

Listed explicitly so nobody mistakes absence for a pass. These are genuine gaps, not claims:

Test	Status
Context scaling (2K / 8K / 16K / 64K / 128K)	❓ only 32768 measured
Prompt-processing tok/s, isolated	❓ not separately instrumented
Sustained generation (1K / 4K tokens)	❓ not measured
Perplexity / KL divergence vs BF16	❓ not measured
MMLU-Pro, GPQA, GSM8K, HumanEval+, MBPP+	❓ not run
Long-context needle retrieval	❓ not run
DFlash n-max sweep (2 / 4 / 8 / 24)	❓ only n=15 measured
5-run statistics with std dev	⚠️ 9 samples per arm, median reported; std dev not published
Independent reproduction	❓ none yet

Measurement conditions: the tok/s figures were taken on a machine that also served other traffic during the run. The ordering across builds is wide enough to be reliable; the exact ratios are not trustworthy to three significant figures. A re-run on a quiesced box is planned.

Quality caveat: the 3/3 figure is a smoke check over factual recall, arithmetic and instruction-following, scored by substring match. It is a regression guard against a broken quantisation, not a benchmark suite, and it does not measure answer depth. No claim of "no quality loss" is made — that would require the perplexity and standardized evaluations listed above.

Independent results

None yet. If you run this build, please open a discussion with: hardware, GPU, OS, ROCm version, runtime commit, exact command, context, prompt-processing tok/s, generation tok/s and peak RAM. Independent reproductions will be listed separately from author benchmarks and carry more weight.

Known issues

Vulkan/CUDA/CPU cannot load these files — ROCmFP4 is a ROCm-only tensor format.
Vision requires -fa off, costing ~21% on text at 32K context.
Parallel tool calls fail — model-level, reproduced identically on Meta's own weights.
Small max_tokens returns empty content — the budget goes to reasoning_content. Allow several hundred tokens.
--reasoning-budget is not enforced on this model; use reasoning_strength.

License and attribution

Base model, DFlash drafter and vision projector are Meta's, under the base model's licence. ROCmFP4 quantisation types are from the ROCmFPX fork of llama.cpp. This repository contains the quantised weights and the measurements above.

Other public builds of this model

Compiled from Hugging Face repository metadata — file sizes, shipped files, quant variant as named by each repo. No third-party build was run or benchmarked here, so this table makes no speed or quality claim about any of them. It is here so you can see the size and format options at a glance and pick what fits your hardware.

Repository	Largest model file	Variant	Ships	Downloads	Likes
`RadixArk/Muse-Glimmer-NVFP4`	4.00 GiB	NVFP4	safetensors	40	5
`kingjones777/Muse-Glimmer-30B-ROCmFP4-Strix-Halo-DFlash-GGUF` (this repo)	16.87 GiB	STRIX	4 model files, drafter, vision	0	1
`Preyazz/Muse-Glimmer-30B-NVFP4`	17.37 GiB	NVFP4	safetensors	0	5
`cloudnathan5/Muse-Glimmer-30B-NVFP4`	18.63 GiB	NVFP4	safetensors	0	2
`RedHatAI/Muse-Glimmer-30B-NVFP4`	18.63 GiB	NVFP4	safetensors	0	7
`vmlinux/Muse-Glimmer-30B-ROCmFPX-GGUF`	26.77 GiB	ROCmFPX	3 model files, drafter, vision	0	13

Base model: meta-models/Muse-Glimmer-30B. Generated from Hub metadata; download counts move over time.

Acknowledgements

This build would not exist without the work below. Please star and follow these projects — the quantisation format used here is their engineering, not mine.

ROCmFPX — maintained by charlie12345 / caf The ROCmFP4 / ROCmFPX tensor formats (ggml types 100–106) exist only in this fork. Every ROCmFP4 file in this repository was produced with its llama-quantize, and runs on its runtime. The fork also credits collaborators ciru-ai, Tom Turney, PlunderStruck and Aydan S., and acknowledges AMD for hardware support. Licensed MIT, based on upstream llama.cpp.

llama.cpp — ggml-org and contributors The inference engine, GGUF format and conversion tooling everything here is built on.

AMD ROCm The compute platform these builds target — ROCm 7.2.4 on gfx1151 / Radeon 8060S.

Base model authors — see base_model in the metadata above; all model weights, licences and capabilities are theirs. This repository contributes quantisation and measurement only.

If you use these files, please credit ROCmFPX alongside this repository.

🩹 The muse-glimmer architecture port — complete, three stages

patches/muse-glimmer-complete.patch — 81,968 bytes, 20 files, git apply --check clean against charlie12345/ROCmFPX.

bash

git clone https://github.com/charlie12345/ROCmFPX.git && cd ROCmFPX
git apply --check ../patches/muse-glimmer-complete.patch && \
git apply           ../patches/muse-glimmer-complete.patch
cmake -S . -B build-muse -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 \
      -DLLAMA_BUILD_WEBUI=OFF -DCMAKE_BUILD_TYPE=Release
cmake --build build-muse --target llama-server --target llama-quantize \
      --target llama-mtmd-cli -j 6

muse-glimmer is not an upstream architecture. Running it end-to-end takes three independent pieces of work, and all three are in this patch.

Stage 1 — text graph and conversion

File	Role
`src/llama-arch.{h,cpp}`	`LLM_ARCH_MUSE_GLIMMER` + tensor-name table
`src/llama-model.cpp`	model factory case
`src/models/muse-glimmer.cpp`	the graph itself
`src/models/models.h`	declaration
`gguf-py/gguf/{constants,tensor_mapping}.py`	GGUF constants + tensor map
`conversion/{__init__,muse_glimmer}.py`	safetensors → GGUF converter

⚠️ convert_hf_to_gguf.py does not know MuseGlimmer. The converter is conversion/muse_glimmer.py, driven through conversion.get_model_class.

Stage 2 — vision projector (clip / mtmd)

File	Role
`tools/mtmd/models/muse-glimmer.cpp`	projector graph
`tools/mtmd/clip-impl.h`	`PROJECTOR_TYPE_MUSE_GLIMMER`
`tools/mtmd/clip-model.h`	hparams + LANCZOS enum
`tools/mtmd/clip-graph.h`	`build_vit_opts` + 7-arg overload
`tools/mtmd/clip.cpp`, `models/models.h`, `CMakeLists.txt`	wiring
`tools/mtmd/mtmd-image.{cpp,h}`	preprocessor; LANCZOS → bicubic_pillow fallback
`tools/mtmd/mtmd.cpp`	`<\|image_start\|>` / `<\|image_end\|>` markers
`gguf-py/gguf/tensor_mapping.py`	`model.vision_tower.layers.{bid}.attn.{q,k,v,proj}`, `norm1/2`, `mlp.fc1/2`, `ln_post`
`gguf-py/gguf/constants.py`	`VisionProjectorType.MUSE_GLIMMER`

This stage is what makes --mmproj work. This repo ships mmproj-kquant.gguf (1.30 GiB); a BF16 projector built through this stage — mmproj-muse-glimmer-30B-BF16.gguf, 3,849,174,048 bytes, 809 tensors, clip.projector_type = muse-glimmer, merge 2, patch 14, image_size 896 — is published alongside the 8-bit builds in Muse-Glimmer-30B-ROCmFPX-Q8_0-GGUF.

Stage 3 — chat parser

common/chat.cpp — common_chat_params_init_muse_glimmer, PEG_NATIVE.

Without it the model's to=self<|message|> control sequence is emitted into content. With it, content is clean: "The capital of Japan is Tokyo."

Build note

cmake's source GLOB is configure-time. After the patch adds src/models/muse-glimmer.cpp you must re-run the cmake -S . -B build-muse configure step, not just --build. -DLLAMA_BUILD_WEBUI=OFF avoids a node/npm requirement.

Applying to a different base commit

The patch header names commit 3edc3d3, and it applies cleanly to later revisions (verified on b41ce12). On trees where cohere2moe and bailing_hybrid model sources are absent, their factory cases in llama-model.cpp reference symbols that do not exist in that tree — build those two out, or apply on 3edc3d3 where their .cpp files are present. The muse-glimmer factory case and graph are independent of both.