You can deploy this model and start earning money today!

Joakimpalm-Zen/
Muse-Glimmer-30B-runner-quant-frontier-report

Name: Muse-Glimmer-30B-runner-quant-frontier-report
Availability: InStock

meta-models/Muse-Glimmer-30B

finetune

apache-2.0

Views

No views yet

Muse Glimmer 30B — what a Runner selective quant can and cannot beat

No weights are republished here. This is a measurement report over Meta's own files: six selective-precision candidates plus the published k-quant, each gated at 400 teacher-forced positions against a BF16 parent converted from Meta's own safetensors, on one machine, on one day, with every plan and command included.

The headline is not the one this stage set out to produce:

Meta's own 16.76 GB KQuant-17GB-Q4_K_M passes this project's quality bar against the BF16 checkpoint it came from — margin-qualified top-1 99.25%, mean KLD 0.030, against a bar of ≥97% and ≤0.05. It is a well-made file.
No Runner-built plan at or below its size beats it. Six were built and gated. The best measures 0.056 KLD; the rest, 0.090 to 0.170.
The binding constraint is Runner's own quantizer, not the checkpoint. --type-plan can write q8_0, q4_0, q3_k and f16 — no k-quants. The closest candidate clears margin-qualified top-1 at 97.75% and misses mean KLD at 0.056, and every route to the missing ~11% of divergence costs more bytes than the comparator has.

Serve with `xyntetik-runner` — tool calls that still parse when the budget runs out

This GGUF is built to be served with xyntetik-runner, a single-binary, dependency-free C11 inference engine (CPU / CUDA / Metal, OpenAI-compatible server). Its one differentiator you cannot get from another engine:

When a tool call runs past its token budget, Runner closes it to the smallest schema-legal document, so the arguments still parse.

Most OpenAI-compatible engines return finish_reason: "length" with an empty or malformed tool_calls the caller has to repair or retry. Runner does forced-truncation recovery: once a JSON document starts, it emits a legal ending when the budget expires. On local models — tight context, slow generation — that is the difference between an agent loop that finishes and one that retries from scratch.

Measured — what each engine hands the caller when the token budget cuts a tool call short (same box, same tool schema, same prompt, tool_choice:"required", temperature 0, budgets 1→64):

engine	budget too small (1–16 tokens)	enough budget (64, control)
Runner	executable `tool_calls`, arguments parse	completes
vLLM 0.27.1	no call; protocol framing leaks into `content`	completes
llama.cpp b10488	no call; leak, then `tool_calls` with unparseable args	completes
Ollama 0.32.14	no call; empty content, then HTTP 500	completes
TensorRT-LLM 1.2.1 †	no call; `<tool_call>` leak, then empty content	completes
SGLang 0.5.17 †	no call; `<tool_call>` leak, then empty content	completes

Every engine completes at 64 tokens — the failure is truncation, not misconfiguration. Below that, only Runner returns an executable call. This is the behaviour across every OpenAI-compatible engine measured, not a claim about engines not measured. † TensorRT-LLM and SGLang were measured on a Qwen3-1.7B substitute; truncation recovery is a property of the runtime, not the model.

Constrained decoding also holds the shape of a tool call under quantization: on a full quant ladder, schema conformance and tool selection stayed 100% down to Q4_0 (argument agreement decayed to 50%) — it guarantees the shape of a call at any quantization, not its contents.

Quickstart

bash

# build the single binary (no deps), then serve this GGUF:
runner -m Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf --serve
# point any OpenAI-compatible client at http://localhost:8080/v1

Full recipe, raw responses, and the per-release regression gate: the truncation benchmark.

Muse Glimmer is one of the families with a native tool protocol in Runner: atem definitions and results, recipient-constrained generation, truncation recovery, multi-call mapping and buffered/SSE parsing are selected automatically when a loaded Muse model receives tools. Evidence: docs/muse-atem-cert-2026-08-11.md.

The parent, and why it had to be built

No BF16 or F16 Muse GGUF is published. A selective-precision candidate carries a precision claim, so gating it against Meta's own 4-bit file would have been quant→quant and could not support that claim. The 59.5 GB safetensors were therefore downloaded and converted text-only:

python3 convert_hf_to_gguf.py Muse-Glimmer-30B/ --outtype bf16 \
    --outfile Muse-Glimmer-30B-BF16.gguf     # llama.cpp 521a64cd

731 tensors, 55,725,514,176 bytes, sha256 d22e290d242aaf1eee6ee185e15e905fce41afba5fb03ac22aad7c08a0603406. The vision encoder is not converted — mmproj is a separate upstream file and Runner has no vision path — and the tokenizer and vocabulary are untouched, multimodal special tokens included. Full detail in PROVENANCE.md.

The table

400 teacher-forced positions per arm, 0 failed, zero point exact (0.000 / 100.00% / 100.00%), every arm against that BF16 parent, same machine and same Runner build. Bar: margin-qualified top-1 ≥ 97% and mean KLD ≤ 0.05.

arm	bytes	vs Meta	plain top-1	margin-qualified	mean KLD	verdict
BF16 parent vs itself (zero point)	55,725,514,176	—	100.00%	100.00%	0.000	exact
Meta `KQuant-17GB-Q4_K_M`	16,756,683,904	—	90.25%	99.25%	0.030	passes
M1 — FFN Q3_K, everything else Q8_0	16,492,547,520	‑1.6%	80.25%	92.00%	0.123	fails
M2 — M1 + embed/output Q4_0	15,147,716,032	‑9.6%	79.25%	92.00%	0.125	fails
M4 — Q3_K except `attn_k/v` + `output`	13,801,180,608	‑17.6%	77.00%	89.75%	0.170	fails
M5 — gate/up Q3_K, `ffn_down` Q4_0, embed Q4_0	16,065,605,056	‑4.1%	83.00%	94.50%	0.090	fails
M7 — no Q3_K at all: Q4_0 but `attn_v` Q8_0	15,730,568,640	‑6.1%	89.50%	97.75%	0.056	fails

Raw JSON for every row is in gates/; every plan is in plans/.

What the six candidates actually taught

The geometry is not the obstacle — it is unusually friendly. Every quantizable Muse row width divides by 256 (6656 = 256·26, 19968 = 256·78, 4096, 256), so Q3_K applies everywhere. (Compare nemotron_h_moe, where every tensor carrying mass has a row width that is an odd multiple of 128 and Q3_K is silently declined.) Q3_K being available turned out not to help.

1. Q3_K is the damage, not the plan around it. M1 spends Q8_0 on every attention tensor and both embeddings and still measures 0.123 — four times Meta's 0.030, at 1.6% less size.

2. The embeddings are nearly free. M1 → M2 drops token_embd and output from Q8_0 to Q4_0: 1.35 GB saved (‑8.2%) for a KLD move of 0.123 → 0.125. On a 202k-token vocabulary that is the cheapest 1.3 GB on the table, and it is reusable knowledge for any plan on this checkpoint.

3. ffn_down is the sensitive FFN tensor — Meta's own file says so. Meta protects ffn_down with Q6_K on 26 of 52 layers while leaving ffn_gate/ ffn_up at Q4_K. Moving only ffn_down off Q3_K to Q4_0 (M5) improved 0.123 → 0.090 for +0.9 GB, with gate/up still at Q3_K. The asymmetry is real and reproducible.

4. The type menu is the binding constraint. M7 removes Q3_K entirely and lands at 97.75% margin-qualified — over the bar — with 0.056 mean KLD, under it, 6.1% smaller than Meta's file. It is the closest anything came, and it is a plain Q4_0: a commodity quant with no Runner-specific advantage, so it would be rejected on that ground even without the near-miss. Closing the remaining ~11% of divergence needs precision spent where Runner cannot spend it — ffn_down at Q8_0 alone adds 3.45 GB and lands at 19.2 GB against Meta's 16.76 GB.

The follow-up this produces is a Runner change, not an artifact one: teach --quantize to WRITE Q4_K/Q5_K/Q6_K. Runner reads them already. Until it can write them, a Runner-built selective quant cannot match a well-made k-quant at the 4-bit size point on a dense model — measured here, not assumed.

Reproduce it

git clone https://github.com/Joakimpalm-Zen/xyntetik-runner && cd xyntetik-runner
make && ./runner --version                      # runner 0.1.20-alpha

# size a plan from the header before spending a build on it — this also
# catches a rule the quantizer will silently decline
python3 scripts/type-plan-size.py Muse-Glimmer-30B-BF16.gguf plans/plan-m7-allQ4_0-attnQ4.json

./runner -m Muse-Glimmer-30B-BF16.gguf --quantize m7.gguf \
    --type-plan plans/plan-m7-allQ4_0-attnQ4.json

# zero point first — it must read exactly 0.0 / 100% / 100%
python3 scripts/kld-compare-raw.py --model-a Muse-Glimmer-30B-BF16.gguf \
    --model-b Muse-Glimmer-30B-BF16.gguf --runner ./runner \
    --corpus tests/fixtures/mixed-corpus.txt --max-positions 400 --out zero.json
python3 scripts/kld-compare-raw.py --model-a m7.gguf \
    --model-b Muse-Glimmer-30B-BF16.gguf --runner ./runner \
    --corpus tests/fixtures/mixed-corpus.txt --max-positions 400 --out m7.json

Each predicted size in plans/ was confirmed exact against the built file (M2 predicted 15,147,716,032 and built 15,147,716,032; M5 and M7 likewise).

Scope and limits

The gate measures agreement with the parent's next-token distribution, not downstream capability. Meta's file passing it does not make it equal to the BF16 checkpoint on any benchmark, and the candidates failing it does not make them unusable.
Not measured: downstream task benchmarks; native atem tool fidelity on the candidates (the tool-fidelity edition selects among survivors, and there are none); Metal or CUDA paths — every arm is CPU, because the 55.7 GB parent does not fit the 23.6 GB MIG slice and one protocol per table matters more than one fast arm; context beyond the corpus; --kv q8, which is a separate runtime profile and is never folded into a weight claim.
Untouched in every candidate: the tokenizer and vocabulary, including the multimodal special tokens, and the vision encoder, which is a separate upstream file.
One corpus, one machine, one day.
docs/muse-glimmer-cert-2026-08-11.md in the Runner repo certifies the k-quant at sha256 7e9b74b7…; upstream has since re-published it and the current blob is 4cc57c0f…, 2,848 bytes longer. The numbers here use the current file.

Publication policy

This project publishes only gated artifacts, and re-measures published claims under the current standard. Negative results are published rather than discarded — this report is one. No wording here implies that any candidate is equivalent to its parent, and none of them is offered as a download.

Attribution and license

Base model: meta-models/Muse-Glimmer-30B by Meta. All model weights, capabilities and credit belong to Meta. The 4-bit GGUF measured as the comparator is Meta's own. This repository republishes no weights — only measurements, plans and commands.
What was modified: nothing in any published file. The BF16 parent and the six candidates were built locally and are not distributed.
License: Apache-2.0, together with Meta's USAGE_POLICY.md, reproduced here verbatim as the upstream repository requires.
Not affiliated with, sponsored by, or endorsed by Meta.

Joakimpalm-Zen/Muse-Glimmer-30B-runner-quant-frontier-report