Stock llama.cpp will not load this file. You need both the bailing-hybrid architecture
and the ROCmFP4 tensor types in one tree. Upstream
charlie12345/ROCmFPX has the ROCmFP4 types but
not bailing-hybrid. Our fork has both:
Verified 2026-08-27 on gfx1151: clean clone → 0 build errors → llama-server loads a
bailing-hybrid ROCmFP4 GGUF from this family and generates coherent text.
Ling-3.0-flash-base — ROCmFP4 for AMD Strix Halo (gfx1151)
✅ the first ROCmFP4 build of Ling-3.0-flash-base, published with a measured MTP curve
Checked 2026-08-22 against every public GGUF of this checkpoint. The two other GGUF builds
— mradermacher/Ling-3.0-flash-base-GGUF (Q2_K through Q5_K_S) and
avar6/Ling-3.0-flash-base-gguf (Q5_K_M, uploaded 2026-08-22) — both ship standard
k-quants. ROCmFP4 is a runtime tensor format that exists only in the
ROCmFPX fork of llama.cpp. Repository-content
comparison only — no third-party build was run or benchmarked here.
A 4-bit ROCmFP4 quantisation of Ling-3.0-flash-base for AMD Ryzen AI Max+ 395 / Radeon 8060S /
gfx1151, with the multi-token-prediction (MTP) draft head preserved.
⚠️ This is a base checkpoint, not an instruct model
Ling-3.0-flash-base is a pretrained / base checkpoint released by inclusionAI for continued
pretraining, domain adaptation and fine-tuning. It is not instruction-tuned. It ships a
chat_template.jinja, but that is a tokenizer asset — it does not make the weights conversational.
Prompt it as a text continuation model. For chat, use inclusionAI/Ling-3.0-flash instead.
The file
ftype
102 — Q4_0_ROCMFP4_COHERENT
size
72,123,713,664 bytes (67.16 GiB)
parameters
127.49 B (512 experts × 3.9 B, 8 active)
architecture
bailing-hybrid — hybrid KDA linear attention + MLA
tensors
938 · block_count 43 (42 layers + 1 MTP layer)
context
262,144
Head protection, verified in the finished file (not merely requested at quantise time):
tie_word_embeddings is false on this model, so --output-tensor-type does real work here —
the COHERENT tier on its own leaves output.weight at 4-bit. Both heads were forced to Q6_K and
audited on exact tensor names after the build.
Architecture notes
Ling-3.0-flash interleaves two attention types. head_count_kv is a per-layer array —
[0,0,0,0,0,1,0,0,0,0,0,1,…] — where 0 marks a KDA linear-attention layer and 1 a full MLA
layer: 1 MLA layer in every 6. MLA uses a compressed KV path (kv_lora_rank 512) with a plain
wide query projection (q_lora_rank: null). The blk.42 MTP layer is retained in full, including
nextn.eh_proj, nextn.enorm, nextn.hnorm and nextn.shared_head_norm, with the unfusedattn_k_b / attn_v_b form that the MTP path requires.
MTP is worth +15.6% on this model, with disjoint ranges and a 0.0% baseline spread across
5 repetitions.
The two sibling checkpoints measure the same baseline to the decimal on identical hardware and
flags, which is the cross-check for this figure:
checkpoint
no drafter
MTP n-max 3
effect
Ling-3.0-flash-base (this file)
36.6 t/s
42.3 t/s
+15.6%
Ling-3.0-flash-base-30T
36.6 t/s
41.4 t/s
+13.1%
Ling-3.0-flash-base-midtrain
36.6 t/s
43.0 t/s
+17.5%
MTP is reliably positive across the whole Ling-3.0-flash base family. It is not reliable on
Ling-3.0-tiny, where the same measurement gives +7.5% / +5.1% / −4.4% across the three
checkpoints — the draft head is trained with the model, so its value belongs to the specific
(size, checkpoint) pair rather than to the architecture. Measure before enabling it.
n-max 3 is the recommended setting. Deeper speculation costs prefill throughput: prompt
processing measures ~90–98 t/s at n3 versus ~122–130 t/s with no drafter.
⚠️ MTP is not output-deterministic here
At --temp 0 with a fixed seed, the no-drafter arm produced byte-identical text across all 5
repetitions. Both MTP arms did not — n-max 3 produced 3 distinct outputs across 5 runs, n-max 5
produced 4, and every MTP output differed from the no-drafter output. If bit-reproducible generation
matters more to you than +15.6%, run without the draft head.
Requirements
This file uses the ROCmFP4 tensor format and the bailing-hybrid architecture. It requires a build
of ROCmFPX that carries both. Stock llama.cpp will
not load it. Verify with strings libllama.so | grep bailing-hybrid — the architecture table lives
in the shared library, not in the thin CLI binary.
bash
1llama-cli -m Ling-3.0-flash-base-Q4_0_ROCMFP4_COHERENT.gguf \2 -dio -ngl 999 -c 2048 -n 512\3 --spec-type draft-mtp --spec-draft-ngl 999 --spec-draft-n-max 3\4 -p "The history of mathematics begins in ancient times. One of the earliest known"
-dio (direct I/O) is recommended. At -ngl 999 the HIP backend copies offloaded tensors out of
file-backed pages into device allocations, so without direct I/O the source pages and the device
buffer are resident simultaneously — roughly twice the model size, which is tight on a 128 GB box.
Sample output
Continuation from "The history of mathematics begins in ancient times. One of the earliest known":
mathematical texts is the Rhind Mathematical Papyrus, which dates back to around 1650 BCE in
ancient Egypt. This papyrus, named after the Scottish antiquarian Alexander Henry Rhind who
purchased it in 1858, contains a collection of mathematical problems and solutions that provide
insight into the mathematical knowledge of the time. The problems cover a range of topics,
including arithmetic, geometry, and algebra, and demonstrate the Egyptians' practical approach
to mathematics.
Not measured
Perplexity is not published for this build. No perplexity figure is quoted because none was
completed on this hardware — a 127 B model at this size exceeds a practical evaluation budget on a
single Strix Halo box. Quality evidence here is limited to the coherence check above and the
verified tensor-level audit.
Sibling quantisations
Also built but not published: Q4_0_ROCMFP4_LEAN (72.07 GB — within 0.07% of COHERENT, since the
tiers differ only in token-embedding type, which is negligible on a 127 B model),
Q6_0_ROCMFPX (105.22 GB) and Q8_0_ROCMFPX (131.66 GB — exceeds the addressable memory of a
128 GB Strix Halo box).
Provenance
Converted from inclusionAI/Ling-3.0-flash-base at revision
0a96b0a3c68f93e5565d6f2d56c98557bded74fb to BF16 GGUF (255.1 GB, 938 tensors), then quantised to
ftype 102 with --output-tensor-type q6_K. Licence MIT, inherited from the base model.
