Ling-3.0-flash-base-midtrain — ROCmFP4 for AMD Strix Halo (gfx1151)
✅ the first ROCmFP4 build of Ling-3.0-flash-base-midtrain, published with a measured MTP curve
Checked 2026-08-22 against every public GGUF of this checkpoint. The only other GGUF build
(avar6/Ling-3.0-flash-base-midtrain-gguf) ships a single standard k-quant, Q5_K_M. ROCmFP4 is a
runtime tensor format that exists only in the
ROCmFPX fork of llama.cpp. Repository-content
comparison only — no third-party build was run or benchmarked here.
A 4-bit ROCmFP4 quantisation of Ling-3.0-flash-base-midtrain for AMD Ryzen AI Max+ 395 /
Radeon 8060S / gfx1151, with the multi-token-prediction (MTP) draft head preserved.
⚠️ This is a base checkpoint, not an instruct model
Ling-3.0-flash-base-midtrain is a pretrained / base checkpoint released by inclusionAI for
continued pretraining, domain adaptation and fine-tuning. It is not instruction-tuned. It ships
a chat_template.jinja, but that is a tokenizer asset — it does not make the weights
conversational. Prompt it as a text continuation model. For chat, use
inclusionAI/Ling-3.0-flash instead.
midtrain is the mid-training checkpoint — after the 30T pretraining stage and after long-context extension, but before any instruction tuning.
This checkpoint carries the full context_length = 262,144 and rope_theta = 6000000 of the
long-context-extended Flash base line. (The 30T checkpoint, by contrast, declares only 8,192.)
The file
ftype
102 — Q4_0_ROCMFP4_COHERENT
size
72,123,713,664 bytes (67.17 GiB)
parameters
127.49 B (512 experts × 3.9 B, 8 active)
architecture
bailing-hybrid — hybrid KDA linear attention + MLA
tensors
938 · block_count 43 (42 layers + 1 MTP layer)
context
262,144
rope_theta
6000000
Head protection, verified in the finished file (not merely requested at quantise time, and
re-audited after the metadata rename that produced the final bytes above):
tie_word_embeddings is false on this model, so --output-tensor-type does real work here —
the COHERENT tier on its own leaves output.weight at 4-bit. Both heads were forced to Q6_K and
audited on exact tensor names.
Architecture notes
Ling-3.0-flash interleaves two attention types. head_count_kv is a per-layer array where 0
marks a KDA linear-attention layer and 1 a full MLA layer: 1 MLA layer in every 6. MLA uses a
compressed KV path (kv_lora_rank 512) with a plain wide query projection (q_lora_rank: null).
The blk.42 MTP layer is retained in full, including nextn.eh_proj, nextn.enorm,
nextn.hnorm and nextn.shared_head_norm, with the unfusedattn_k_b / attn_v_b form that
the MTP path requires.
Measured throughput
AMD Ryzen AI Max+ 395, Radeon 8060S (gfx1151), ROCm 7.2.4, 128 GB unified memory.
llama-cli, -dio -ngl 999 -st -c 2048 -n 512 --temp 0 --seed 1234, 3 repetitions per
config, measured on an otherwise idle box.
MTP is worth +17.5% on this checkpoint, with disjoint ranges.
All three Ling-3.0-flash base checkpoints measure the same no-drafter baseline to the decimal on
identical hardware and flags, which is the cross-check for this figure:
checkpoint
no drafter
MTP n-max 3
effect
Ling-3.0-flash-base
36.6 t/s
42.3 t/s
+15.6%
Ling-3.0-flash-base-30T
36.6 t/s
41.4 t/s
+13.1%
Ling-3.0-flash-base-midtrain (this file)
36.6 t/s
43.0 t/s
+17.5%
MTP is reliably positive across the whole Ling-3.0-flash base family. It is not reliable on
Ling-3.0-tiny, where the same measurement gives +7.5% / +5.1% / −4.4% across the three
checkpoints — the draft head is trained with the model, so its value belongs to the specific
(size, checkpoint) pair rather than to the architecture. Measure before enabling it.
Requirements
This file uses the ROCmFP4 tensor format and the bailing-hybrid architecture. It requires a build
of ROCmFPX that carries both. Stock llama.cpp will
not load it. Verify with strings libllama.so | grep bailing-hybrid — the architecture table lives
in the shared library, not in the thin CLI binary.
bash
1llama-cli -m Ling-3.0-flash-base-midtrain-Q4_0_ROCMFP4_COHERENT.gguf \2 -dio -ngl 999 -c 2048 -n 512\3 --spec-type draft-mtp --spec-draft-ngl 999 --spec-draft-n-max 3\4 -p "The history of mathematics begins in ancient times. One of the earliest known"
-dio (direct I/O) is recommended. At -ngl 999 the HIP backend copies offloaded tensors out of
file-backed pages into device allocations, so without direct I/O the source pages and the device
buffer are resident simultaneously — roughly twice the model size, which is tight on a 128 GB box.
Sample output
Continuation from "The history of mathematics begins in ancient times. One of the earliest known":
mathematical texts is the Rhind Papyrus, which dates back to around 1650 BCE in ancient Egypt. This papyrus, named after the Scottish antiquarian Henry Rhind who purchased it in 1858, contains a collection of mathematical problems and solutions that provide valuable insights into the mathematical knowledge and practices of the time. The Rhind Papyrus includes problems related to arithmeti
Not measured
Perplexity is not published for this build. A 127 B model at this size exceeds a practical
evaluation budget on a single Strix Halo box. Quality evidence here is limited to the coherence
check above and the verified tensor-level audit.
Output determinism under MTP was not tested on this checkpoint. On the sibling
Ling-3.0-flash-base, MTP was found not to be output-deterministic at --temp 0 with a fixed
seed. Assume the same here unless you verify it.
n-max 5 was not swept on this checkpoint.n-max 3 is the published setting.
Provenance
Converted from inclusionAI/Ling-3.0-flash-base-midtrain at revision
34f7c1ed096bdb3118ec1474132ad21794d4510a to BF16 GGUF (938 tensors), then quantised to ftype 102 with
--output-tensor-type q6_K, then general.name set to Ling-3.0-flash-base-midtrain and the heads
re-audited on the finished file. Licence MIT, inherited from the base model.
