Stock llama.cpp will not load this file. You need both the bailing-hybrid architecture
and the ROCmFP4 tensor types in one tree. Upstream
charlie12345/ROCmFPX has the ROCmFP4 types but
not bailing-hybrid. Our fork has both:
Verified 2026-08-27 on gfx1151: clean clone → 0 build errors → llama-server loads a
bailing-hybrid ROCmFP4 GGUF from this family and generates coherent text.
Ling-3.0-tiny-base-midtrain — ROCmFP4 for AMD Strix Halo (gfx1151)
4-bit ROCmFP4 quantisation of Ling-3.0-tiny-base-midtrain, one of the six Ling-3.0 base/training
checkpoints inclusionAI released on 2026-08-20. The multi-token-prediction (MTP) head is preserved.
⚠️ This is a base checkpoint, not an instruct model
Released for continued pretraining, domain adaptation and fine-tuning. Not instruction-tuned.
It ships a chat_template.jinja, but that is a tokenizer asset and does not make the weights
conversational. Prompt it as a text continuation model. For chat, use inclusionAI/Ling-3.0-tiny.
The file
ftype
102 — Q4_0_ROCMFP4_COHERENT
size
4,769,679,040 bytes
architecture
bailing-hybrid — hybrid KDA linear attention + MLA
tensors
549 · block_count 25 (24 layers + 1 MTP layer)
q_lora_rank
256 — low-rank compressed queries
context
262,144
Head protection, verified in the finished file:
output.weight Q6_K 1536 x 157184
token_embd.weight Q6_K 1536 x 157184
tie_word_embeddings is false, so --output-tensor-type does real work here — the COHERENT tier
alone leaves output.weight at 4-bit. Both heads were forced to Q6_K and audited on exact tensor
names after the build.
⚙️ Requires a patched llama.cpp — details
Ling-3.0-tiny sets q_lora_rank: 256, so its MLA layers use a two-stage compressed query
(q_a_proj → RMS norm → q_b_proj). Ling-3.0-flash sets q_lora_rank: null and uses a single
wide q_proj. A bailing-hybrid implementation written against flash therefore cannot load tiny.
The patch, against ROCmFPX, mirrors the existing
DeepSeek-V2 low-rank query path:
add those three tensors to MODEL_ARCH.BAILING_HYBRID (TensorNameMap skips anything not in the arch list)
convert_hf_to_gguf.py
emit add_q_lora_rank() when set; the null path is unchanged
src/models/bailing-hybrid.cpp
read Q_LORA_RANK as optional; when n_lora_q > 0 create wq_a / wq_b / attn_q_a_norm and run q_a → RMS → q_b at both graph sites, else keep the wide wq
Because the KV is read as optional, flash GGUFs (which lack it) keep n_lora_q = 0 and take the
original path unchanged. ~51 lines across 5 files.
Verify a build carries it with strings libllama.so | grep bailing-hybrid — the architecture table
lives in the shared library, not the thin CLI binary.
+5.1% from MTP.Enable MTP. n-max 3 is worth +5.1% on this checkpoint.
The three Ling-3.0-tiny base checkpoints do not agree on this. Measured on identical hardware
with identical flags: tiny-base+7.5%, tiny-base-midtrain+5.1%, tiny-base-30T−4.4%. Every comparison was range-disjoint against a 0.2–0.7% baseline spread. The MTP draft
head is trained alongside the model, so its quality is a property of the training checkpoint, not
of the architecture — and a checkpoint can be fully competitive on quality and speed while shipping
a draft head that is a net negative. Measure before enabling it.
Sample output
Continuation from "The history of mathematics begins in ancient times. One of the earliest known":
The history of mathematics begins in ancient times. One of the earliest known mathematical texts is the Plimpton 322 tablet, which dates back to around 1800 BCE in Mesopotamia. This tablet contains a list of Pythagorean triples, which are sets of three positive integers that satisfy the Pythagorean theorem. The ta
Not measured
Perplexity is not published for this build. No perplexity figure is quoted because none was
completed. Quality evidence is the coherence check above plus the tensor-level audit.
Provenance
Converted from inclusionAI/Ling-3.0-tiny-base-midtrain at revision 2c07d29ee370592bd804a0a865db5503d8a4bfa2 to BF16 GGUF (549 tensors),
then quantised to ftype 102 with --output-tensor-type q6_K. Licence MIT, inherited from the base model.
