Stock llama.cpp will not load this file. You need both the bailing-hybrid architecture
and the ROCmFP4 tensor types in one tree. Upstream
charlie12345/ROCmFPX has the ROCmFP4 types but
not bailing-hybrid. Our fork has both:
Verified 2026-08-27 on gfx1151: clean clone → 0 build errors → llama-server loads a
bailing-hybrid ROCmFP4 GGUF from this family and generates coherent text.
Ling-3.0-tiny-base-30T — ROCmFP4 for AMD Strix Halo (gfx1151)
4-bit ROCmFP4 quantisation of Ling-3.0-tiny-base-30T, one of the six Ling-3.0 base/training
checkpoints inclusionAI released on 2026-08-20. The multi-token-prediction (MTP) head is preserved.
⚠️ This is a base checkpoint, not an instruct model
Released for continued pretraining, domain adaptation and fine-tuning. Not instruction-tuned.
It ships a chat_template.jinja, but that is a tokenizer asset and does not make the weights
conversational. Prompt it as a text continuation model. For chat, use inclusionAI/Ling-3.0-tiny.
The file
ftype
102 — Q4_0_ROCMFP4_COHERENT
size
4,769,674,624 bytes
architecture
bailing-hybrid — hybrid KDA linear attention + MLA
tensors
549 · block_count 25 (24 layers + 1 MTP layer)
q_lora_rank
256 — low-rank compressed queries
context
262,144
Head protection, verified in the finished file:
output.weight Q6_K 1536 x 157184
token_embd.weight Q6_K 1536 x 157184
tie_word_embeddings is false, so --output-tensor-type does real work here — the COHERENT tier
alone leaves output.weight at 4-bit. Both heads were forced to Q6_K and audited on exact tensor
names after the build.
⚙️ Requires a patched llama.cpp — details
Ling-3.0-tiny sets q_lora_rank: 256, so its MLA layers use a two-stage compressed query
(q_a_proj → RMS norm → q_b_proj). Ling-3.0-flash sets q_lora_rank: null and uses a single
wide q_proj. A bailing-hybrid implementation written against flash therefore cannot load tiny.
The patch, against ROCmFPX, mirrors the existing
DeepSeek-V2 low-rank query path:
add those three tensors to MODEL_ARCH.BAILING_HYBRID (TensorNameMap skips anything not in the arch list)
convert_hf_to_gguf.py
emit add_q_lora_rank() when set; the null path is unchanged
src/models/bailing-hybrid.cpp
read Q_LORA_RANK as optional; when n_lora_q > 0 create wq_a / wq_b / attn_q_a_norm and run q_a → RMS → q_b at both graph sites, else keep the wide wq
Because the KV is read as optional, flash GGUFs (which lack it) keep n_lora_q = 0 and take the
original path unchanged. ~51 lines across 5 files.
Verify a build carries it with strings libllama.so | grep bailing-hybrid — the architecture table
lives in the shared library, not the thin CLI binary.
-4.4% from MTP.Do not enable MTP on this checkpoint. It is a measured net loss of 4.4%.
The three Ling-3.0-tiny base checkpoints do not agree on this. Measured on identical hardware
with identical flags: tiny-base+7.5%, tiny-base-midtrain+5.1%, tiny-base-30T−4.4%. Every comparison was range-disjoint against a 0.2–0.7% baseline spread. The MTP draft
head is trained alongside the model, so its quality is a property of the training checkpoint, not
of the architecture — and a checkpoint can be fully competitive on quality and speed while shipping
a draft head that is a net negative. Measure before enabling it.
Sample output
Continuation from "The history of mathematics begins in ancient times. One of the earliest known":
The history of mathematics begins in ancient times. One of the earliest known mathematical texts is the Rhind Papyrus, which dates back to around 1650 BCE. This document, discovered in Egypt, contains problems and solutions related to arithmetic, geometry, and algebra. It was used by the ancient Egyptians to teach
Not measured
Perplexity is not published for this build. No perplexity figure is quoted because none was
completed. Quality evidence is the coherence check above plus the tensor-level audit.
Provenance
Converted from inclusionAI/Ling-3.0-tiny-base-30T at revision 5ffd60dad2cd5df60fb916ab431682c563290797 to BF16 GGUF (549 tensors),
then quantised to ftype 102 with --output-tensor-type q6_K. Licence MIT, inherited from the base model.
