Ornith-1.5-9B — FAST — ROCmFP4 / ROCmFPX GGUF
The speed-optimised 4-bit of ornith-ai/Ornith-1.5-9B — ftype 104 Q4_0_ROCMFP4_FAST_COHERENT, single-scale layout with Q6_K token embeddings and a Q6_K output head.
Built for
AMD Strix Halo (gfx1151) — Ryzen AI MAX+ 395, 128 GB unified memory, ROCm 7.13.0 — using the
ROCmFPX llama.cpp fork, which adds AMD-native FP4/FP8 tensor types that mainline llama.cpp does not have.
⚠️ These files require a ROCmFPX-capable llama.cpp build. They will not load in
stock llama.cpp / Ollama / LM Studio — the Q4_0_ROCMFP4_* and Q*_0_ROCMFPX* tensor
types are not in mainline.
Variants in this repo
| file | ftype | size | BPW | token_embd | output.weight | decode |
|---|
Ornith-1.5-9B-Q4_0_ROCMFP4_FAST_COHERENT.gguf | 104 | 4.99 GiB | 4.78 | Q6_K | Q6_K | 37.68 t/s |
Which to pick: FAST is the fastest build here at 37.68 t/s, and keeps Q6_K token embeddings. This model ships no usable drafter, so the no-drafter column is the only serving condition — which is exactly where FAST is expected to lead, and does.
Head protection — verified in the file, not assumed
tie_word_embeddings is false on this model, so output.weight is a real standalone tensor and --output-tensor-type genuinely bites. Every artifact was re-opened after quantization and its header read back with an independent parser:
| ftype | token_embd.weight | output.weight |
|---|
| 106 STRIX_LEAN | Q5_K | Q6_K |
| 104 FAST_COHERENT | Q6_K | Q6_K |
| 102 COHERENT | Q6_K | Q6_K |
| 114 / 111 / 115 | Q8_0 | Q8_0 |
⚠️ Note the LEAN row: it keeps the Q5_K token embeddings that make it lean while still protecting the output head at Q6_K. ftype 106 does not do this on its own — left to its defaults it emits a 4-bit output.weight. Head protection has to be requested and then verified in the file; an ftype name is not a guarantee.
Measured — not estimated
Hardware: AMD Ryzen AI MAX+ 395 (Strix Halo, gfx1151), 128 GB unified, ROCm 7.13.0. Idle box, 2 warm-ups discarded, median of 5, 300 tokens, identical prompt across every sample.
| quant | run 1 | run 2 | run 3 | run 4 | run 5 | median |
|---|
| 104 | 36.51 | 36.82 | 37.84 | 37.84 | 37.68 | 37.68 |
Speculative decoding
⛔ Not available for this model. config.json declares mtp_num_hidden_layers: 1, but the published safetensors contain no MTP weights and the converted GGUF has no nextn_predict_layers key — we checked both. No draft head ships with this model, and no GGUF drafter is published for it. The decode figures below are therefore the deployed numbers, with no speculative multiplier left on the table.
Source — byte-verified, not re-converted
Quantized from ornith-ai/Ornith-1.5-9B-GGUF → Ornith-1.5-9B-BF16.gguf, 17,920,696,768 bytes. The vision projector mmproj-Ornith-1.5-9B-BF16.gguf (921,704,416 bytes) is included. Both byte-verified against the Hub before quantization. No re-conversion from safetensors.
Architecture: Qwen3_5ForConditionalGeneration / qwen35, 427 tensors, 32 layers, hidden 4096, vocab 248,320 — a hybrid stack (24 gated-deltanet + 8 full-attention layers), not a pure dense transformer and not an MoE.
Verification
Every artifact: loaded at -ngl 999 -c 4096 -fit off -fa on, 3/3 correctness (17×23 → 391, capital of Japan → Tokyo, days in 2024 → 366) asserted on both content and reasoning_content, and 4/4 vision on a four-quadrant colour image via the mmproj.
File sizes were checked against --dry-run projections; the header delta is constant across artifacts, which is the signature of complete, untruncated files.
