🔧 Runtime: build the ROCmFPX fork below
Stock
llama.cpp will not load this file. You need
both the
zaya architecture
and the ROCmFP4 tensor types in one tree. Upstream
charlie12345/ROCmFPX has the ROCmFP4 types but
not
zaya. Our fork has both:
kingjones30/ROCmFPX — a fork of
charlie12345/ROCmFPX, branch
main.
1git clone https://github.com/kingjones30/ROCmFPX.git
2cd ROCmFPX
3cmake -B build -DGGML_HIP=ON -DGPU_TARGETS=gfx1151 -DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release
4cmake --build build --target llama-server llama-quantize -j$(nproc)
Verified 2026-08-27 on gfx1151: clean clone → 0 build errors → llama-server loads a
zaya ROCmFP4 GGUF from this family and generates coherent text.
⚠️ STOCK llama.cpp WILL NOT LOAD THIS MODEL
The
zaya architecture is not merged upstream (draft PR
#23112). Ignore the auto-generated
"Use this model" commands above — build the
ROCmFPX fork linked just below.
📦 4.86 GiB, smaller than Q4_K_M (5.19 GiB), from a 16.52 GiB BF16 source.
⚠️ Speed is parity, not a win — see the honest benchmark below.
ZAYA1-8B — ROCmFP4 (tier 102 COHERENT) GGUF
A 4-bit
ROCmFP4 quantization of
Zyphra/ZAYA1-8B
for
AMD gfx1151 (Ryzen AI MAX+ 395 / Strix Halo).
| |
|---|
| File | ZAYA1-8B-Q4_0_ROCMFP4_COHERENT.gguf |
| Size | 4.8629 GiB (5,221,551,424 bytes) |
| BPW | 4.71 |
| ftype | Q4_0_ROCMFP4_COHERENT (102) |
| Source | BF16 GGUF (16.52 GiB) — lossless source, not a requantization |
| sha256 | e663e10d86bd63b1d4cccb5cd3bde5eb0d7d7442f9cd0f5e3857d0fe2739357c |
⚠️ Read this before choosing 4-bit for ZAYA1
ZAYA1 is not weight-bandwidth-bound, so 4-bit buys size — not speed. We measured all three
builds on the same machine, median of 3, warm-up discarded, on an otherwise idle box:
| build | size | median decode | range |
|---|
| ROCmFP4 (this) | 4.86 GiB | 15.8 tok/s | [14.1 – 20.49] |
| Q4_K_M | 5.19 GiB | 19.26 tok/s | [15.26 – 19.88] |
| BF16 | 16.52 GiB | 17.0 tok/s | [16.38 – 17.41] |
BF16 reads 3.4× more bytes per token and decodes at the same rate. A bandwidth-bound model
would show roughly 3× separation here; ZAYA1 shows none. The bottleneck is compute — the
Compressed Convolutional Attention grouped-conv path (cca_conv_grp, 10 groups per even layer).
All three ranges overlap, so we make no speed claim in either direction — this build is
neither meaningfully faster nor slower than Q4_K_M. Take it for the 0.32 GiB size saving and
the fact that it fits alongside more models, not for throughput.
Where the time actually goes
We localized decode cost by structurally skipping the CCA grouped convolution and
re-measuring (rocprof is unavailable on this hardware):
| build | baseline | conv skipped | conv share |
|---|
| ROCmFP4 | 25.0 / 24.5 tok/s | 56.7 / 52.7 | ~54% |
| Q4_K_M | 24.9 / 24.5 tok/s | 55.5 | ~56% |
The CCA grouped convolution is roughly 55% of decode time, and it is identical across
quantizations — the conv weights are BF16 in both. Fitting t = fixed + bytes/BW to the BF16 and
4-bit points puts ~37 ms of ~42 ms per token in weight-independent fixed cost, i.e. decode is
~88% independent of the weight format.
⇒ No 4-bit format can make this model faster. A proper grouped-conv kernel could be worth up
to ~2× end-to-end, but that is a llama.cpp kernel/conversion fix, not a quantization one. We
prototyped a single-GEMM rewrite and did not ship it — it was not numerics-identical, and a
speedup that changes output is not a speedup.
Correctness
Official-sampling checks, reading both reasoning_content and content with a 2500-token budget
(ZAYA1 reasons at length — a small cap will make it look broken when it is not):
| check | result |
|---|
| 17 × 23 | ✅ 391 |
| capital of Japan | ✅ Tokyo |
| days in 2024 | ✅ 366 |
Per-tensor types (audited in the finished file, 1283 tensors)
| tensor class | type |
|---|
token_embd.weight | Q6_K |
cca_conv_grp.weight | BF16 — never quantized |
cca_conv_grp.bias, cca_k_scale | F32 |
ssm_conv1d.{weight,bias} (80) | F32 |
ffn_gate_inp router (80) | F32 |
| norms (121) | F32 |
routed experts, cca_val_proj1/2 | 4-bit |
⚠️ tie_word_embeddings is true on this model, so there is no separate output.weight and
--output-tensor-type is a silent no-op. --token-embedding-type q6_K is the flag that
actually lands — and on a 262,272-token vocabulary that matters a great deal.
cca_conv_grp is excluded from quantization in the patch. Those tensors drive the compressed
convolutional attention and are highly sensitive; other ZAYA1 quantizers exclude them too.
What was NOT measured
- No perplexity run, and no quality A/B against Q4_K_M or BF16. The checks above are
memorized-fact prompts — necessary but not sufficient.
- No long-context testing (the model supports 131,072).
- No tool-calling evaluation.
- No coding or reasoning benchmark.
Model
ZayaForCausalLM / zaya. 40 layers, all hybrid · hidden 2048 · vocab 262,272 ·
16 experts, 1 active · ssm_d_conv = 2 · 1283 tensors · context 131,072 ·
tie_word_embeddings: true. ~8.4B total / ~760M active.
Base model licence: Apache-2.0 (inherited). Credit for the model itself goes to
Zyphra.
All quant variants
Three builds of this model, all measured in one session on one box with one binary
(Ryzen AI MAX+ 395, gfx1151, ROCm 7.2.4, ROCmFPX-2809dc5) — so these rows are directly
comparable. Median of 3, warm-up discarded, otherwise-idle box.
⚠️ Decode is ~88% weight-independent on this architecture (the CCA grouped conv is ~55% of decode). All three builds land within ~10% of each other; the 4-bit is smallest and marginally fastest. No 8-bit or 4-bit format will make this model meaningfully faster.
What AGENT actually changes: it keeps far more tensors at true Q8_0 instead of the
packed 8-bit type — measured in these files, 154 tensors vs 1 tensor. On models with an
MTP draft head that raises draft acceptance and wins ~6%; these two models have no MTP head,
and here the two 8-bit builds are within noise of each other.
