ROCmFP4 quantization of Ornith 1.5 35B-A3B for AMD Strix Halo (gfx1151) and RDNA 3.5 GPUs, produced with the Q4_0_ROCMFP4_STRIX_LEAN preset (FP16 embedding/norm preservation, nextn MTP head kept at q8_0).

Quantization Details

Property	Value
Quant format	`Q4_0_ROCMFP4_STRIX_LEAN` (ROCmFP4)
Bits per weight	4.29
File size	18.16 GiB
SHA256	`b42fb74cd32ce3ab5fc0b10214ae7b4030d852f49b749dcdd0e5ad6b35e1510f`
Vision projector	`mmproj-Ornith-1.5-35B-BF16.gguf` (0.84 GiB)
Projector SHA256	`d9ce31026d1cb1f3f8d5152e2e2a014d9d2b302b6c93a7dc07bb0a0487f52837`
Max Context	262,144 tokens (256K)
Source	Ornith-1.5-35B-A3B-GGUF Q8_0 (allow-requantize)
Notes	MTP head tensors preserved at q8_0

Measured Performance (AMD Ryzen AI Max+ 395, Radeon 8060S, Mesa RADV Wave64)

Configuration	Decode
ROCmFP4 bare decode	76.9 tok/s
Q4_K_M baseline	71.5–71.7 tok/s
ROCmFP4 MTP (K4 p0.0)	35–50 tok/s (net loss — keep MTP disabled)

ROCmFP4 is ~7.5% faster and 16.7% smaller than stock Q4_K_M.

Serving

HaloFPX (Text + Vision)

halofpx pull downloads and verifies both the ROCmFP4 weights and BF16 vision projector:

bash

halofpx pull ornith-1.5-35b
halofpx serve
halofpx load ornith-1.5-35b

halofpx list reports model-weight and vision-projector readiness separately.

Direct llama-server Vision

bash

llama-server \
  -m Ornith-1.5-35B-A3B-ROCmFP4.gguf \
  -mm mmproj-Ornith-1.5-35B-BF16.gguf \
  -ngl 99 -c 131072 -fa on --no-mmap

Full 256K Max Context (Strix Halo 128GB UMA):

bash

llama-server \
  -m Ornith-1.5-35B-A3B-ROCmFP4.gguf \
  -ngl 99 -c 262144 -fa on --no-mmap

Multi-Slot / Conservative Memory (131K Context):

bash

llama-server \
  -m Ornith-1.5-35B-A3B-ROCmFP4.gguf \
  -ngl 99 -c 131072 -fa on --no-mmap

Note: speculative decoding (MTP, --spec-type draft-mtp) is not recommended on this model (hybrid linear-attention MoE: ~16% draft acceptance, net throughput loss). Use bare decode (default --spec-type none).

julianmb/Ornith-1.5-35B-A3B-ROCmFP4-GGUF

Ornith-1.5-35B-A3B ROCmFP4

Quantization Details

Measured Performance (AMD Ryzen AI Max+ 395, Radeon 8060S, Mesa RADV Wave64)

Serving

HaloFPX (Text + Vision)

Direct llama-server Vision

Full 256K Max Context (Strix Halo 128GB UMA):

Multi-Slot / Conservative Memory (131K Context):

julianmb/
Ornith-1.5-35B-A3B-ROCmFP4-GGUF