Views
No views yet
Q4_0_ROCMFP4_COHERENT tensor-protected format from charlie12345/ROCmFPX.Laguna-S-2.1-Q4_0_ROCMFP4_COHERENT.gguf — 58.3 GiB, ~4.3 bpw effective.laguna.rope.scaling.yarn_attn_factor is baked to 1.0 per poolside's upstream fix ("llama.cpp derives mscale") — upstream-converted GGUFs from before 2026-07-24 carry 1.4852, which double-applies YaRN attention scaling. No --override-kv needed with this file.c190e435, 2026-07-23):git clone https://github.com/charlie12345/ROCmFPX
env JOBS=16 scripts/build-strix-rocmfp4-mtp.sh # gfx1151 / Strix Halo| Metric | Value |
|---|---|
| Decode (tg128) | ~32–35 t/s (HIP), ~37 t/s (Vulkan) |
| Prefill (pp512) | ~405 t/s (HIP) |
| 256k context serving | validated (HIP device; ~194k-token cold prompt in ~12 min) |
| Warm-turn TTFT (49k-token cached prefix) | ~0.3 s prefill (--cache-reuse 256) |
| KV q8_0 decode cost | negligible (<1%) |
HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1 \
llama-server -m Laguna-S-2.1-Q4_0_ROCMFP4_COHERENT.gguf \
-ngl 999 -fa on -dev ROCm0 -c 262144 --jinja \
-b 2048 -ub 1024 --cache-reuse 256 \
--temp 0.7 --top-p 0.95 --top-k 20 --min-p 0chat_template_kwargs: {"enable_thinking": false}); give thinking ≥32k max_tokens or it can exhaust the budget mid-reasoning. Vulkan devices failed very large (>190k-token) single fills in our testing across all runtimes — use the HIP device for extreme contexts.