You can deploy this model and start earning money today!

kingjones777/
Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF

Name: Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF
Availability: InStock

poolside/Laguna-S-2.1

Views

No views yet

Laguna-S-2.1 ROCmFP4 STRIX_LEAN (GGUF) — AMD Ryzen AI Max+ 395 / Strix Halo / gfx1151

First public-ready Laguna-S-2.1 ROCmFP4 "Strix Lean" quant for AMD Ryzen AI Max+ 395 (gfx1151 / Radeon 8060S).

⚠️ Not compatible with upstream llama.cpp. Requires the charlie12345/ROCmFPX fork built with HIP + ROCmFP4 kernels.

Files

File	Size	Notes
`Laguna-S-2.1-Q4_0_ROCMFP4_STRIX_LEAN.gguf`	~58.34 GiB	4.26 BPW (quantize report)
`SHA256SUMS`

Base model: poolside/Laguna-S-2.1
F16 source: poolside/Laguna-S-2.1-GGUF laguna-s-2.1-F16.gguf (235202258240 bytes)

Hardware / stack (validated)

Box: Ryzen AI Max+ 395, gfx1151, 128 GB unified
ROCm 7.2.4 (/opt/rocm-7.2.4, clang 22)
Fork: charlie12345/ROCmFPX @ 3edc3d31ee5ebcea47fd7e0f42c89767bb4245db

Build recipe (HIP-only)

bash

export PATH=/opt/rocm-7.2.4/bin:$PATH
export HIP_PATH=$(hipconfig -R)
export HIPCXX=$(hipconfig -l)/clang
cmake -B build \
  -DGGML_HIP=ON -DGPU_TARGETS=gfx1151 \
  -DGGML_HIP_ROCWMMA_FATTN=ON -DGGML_HIP_NO_VMM=ON -DGGML_HIP_MMQ_MFMA=ON \
  -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=OFF -DLLAMA_BUILD_WEBUI=OFF
cmake --build build -j 4   # prefer -j 4..8 on 128GB Strix; avoid -j32

Quantize

bash

build/bin/llama-quantize \
  laguna-s-2.1-F16.gguf \
  Laguna-S-2.1-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  Q4_0_ROCMFP4_STRIX_LEAN 8
# quant size = 59739.82 MiB (4.26 BPW)

Runtime (required)

bash

export LD_LIBRARY_PATH=<build/bin>:/opt/rocm/lib
export HSA_OVERRIDE_GFX_VERSION=11.5.1
export GGML_HIP_ENABLE_UNIFIED_MEMORY=1

llama-server --host 127.0.0.1 --port 8099 \
  --n-gpu-layers 999 --flash-attn on -dio --no-warmup --jinja \
  --model Laguna-S-2.1-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  --ctx-size 65536 --cache-type-k q8_0 --cache-type-v q8_0 --parallel 1 \
  --temp 0.2 --top-p 0.95 --top-k 20 --min-p 0.05 --repeat-penalty 1.1 --repeat-last-n 256 \
  --chat-template-kwargs '{"enable_thinking":true}' --reasoning-budget 512

-dio is required for reliable cold load of large Laguna GGUFs on this stack (mmap path can hang).

A/B vs Q4_K_M (same binary, same flags, cold load)

Quant	prompt_n	prompt tok/s	decode tok/s (256-cap gen)	GGUF size
Q4_K_M	8116 (~8K)	445.6	17.36	71 GiB
ROCmFP4 STRIX_LEAN	8116 (~8K)	381.8	28.23	58.3 GiB
Q4_K_M	32223 (~32K)	372.4	14.15	71 GiB
ROCmFP4 STRIX_LEAN	32223 (~32K)	327.3	20.32	58.3 GiB

Decode speedup: +62.6% @ ~8K, +43.6% @ ~32K
Size: −18% (58.3 vs 71 GiB)
Prompt processing slightly slower on ROCmFP4 (−14% / −12%)

Quality / tools / reasoning

Shared prompts (reasoning, code, tool-call plan, short math with thinking):

Check	Q4_K_M	ROCmFP4	Verdict
Bat/ball $0.05	correct	correct	parity
is_palindrome plan	coherent thinking	coherent thinking	parity
get_weather Paris/Tokyo tool JSON	clean calls	clean calls	parity
enable_thinking / reasoning-budget	works	works	parity

Did we lose anything? No quality regression observed on the four shared prompts. We gain decode speed and lose model file size; prefill is slightly slower.

⚡ DFlash speculative decoding — +23% decode (added 2026-08-08)

DFlash works with this quant and is worth turning on: 32.6 → 40.1 tok/s (1.234×) on my box, same binary, same flags, only the speculation changed.

config	decode tok/s	vs off
off	32.6	1.000×
dflash `n-max 2`	39.6	1.215×
dflash `n-max 4`	40.1	1.234×
dflash `n-max 8`	25.2	0.773× — too deep, don't

Draft head: wimmmm/poolside-Laguna-S-2.1-DFlash-GGUF, Q8_0, 1.19 GB (the GGUF twin of poolside/Laguna-S-2.1-DFlash-NVFP4). Keep the head at Q8_0 — draft quality drives acceptance.

Two things will stop you cold. Both are fixable.

1. The fork commit pinned above is too old for DFlash

Building at 3edc3d3 (the commit in the recipe above) gives:

error loading model: done_getting_tensors: wrong number of tensors; expected 76, got 69

The 7 unmapped tensors are 6× blk.N.attn_gate.weight plus enc.aux_norm.weight. Current ROCmFPX main knows them; that commit does not. Quick check on your own build:

bash

grep -ac aux_norm    build/bin/libllama.so   # newer: non-zero, old: 0
grep -ac decoder_arch build/bin/libllama.so  # newer: non-zero, old: 0

Build ROCmFPX main (add -DLLAMA_BUILD_WEBUI=OFF — the WebUI asset step fails without Node/npm and takes llama-server down with it at 100%). You can keep your existing build for everything else and point only Laguna at the new one.

2. The DFlash head declares a target layer that doesn't exist

The head ships dflash.target_layers = [2, 11, 20, 30, 39, 48], but Laguna-S-2.1 is block_count = 48, so valid indices are 0–47. llama.cpp asserts and aborts:

llama-context.cpp: GGML_ASSERT(lid < model.hparams.n_layer) failed

vLLM tolerates 48 as "the final hidden state"; llama.cpp does not. Patch the last element to 47 — it is a single 4-byte value of the same width, so it rewrites in place with no re-encode:

python

import struct, shutil
src="laguna-s-2.1-dflash-Q8_0.gguf"; dst="laguna-s-2.1-dflash-Q8_0-fix47.gguf"
shutil.copy(src, dst)
f=open(dst,"r+b"); assert f.read(4)==b"GGUF"
struct.unpack("<I",f.read(4)); struct.unpack("<Q",f.read(8))
n_kv=struct.unpack("<Q",f.read(8))[0]
def rs():
    n=struct.unpack("<Q",f.read(8))[0]; return f.read(n).decode()
SZ={0:1,1:1,7:1,2:2,3:2,4:4,5:4,6:4,10:8,11:8,12:8}
for _ in range(n_kv):
    k=rs(); t=struct.unpack("<I",f.read(4))[0]
    if t==9:
        et=struct.unpack("<I",f.read(4))[0]; ln=struct.unpack("<Q",f.read(8))[0]
        start=f.tell()
        if k=="dflash.target_layers":
            fmt={4:"<I",5:"<i",10:"<Q",11:"<q"}[et]
            f.seek(start+(ln-1)*SZ[et]); f.write(struct.pack(fmt,47))
            print("patched last target layer -> 47"); break
        if et==8:
            for _ in range(ln): rs()
        else: f.seek(SZ[et]*ln,1)
    elif t==8: rs()
    else: f.seek(SZ[t],1)
f.close()

Serving with DFlash

bash

llama-server --host 127.0.0.1 --port 8099 \
  --n-gpu-layers 999 --flash-attn on -dio --no-warmup --jinja \
  --model Laguna-S-2.1-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  --spec-type draft-dflash \
  -md laguna-s-2.1-dflash-Q8_0-fix47.gguf -ngld 999 \
  --spec-draft-n-max 4 --spec-draft-p-min 0.5 \
  --ctx-size 65536 --cache-type-k q8_0 --cache-type-v q8_0 --parallel 1

⚠️ You must set --spec-draft-n-max. The default of 16 exceeds the head's trained block size of 15 and the server aborts in the DFlash constructor before it ever listens.

Credit for the GGUF draft head to wimmmm; the target-layer patch and the build-version finding are mine.

License

Follow the base model (poolside/Laguna-S-2.1) license terms.

Other public builds of this model

Compiled from Hugging Face repository metadata — file sizes, shipped files, quant variant as named by each repo. No third-party build was run or benchmarked here, so this table makes no speed or quality claim about any of them. It is here so you can see the size and format options at a glance and pick what fits your hardware.

Repository	Largest model file	Variant	Ships	Downloads	Likes
`kkuspa/Laguna-S-2.1-NVFP4-0804`	4.26 GiB	NVFP4	safetensors	1220	5
`kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF` (this repo)	58.34 GiB	STRIX_LEAN	single model file	461	2
`1337Hero/Laguna-S-2.1-Q4_0-ROCMFP4-GGUF`	58.34 GiB	ROCmFP4	single model file	219	0
`raulvidis/Laguna-S-2.1-ROCmFP4-STRIX-GGUF`	58.38 GiB	STRIX	single model file	455	4
`jcbtc/Laguna-S-2.1-Chadrock-ROCmFP4-StrixKVSpine-V4-GGUF`	60.94 GiB	STRIX	single model file	5094	33
`raulvidis/Laguna-S-2.1-ROCmFP4-COHERENT-GGUF`	61.78 GiB	COHERENT	single model file	455	2
`cnuland/legunas21-custom`	—	—	drafter	80	1

Base model: poolside/Laguna-S-2.1. Generated from Hub metadata; download counts move over time.

Acknowledgements

This build would not exist without the work below. Please star and follow these projects — the quantisation format used here is their engineering, not mine.

ROCmFPX — maintained by charlie12345 / caf The ROCmFP4 / ROCmFPX tensor formats (ggml types 100–106) exist only in this fork. Every ROCmFP4 file in this repository was produced with its llama-quantize, and runs on its runtime. The fork also credits collaborators ciru-ai, Tom Turney, PlunderStruck and Aydan S., and acknowledges AMD for hardware support. Licensed MIT, based on upstream llama.cpp.

llama.cpp — ggml-org and contributors The inference engine, GGUF format and conversion tooling everything here is built on.

AMD ROCm The compute platform these builds target — ROCm 7.2.4 on gfx1151 / Radeon 8060S.

Base model authors — see base_model in the metadata above; all model weights, licences and capabilities are theirs. This repository contributes quantisation and measurement only.

If you use these files, please credit ROCmFPX alongside this repository.

kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF

Laguna-S-2.1 ROCmFP4 STRIX_LEAN (GGUF) — AMD Ryzen AI Max+ 395 / Strix Halo / gfx1151

Files

Hardware / stack (validated)

Build recipe (HIP-only)

Quantize

Runtime (required)

A/B vs Q4_K_M (same binary, same flags, cold load)

Quality / tools / reasoning

⚡ DFlash speculative decoding — +23% decode (added 2026-08-08)

1. The fork commit pinned above is too old for DFlash

2. The DFlash head declares a target layer that doesn't exist

Serving with DFlash

License

Other public builds of this model

Acknowledgements

kingjones777/
Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF