Views
No views yet
unsloth/Laguna-S-2.1-GGUF (BF16)
(the original model is Poolside's Laguna S 2.1,
a 118B-total / 8B-activated MoE with 256 routed experts).[!IMPORTANT] You need the ROCmFPX fork of llama.cpp (or a llama.cpp build with ROCmFPX support). This file uses the experimentalq4_0_rocmfp4_fast(type 101) andq8_0_rocmfpx(type 103) weight formats, which stock llama.cpp releases do not understand — loading it elsewhere will fail with an unknown tensor type error.
| Tensor group | Type | Count |
|---|---|---|
Routed experts: blk.N.ffn_{gate,up,down}_exps | q4_0_rocmfp4_fast (4.25 bpw) | 141 |
| Everything else quantizable (attention, shared experts, embeddings, output head) | q8_0_rocmfpx (8.25 bpw) | 386 |
| Norms, biases, router weights/scales | f32 (untouched) | 287 |
1# from the ROCmFPX fork (CPU-only build works fine for quantization)
2llama-quantize \
3 --tensor-type "ffn_(gate|up|down)_exps=q4_0_rocmfp4_fast" \
4 Laguna-S-2.1-BF16-00001-of-00005.gguf \
5 Laguna-S-2.1-Q8_0_ROCMFPX-Q4FAST-experts.gguf Q8_0_ROCMFPX
6# then merged from 5 shards: llama-gguf-split --merge ...blk.0) keeps its dense FFN at q8_0_rocmfpx — only the routed
expert tensors were overridden.1# build ROCmFPX for your GPU (see the repo README; e.g. Strix Halo):
2env JOBS=16 scripts/build-strix-rocmfp4-mtp.sh
3
4# run (Vulkan was fastest in upstream tests on Strix Halo):
5./build-strix-rocmfp4/bin/llama-cli \
6 -m Laguna-S-2.1-Q8_0_ROCMFPX-Q4FAST-experts.gguf \
7 -dev Vulkan0 -ngl 999 -fa on --jinja[!NOTE] Benchmarks are pending — placeholder table below.
| Backend / GPU | Prompt (tok/s) | Generation (tok/s) | Context | Notes |
|---|---|---|---|---|
| TBD | TBD | TBD | TBD | TBD |
unsloth/Laguna-S-2.1-GGUF (BF16 shards)openmdw-1.1 (inherited from the source model)