Step-3.7-Flash — ROCmFP4 STRIX_LEAN GGUF — AMD Ryzen AI Max+ 395 / Strix Halo / gfx1151
✅ only public build shipping the vision projector
Verified against Hugging Face repository metadata for all 2 public ROCmFP4 builds of this base model. Size and file facts only — no third-party build was benchmarked.
This is a ROCmFP4 "Strix Lean" quant of
stepfun-ai/Step-3.7-Flash, built and tested on a Ryzen AI Max+ 395 (Strix Halo, gfx1151, 128 GB unified memory, ROCm 7.2.4).
I run a lot of big models on Strix Halo hardware, and the ROCmFP4 format has been the best decode-speed lever I've found for it. Every ROCmFP4 build I could find was Qwen, Ornith, Laguna or Tess class, so I made one for Step-3.7-Flash. As far as I can tell this is the first one.
Short version: it's faster than IQ4_XS at both ends — about 18–20% more decode throughput and 8–12% more prefill — with equivalent output quality. It costs about 10% more disk.
Requires the ROCmFP4 fork
Q4_0_ROCMFP4_STRIX_LEAN is not a stock llama.cpp quant type. These files will not load on upstream llama.cpp, Ollama, LM Studio, or anything else that hasn't got the ROCmFP4 kernels. You need a build of the fork:
1git clone https://github.com/charlie12345/ROCmFPX
2cd ROCmFPX
3HIPCXX=$(hipconfig -l)/clang HIP_PATH=$(hipconfig -R) cmake -B build \
4 -DGGML_HIP=ON -DGPU_TARGETS=gfx1151 -DGGML_HIP_ROCWMMA_FATTN=ON \
5 -DGGML_HIP_NO_VMM=ON -DGGML_HIP_MMQ_MFMA=ON -DCMAKE_BUILD_TYPE=Release \
6 -DGGML_VULKAN=OFF -DLLAMA_BUILD_WEBUI=OFF
7cmake --build build -j 4
Keep the job count low. On a 128 GB unified-memory box the HIP flash-attention kernels spike RAM hard at -j 8 or higher and will OOM the machine.
Files
| File | Size |
|---|
Step-3.7-Flash-Q4_0_ROCMFP4_STRIX_LEAN-00001-of-00003.gguf | 41.7 GiB |
Step-3.7-Flash-Q4_0_ROCMFP4_STRIX_LEAN-00002-of-00003.gguf | 41.5 GiB |
Step-3.7-Flash-Q4_0_ROCMFP4_STRIX_LEAN-00003-of-00003.gguf | 14.6 GiB |
mmproj-Step-3.7-Flash-f16.gguf | 3.7 GiB |
97.7 GiB total for the model, 4.26 BPW. Point llama.cpp at the first shard and it picks up the rest automatically. The mmproj file is the original f16 vision projector from StepFun — you only need it if you want image input.
Running it
1env LD_LIBRARY_PATH=/path/to/ROCmFPX/build/bin:/opt/rocm/lib \
2 HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1 \
3 /path/to/ROCmFPX/build/bin/llama-server \
4 --host 127.0.0.1 --port 8080 \
5 --model Step-3.7-Flash-Q4_0_ROCMFP4_STRIX_LEAN-00001-of-00003.gguf \
6 --mmproj mmproj-Step-3.7-Flash-f16.gguf \
7 --n-gpu-layers 999 --flash-attn on -dio --no-warmup --jinja \
8 --ctx-size 32768 --cache-type-k q8_0 --cache-type-v q8_0 \
9 --parallel 1 --temp 0.6 --top-p 0.95 --reasoning-budget 512
Three things in there are not optional, and each one cost me time before I worked it out:
-dio. Without direct I/O the mmap path crawls up to around 120 GB RSS and sits there looking hung. It isn't hung and the file isn't corrupt, it's just the loader. With -dio it cold-loads in about 60 seconds.
The env block. HSA_OVERRIDE_GFX_VERSION=11.5.1 and unified memory are needed for gfx1151, and putting the fork's build/bin first on LD_LIBRARY_PATH avoids a soname clash if you also have a Vulkan llama.cpp build on the box.
--reasoning-budget 512. Step-3.7-Flash thinks a lot. Left alone it will spend your entire max_tokens on chain-of-thought and hand you back an empty content field with finish_reason: length, which looks exactly like a broken quant. It isn't. Cap the thinking and the answer lands. 512 works well for chat; raise it for hard reasoning work.
Benchmarks
Both quants cold-loaded one at a time on the same machine, same binary, same environment, same sampling. Every run generated exactly 256 tokens so decode numbers are actually comparable, and every prompt started with a unique nonce so nothing was served out of the prefix cache. Three runs each, medians below.
| This quant | UD-IQ4_XS | Difference |
|---|
| Prefill @ 8K | 331 tok/s | 296 tok/s | +12% |
| Decode @ 8K | 18.8 tok/s | 15.9 tok/s | +18% |
| Prefill @ 32K | 283 tok/s | 261 tok/s | +8% |
| Decode @ 32K | 14.9 tok/s | 12.4 tok/s | +20% |
| Size | 97.7 GiB | 88.8 GiB | +10% |
Compared against
unsloth/Step-3.7-Flash-GGUF UD-IQ4_XS, tested at 32K context with q8_0 KV cache.
Be fair about the size column. This is a ~4.26 BPW quant and UD-IQ4_XS is roughly 3.9 BPW, so it's a higher-bit quant winning on speed while costing more disk. If you're comparing against Q4_K_M instead, ROCmFP4 normally comes out around 12–18% smaller. Against a sub-4-bit IQ quant it does not, and I'd rather say that up front than let the table imply a free lunch.
Quality spot checks came out equivalent, not identical — same correct answers on a palindrome-function task and a reasoning riddle, with different wording. Worth noting that IQ4_XS was noticeably more verbose in its chain-of-thought and needed a higher token budget before its final answer appeared, which is a practical point in this quant's favour if you're serving it.
How it was made
1llama-quantize --allow-requantize \
2 Step-3.7-Flash-Q8_0-00001-of-000NN.gguf \
3 Step-3.7-Flash-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
4 Q4_0_ROCMFP4_STRIX_LEAN 8
Quantized from the official Q8_0 GGUF, not from BF16. The BF16 weights are 394 GB and there was no way to hold them alongside the output on this machine, so this is a requantization and --allow-requantize was required. I'd rather disclose that than have someone assume an F16 source. It measured and benched clean, but if you want a from-BF16 build, this isn't it.
Verified after quantizing: coherent output on a load proof before anything got deleted, the full benchmark run above, and image input confirmed working through the mmproj.
Model
Step-3.7-Flash is StepFun's 198B MoE — 196B language model plus a 1.8B vision encoder, roughly 11B active parameters, 256K native context. Everything about the model's behaviour, prompt format and capabilities comes from upstream; see the
base model card. All I've changed is the numeric format.
Apache 2.0, same as the base model.
Credit
The ROCmFP4 format and kernels are not my work — that's the
ROCmFPX fork. I built the quant, ran the benchmarks and wrote them down.
Other public builds of this model
Compiled from Hugging Face repository metadata — file sizes, shipped files, quant variant as named by each repo. No third-party build was run or benchmarked here, so this table makes no speed or quality claim about any of them. It is here so you can see the size and format options at a glance and pick what fits your hardware.
Base model: stepfun-ai/Step-3.7-Flash. Generated from Hub metadata; download counts move over time.
Acknowledgements
This build would not exist without the work below. Please star and follow these
projects — the quantisation format used here is their engineering, not mine.
ROCmFPX — maintained by
charlie12345 / caf
The ROCmFP4 / ROCmFPX tensor formats (ggml types 100–106) exist only in this fork.
Every ROCmFP4 file in this repository was produced with its
llama-quantize, and
runs on its runtime. The fork also credits collaborators
ciru-ai,
Tom Turney,
PlunderStruck and
Aydan S., and acknowledges AMD for hardware support.
Licensed MIT, based on upstream llama.cpp.
llama.cpp — ggml-org and contributors
The inference engine, GGUF format and conversion tooling everything here is built on.
AMD ROCm
The compute platform these builds target — ROCm 7.2.4 on gfx1151 / Radeon 8060S.
Base model authors — see base_model in the metadata above; all model weights,
licences and capabilities are theirs. This repository contributes quantisation and
measurement only.
If you use these files, please credit ROCmFPX alongside this repository.
