This is an extremely high quality FPX3 / ROCmFPX Q3 GGUF build of stepfun-ai/Step-3.7-Flash, tuned for AMD Strix Halo local serving with Step MTP.
The goal is simple: keep Step 3.7 Flash useful at 256K context, keep the quality as high as possible, and keep it as small as possible. This release is a true tight Q3-weight build: 3.57 BPW, 81.77 GiB of language-model shards, and strong agent/tool behavior in local evals.
Use this if you want the Step 3.7 behavior profile, MTP support, and a much smaller local footprint than the stock GGUF Q3_K_L or ROCmFP4 STRIX_LEAN builds.
Required runtime: these GGUFs do not run on stock upstream llama.cpp. They use ROCmFPX tensor types such as q3_0_rocmfpx plus Chadrock/ROCmFPX serving support for Step MTP. Build the pinned Ciru ROCmFPX runner below before trying to load the model.
Why This One
Step 3.7 is huge. The practical local problem is not only speed; it is fitting enough context, KV, and agent workload into memory.
This FPX3/Q3 QualityPlus recipe was built for that constraint:
3.57 BPW effective language-model size
81.77 GiB total language GGUF shards
16.31% smaller than the local ROCmFP4 STRIX_LEAN build
14.35% smaller than StepFun's original Q3_K_L GGUF split
up to 256K one-slot serving profile with q8_0 target KV and q8_0 draft KV
Step MTP Q8 draft support through draft-mtp
downloadable fixed Step tool/chat template using native tool_response observations and protocol-boundary escaping
In practice, the original StepFun Q3_K_L local split was not a compact 3-bit-feeling model: it measured about 95.46 GiB, or roughly 4.17 BPW by effective size. This QualityPlus build is the one I would publish/use as the FPX3 lane.
Size Comparison
Measured from local GGUF shards:
Build
Effective BPW
Shard total
Difference vs this release
ROCmFPX Q3 QualityPlus
3.57 BPW
81.77 GiB
baseline
StepFun original Q3_K_L
~4.17 BPW
95.46 GiB
+13.70 GiB larger
ROCmFP4 STRIX_LEAN
~4.27 BPW
97.70 GiB
+15.93 GiB larger
That size gap matters because Step 3.7 needs memory for long context, q8 KV, and MTP draft state. On the tested Strix Halo host, the Q3 QualityPlus 64K MTP profile used about 96.3 GiB peak pooled GPU memory during long tool/Hermes runs, leaving enough RAM headroom to run the evals cleanly.
Quality Highlights
This is not a throwaway low-bit build. The recipe protects the tensors that were most important for behavior while pushing the giant expert FFN tensors into q3_0_rocmfpx.
Local quality results on AMD Ryzen AI Max+ 395 / Strix Halo:
At 128K, MTP initialized but produced no accepted drafts in that particular row, so treat the 128K decode number as an effective no-draft long-context decode reference.
256K load proof:
Context
Proof
Memory state
262144
target + Q8 MTP draft loaded, one slot, draft-mtp, /v1/models reports n_ctx=262144 and n_ctx_train=262144
~99.04 GiB pooled GPU used, ~16 GiB system RAM available
The 256K row is a load/allocation proof, not a 256K prompt prefill benchmark.
Files
Published shard names intentionally match the model name:
The Step MTP draft model is not duplicated here. If you enable draft-mtp, you
must also download and pass the separate Q8 draft from
notSnix/Step-3.7-Flash-MTP-Draft-GGUF,
for example Step-3.7-Flash-MTP-Q8_0.gguf. The main Q3 target GGUF does not
contain the MTP draft layers.
This repo also includes the tested chat/tool template:
This model is tied to the Charlie/Ciru ROCmFPX llama.cpp runner family. A stock llama-server will not understand the ROCmFPX tensor types in these shards and will not reproduce the MTP serving behavior used for the benchmark rows.
Use the pinned Ciru runner:
text
1repo: https://github.com/ciru-ai/ROCmFPX
2current recommended pin: c14872f8407f5b0642f3c395439b929021df753a
3branch at time of pin: main
4upstream lineage: charlie12345/ROCmFPX
This pin includes the upstream StepFun pre-render whitespace fix from
ggml-org/llama.cpp@2d97363.
It trims accumulated whitespace from prior content, reasoning, and typed text
parts before the Step template is rendered, preventing the progressive
Let me ... Actually ... Wait ... reasoning loops seen in long multi-turn tool
sessions. Keep using the downloadable fixed template below; the runner-side fix
also protects clients that fall back to the GGUF's embedded Step template.
The earlier Chadrock v2 speed-runner tag remains useful for historical comparison:
The Q3 QualityPlus Step 3.7 rows on this card were validated with the Chadrock/ROCmFPX runner path on AMD Ryzen AI Max+ 395 / Strix Halo. For fresh installs, use the current Ciru pin above unless you are reproducing an older benchmark exactly.
Build the runner on a Linux system with a working ROCm/HIP toolchain, Vulkan
development headers, CMake, and a C++ compiler. This is the pinned Strix Halo
reference build used by Ciru; it is not a universal distro installer, so package
names and ROCm paths may differ on Ubuntu, Arch, Fedora, NixOS, and other
distros.
If your ROCm or rocWMMA headers live outside the script defaults, set the relevant environment variables before running the build, for example ROCM_WMMA_INCLUDE=/path/to/rocWMMA/library/include. If your GPU is not Strix Halo / gfx1151, change CMAKE_HIP_ARCHITECTURES for your target.
The script and build directory still use the historical rocmfp4 name, but this is the ROCmFPX/Chadrock runner. For this model, the required support is ROCmFPX Q3 tensor support, not a ROCmFP4-only runtime.
The server binary should be:
./build-strix-rocmfp4/bin/llama-server
Again, build-strix-rocmfp4 is the historical build-directory name used by the ROCmFPX runner script.
If the model load fails with an unknown GGUF tensor type, you are using the wrong runner.
Recommended Serving Profile
The locally tested long-context profile:
text
1context: up to 262144
2slots: 1
3backend: Vulkan0 target + Vulkan0 draft
4MTP: --spec-type draft-mtp
5draft model: Step-3.7-Flash-MTP-Q8_0.gguf from notSnix/Step-3.7-Flash-MTP-Draft-GGUF
6speculative.n_max: 2
7speculative.n_min: 0
8speculative.p_min: 0.75
9speculative.p_split: 0.10
10batch / ubatch: 8192 / 2048
11target KV: q8_0 / q8_0
12draft KV: q8_0 / q8_0
13prompt cache: disabled for 256K fit runs
14sampler: temperature 1.0, top_p 0.95, min_p 0.0, repeat_penalty 1.0
15reasoning: on, DeepSeek format
16chat template: Step native tool_response template with protocol-boundary escaping
Temperature Guidance
For coding agents, tool calls, patches, exact filenames, JSON, and reading source files, use a lower temperature than the general profile above:
text
1temperature: 0.0 to 0.1
2top_p: 1.0
3min_p: 0.0
4repeat_penalty: 1.0
Use temperature: 0.0 when exactness and reproducibility matter most, or 0.1 for a small amount of flexibility. Higher temperatures such as 0.7 to 1.0 are better reserved for brainstorming, varied prose, and creative exploration; they can make Step less reliable when reproducing identifiers or interpreting long tool/file output.
Serving backend note: on the tested AMD Ryzen AI Max+ 395 / Strix Halo system, this Step 3.7 Q3 build worked best through the ROCmFPX/Chadrock runner serving on Vulkan0 for both target and draft. In the command below, ROCmFPX is the required tensor/runtime support; -dev Vulkan0 and --spec-draft-device Vulkan0 are the recommended serving backend.
For models.ini-style launchers, make sure the draft path is present. Setting
spec-type = draft-mtp without spec-draft-model makes the runner try to build
an MTP draft context from the main target GGUF, which fails because the target
does not contain MTP draft layers.
The best local Step setup uses the included step37-native-tool-response-template.jinja template. It renders tool outputs as tool_response turns and escapes protocol-boundary tokens inside tool output. This is a general protocol-adapter fix: tool/file/search results stay observations instead of being flattened into user text.
That matters for real agents because Step 3.7 can otherwise confuse tool output with conversation authority, especially in file/search-result injection cases.
Build Notes
These are model-build notes, not runner-build instructions. Build the pinned ROCmFPX runner in the section above before serving the GGUFs.
Pinned public runner fork and build recipe: ciru-ai/ROCmFPX, current recommended pin c14872f8407f5b0642f3c395439b929021df753a
Quantization, the ROCmFPX Step 3.7 Q3 QualityPlus recipe, Strix Halo profile, and local benchmark work: Crown / Ciru
Caveats
This is a custom ROCmFPX GGUF release. It requires the compatible ROCmFPX/Chadrock llama.cpp runner; stock llama.cpp is not expected to load it.
Quality numbers are local Strix Halo measurements and depend on runtime, chat template, KV type, and MTP settings.
The model is strong but not perfect at autonomous email/message side effects; it can be cautious and ask for subject/body/recipient details instead of sending with inferred defaults.
