ROCmFP4 quantization of badtheorylabs/BTL-4, built for AMD Strix Halo (gfx1151) with the ROCmFPX fork of llama.cpp.

BTL-4 is a 70.2 GB bf16 MoE vision model — Qwen3_5MoeForConditionalGeneration, 40 layers, 256 experts / 8 active, hidden 2048, vocab 248320. An mmproj is included and is required for image input.

LEAN is −12% on disk and +12% faster than Q4_K_M, at equal-or-better quality. Quality used greedy decoding with the baseline run twice to establish the noise floor (Δ=0 between passes).

A sister build, Q4_0_ROCMFP4_STRIX (4.31 BPW, 17.39 GiB), was produced independently on a second box and measured +11.4% over Q4_K_M at 10/10 = 10/10 quality. The two tiers are within ~67 MB and ~0.02 BPW of each other; prior art on this architecture family favours plain STRIX for quality-critical work.

Q3 quantizes larger than LEAN. Always run --dry-run and read the real BPW — the advertised value is not reliable.

Upstream BTL-4 ships mtp_num_hidden_layers: 0 — there are no MTP tensors. Do not pass a speculative/MTP flag. A spec flag with no draft tensors loads and reports READY while producing garbage.

Requires a ROCmFP4-capable llama.cpp build; stock llama.cpp will not load these tensor types.

Inherits the licence of the base model, badtheorylabs/BTL-4.

Compiled from Hugging Face repository metadata — file sizes, shipped files, quant variant as named by each repo. No third-party build was run or benchmarked here, so this table makes no speed or quality claim about any of them. It is here so you can see the size and format options at a glance and pick what fits your hardware.

This build would not exist without the work below. Please star and follow these projects — the quantisation format used here is their engineering, not mine.

ROCmFPX — maintained by charlie12345 / caf The ROCmFP4 / ROCmFPX tensor formats (ggml types 100–106) exist only in this fork. Every ROCmFP4 file in this repository was produced with its llama-quantize, and runs on its runtime. The fork also credits collaborators ciru-ai, Tom Turney, PlunderStruck and Aydan S., and acknowledges AMD for hardware support. Licensed MIT, based on upstream llama.cpp.

llama.cpp — ggml-org and contributors The inference engine, GGUF format and conversion tooling everything here is built on.

AMD ROCm The compute platform these builds target — ROCm 7.2.4 on gfx1151 / Radeon 8060S.

Base model authors — see base_model in the metadata above; all model weights, licences and capabilities are theirs. This repository contributes quantisation and measurement only.

If you use these files, please credit ROCmFPX alongside this repository.