# Install hipfire
curl -L https://raw.githubusercontent.com/Kaden-Schutt/hipfire/master/scripts/install.sh | bash

# Pull and run any variant
hipfire pull qwen3.5:9b            # HF4 (default — fastest)
hipfire pull qwen3.5:9b-mq4        # MQ4 (quality-gated, near-Q8 output)
hipfire pull qwen3.5:9b-hf6        # HF6 (highest quality, ~15% slower)

hipfire run qwen3.5:9b-mq4 "Hello"

Quantization Formats

HF4 (HFQ4-G256) — flat 4-bit, 256-weight groups (~0.53 B/w including per-group scale + zero). Best raw tok/s. Same storage layout as Q4_K_M in llama.cpp but without the K-quant block descriptors.
HF6 (HFQ6-G256) — flat 6-bit, 256-weight groups (~0.78 B/w). Highest quality, ~15% slower than HF4. Use this if you have VRAM headroom and want the smallest accuracy loss vs FP16.
MQ4 (MagnumQuant 4-bit) ⭐ — FWHT-rotated 4-bit. Storage layout identical to HF4 (4.25 B/w), but the weights are pre-rotated through a Walsh–Hadamard transform at quantization time, and the input x vector is rotated through the same transform on the fly during the GEMV. The rotation flattens outliers, dramatically improving the quantization-error distribution. Result: roughly Q8-grade output quality at Q4 bandwidth.

Every commit that touches kernel or forward-pass code in the hipfire repo is gated against MQ4 byte-exact greedy decoding of 9 reference (model, prompt) pairs — see tests/quality-baselines and scripts/quality-gate.sh. Any silent numerical regression in the forward pass is caught at commit time.

All formats embed the tokenizer and model config inside the model file — no separate tokenizer.json download needed.

About hipfire

Rust + HIP inference engine for AMD consumer GPUs (RDNA1–RDNA4). No Python in the hot path. The 0.1.4-alpha branch lands a kernel-fusion overhaul that roughly doubles forward speed on Qwen3.5 across the lineup vs the previous release.

GitHub: Kaden-Schutt/hipfire
All models: docs/MODELS.md

License

Model weights subject to the original Qwen license. hipfire engine: MIT.

hipfire-models/qwen3.5-9b

Qwen3.5-9B for hipfire

Files

Usage

Quantization Formats

About hipfire

License

hipfire-models/
qwen3.5-9b