FABQ-RC (Fisher-Adaptive Binary Quantization with Residual Codebooks) is a 1-bit quantization method for large language models that adapts per layer rather than using a fixed blocksize. This quantization of Qwen3.6-27B achieves ~1.18 bits per parameter while preserving quality through four key innovations.

Property	Value
Base Model	Qwen/Qwen3.6-27B
Quantization Method	FABQ-RC
Format	Custom PyTorch state dict (`.pth`)
Bits per Parameter	~1.18 bpw
Precision Allocation	Top 5% channels → int8, 95% → binary ±1
Blocksize	Adaptive per-layer (selected from {16, 32, 64, 128, 256})
Residual Codebook	256-entry k-means centroids, per-layer
Architecture	Qwen3.5 hybrid (DeltaNet + Gated Attention), 27B dense, 64 layers, hidden 5120
Modalities	Text only (vision encoder not included)
Context Length	262,144 tokens

How FABQ-RC Works

Fisher-Weighted Channel Importance — Per-channel Fisher Information (expected gradient²) determines which channels matter for loss.
Mixed-Precision Core — Top 5% channels → int8, rest → binary ±1.
Adaptive Blocksize — Per-layer sweep {16, 32, 64, 128, 256}, picked by Fisher-weighted reconstruction error.
Residual Codebook — k-means (256 centroids) on quantization residuals corrects systematic binary bias.

Files in This Repo

File	Size	Description
`quantized_model.pth`	52.9 GB	FABQ-RC quantized state dict (int8 cores + binary weights + codebooks + Fisher masks)
`config.json`	4.3 kB	Architecture metadata, mirrors `Qwen/Qwen3.6-27B` text config
`generation_config.json`	202 B	Sampler defaults
`tokenizer.json`, `tokenizer_config.json`, `vocab.json`, `merges.txt`	~23 MB	Tokenizer (mirrored from base, unchanged)
`chat_template.jinja`	7.8 kB	Chat template (mirrored from base)
`README.md`	this file	Model card

Note: the quantized_model.pth is a custom format. Standard transformers / vLLM / llama.cpp will NOT load it. See Loading below.

Loading & Inference

The loader lives in the FABQ-RC project repo: toxzak/fabq-rc, specifically Main-FABQ-RC-Notebook.ipynb. The notebook contains the FABQ_RC_Model class that reconstructs the binary weights, applies the int8 cores, and runs the residual codebook lookup at inference time.

python

# Pseudocode — see Main-FABQ-RC-Notebook.ipynb for the real loader
import torch
from fabq_rc import FABQ_RC_Model  # from toxzak/fabq-rc repo

state_dict = torch.load("quantized_model.pth", map_location="cuda")
model = FABQ_RC_Model.from_quantized_state(state_dict).cuda()
output = model.generate("The future of 1-bit quantization is", max_new_tokens=128)

Requirements

PyTorch with CUDA (custom 1-bit matmul kernel; no native ops exist for the mixed int8/binary layout)
~16 GB VRAM minimum (1.18 bpw × 27B params ≈ 4 GB weights + activations)
transformers >= 4.57.1 for tokenizer/config parsing
The FABQ-RC loader from the project repo (NOT pip-installable as of this writing)

Why Not GGUF?

The companion bucket (toxzak/Qwen3.6-27B-Q1_K-GGUF-bucket) contains a 3.36 GB GGUF export of the binary weights only, but that export drops the int8 cores and the residual codebook. It is a partial export and will produce degraded output. The .pth in this repo is the complete artifact.

Limitations

Custom format: No standard runtime can load it without the FABQ-RC loader.
Weight-only quantization: Activations are not quantized.
No vision encoder: The vision tower from Qwen3.6-27B is not included; this artifact is text-only.
Single-domain calibration: C4 only; may not generalize to all domains.
Not llama.cpp compatible: Despite the bucket name, the .gguf shard in the companion bucket is not a runnable llama.cpp model.

Training / Quantization Details

Property	Value
Method	FABQ-RC (see spec)
Calibration	C4 dataset, 2048 samples
Sequence Length	32 tokens
Hardware	A100 80GB GPU
Method Origin	Designed by Zachary Maronek, 2026-04-05

Benchmark Comparison

Method	bpw	Perplexity	Notes
FP16	16.0	baseline	Qwen3.6-27B full precision
Q1_0_g128	1.125	degraded	Bonsai's format
BiLLM	1.08	~8.41 (70B)	Best prior work
FABQ-RC	~1.18	TBD	This artifact

Related Artifacts

toxzak/Qwen3.6-27B-Q1_K-GGUF-bucket — partial GGUF export (binary weights only)
toxzak/fabq-rc — full project repo with loader
toxzak/Qwen3.6-35B-A3B-FABQ-RC — 35B-A3B variant

Citation

bibtex

@misc{fabqrc2026,
    author = {Zach Maronek},
    title = {FABQ-RC: Fisher-Adaptive Binary Quantization with Residual Codebooks},
    year = {2026},
    url = {https://huggingface.co/toxzak/fabq-rc}
}

License

Apache 2.0 — same as the base model Qwen/Qwen3.6-27B.

FABQ-RC by Zach Maronek · 2026

toxzak/Qwen3.6-27B-FABQ-RC

Qwen3.6-27B-FABQ-RC

Model Description