Qwen3-8B (pinned revision b968826) quantized with AWQ (llm-compressor, W4A16 scheme: 4-bit weights, 16-bit activations), produced for sm120-quant-bench: a study of where 4-bit damage actually lands, measured with frozen splits, capability probes, and bootstrap 95% CIs on an RTX 5090. Write-up: "NVFP4 on a 5090 names two different machines".

Most quantized checkpoints ship with no damage documentation. This card tells you what this quantization costs, per capability, against the BF16 baseline measured on identical hardware with identical frozen evals.

Damage pools in computation, not structure. Full five-variant table with bootstrap CIs: the study.

Every number traces to a results JSON in the repo, and every eval run recorded which GPU kernel actually served it. This checkpoint's serving receipt on our stack (vLLM 0.26.0, sm_120):

Know what you are trading: by perplexity this checkpoint is identical to the GPTQ variant (19.73 vs 19.66). The capability probes disagree: it gives up ~6 points of HumanEval and ~7 of numeric fidelity relative to GPTQ. If your workload leans on code or arithmetic, prefer the GPTQ variant; if not, this one is equivalent and equally fast.

One of three checkpoints quantized and evaluated together in sm120-quant-bench; siblings: Qwen3-8B-W4A16-GPTQ and Qwen3-8B-NVFP4.

These two checkpoints sit 0.06 perplexity apart and 6.7 points apart on a numeric-fidelity probe. That gap is the study.

recipes/w4a16_awq.py (256 chat-templated ultrachat samples, max_seq 1024: the defaults OOM a 32 GB card). Calibration data is disjoint from every eval split by construction. Full recipe, harness, and raw results: the repo. Reproduction gets byte-identical eval items (seed-pinned, hashed manifest).

Check the log for the kernel-selection line before benchmarking. If a number matters, its serving receipt matters.