Llama-3.2-1B-Instruct — configuration c5 (W3A16) — SIMULATED

⚠️ This is a simulated (quantize–dequantize) checkpoint, stored densely in bf16.

The weights have been rounded onto a 3-bit grid by real GPTQ and then written back at full precision. It is weight-exact but not kernel-exact: it gives you the accuracy of 3-bit quantization with none of the memory or latency benefit, and the file is the same size as the bf16 original.

Why: vLLM 0.11's WNA16_SUPPORTED_TYPES_MAP is {4: uint4b8, 8: uint8b128} — there is no 2- or 3-bit kernel, so these bit-widths cannot be executed natively at all. Simulation is the only way to measure them on the same inference stack.

Grid verified post-hoc — distinct values per weight tensor: [['model.layers.0.self_attn.q_proj', 7], ['model.layers.0.self_attn.k_proj', 6], ['model.layers.0.self_attn.v_proj', 7]] (≤ 2^3 = 8).

The simulation gap is quantified: real c2 vs simulated c2sim differ by 0.0058 pooled ASR, against 0.1491 for either versus the unquantized model.