xpuenabler/gpt-oss-15.5b-23E-SFT-v6-GpNd-separate-mode-decode. Only the mlp.experts weights
were re-quantized; everything else (INT4 attention/dense, FP16
embed_tokens / lm_head / router / norms, graph topology) is preserved.xpuenabler/gpt-oss-15.5b-23E-SFT-v6-GpNd-separate-mode-decodeMultiply_1 (Swish) → Multiply_2 link of the MoE block.mlp.router, lm_head, and embed_tokens are excluded from
quantization (kept in their original precision).mlp.experts quantization parameters (per-group scales and AWQ
activation-side scales) were computed once on the prefill graph using
NNCF compress_weights(mode=NF4, group_size=64, awq=True, scale_estimation=True) with 32 calibration samples at static input shape
[1, 128]. The resulting NF4 weight chains and AWQ multiplies were then
transplanted into the decode graph (which has the same expert weights
but different I/O static shapes), so the two graphs share identical
quantized parameters.| Stage | bin size |
|---|---|
| Source decode (INT8 experts) | 16.4 GB |
| This NF4 decode | 9.985 GB |