Qwen3.5-27B-NVFP4-QAD-W4A16 (final, step 2640) — weight-only serving schema
NVFP4 weight-only (W4A16-serving) export of the NV-W4A16 QAD arm final
checkpoint from the stage-3 W4A4-vs-W4A16 training study on dense
Qwen/Qwen3.5-27B.
Training provenance
- Recipe: QAD temp-1.0 pure-KL distillation (teacher = BF16 Qwen3.5-27B),
QATFactory @
8e7cf1a + the _save_checkpoint empty_cache+barrier fix +
the 2026-07-24 context patches (rank-local RNG state save/load;
torch.cuda.set_device(LOCAL_RANK) at entry).
- Config:
configs/llm/qwen3_5_27b_nvfp4_qad.yaml (W4A16 fake-quant
training), lr 1.0e-5 cosine (1% warmup), 2640 steps (~1 epoch),
global batch 32 (bs 4/GPU x 8 GPUs x accum 1), seq len 8192, seed 42,
BF16, FSDP2, save/250, eval/100.
- Data: openperfectblend_100k Qwen3.5-9B-think train/eval JSONLs
(train md5
0406bb3a7a482352360716a1bc5e9e04; disjoint eval split).
- Hardware: 8x B200 (research-common-b200-jbom-5c-01). Run completed
2026-07-25 02:32:08 PDT, exit 0. The run resumed twice from PVC/NVMe
checkpoints after infra incidents (disk-full at s750, context-OOM at
s1071); resumed KL trajectories matched pre-crash values within 1%.
Result headline
Final held-out eval (epoch 1.0, step 2640): fwd-KL 0.005815 vs the BF16
teacher (top-1 agreement 0.976). The matched NV-W4A4-trained twin
plateaued at fwd-KL ~0.0135 from s1700 onward — for NVFP4 serving, W4A16
training reached 2.3x lower KL than W4A4 training at identical steps,
lr, data, and seed.
Reproduction (export step)
1python scripts/export_nvfp4_vllm.py \
2 --source /scratch/wxu/mxfp4/outputs/27b-nvfp4-w4a16-lr1.0e-5-2640 \
3 --model-assets /scratch/wxu/mxfp4/models/Qwen3.5-27B \
4 --output /scratch/wxu/mxfp4/exports/27b-nvw4a16-s2640-weightonly \
5 --device cuda:0 --weight-only
Load-smoked with vLLM 0.25.1 (vllm serve <dir> --max-model-len 4096):
loads clean and completes text correctly. First engine start pays a
FlashInfer fp4_gemm autotune (~10-25 min); subsequent starts reuse the cache.
Benchmarks
7-benchmark protocol results (RULER-32k, AIME'25 avg@4, GPQA-Diamond,
MMLU-Pro, HumanEval, IFEval, perplexity) will be added to this card when the
matrix wave for arm finals completes.
