Quantized NVFP4 version of
Mistral-Nemo-Instruct-2407 with
Four Over Six adaptive block scaling, created to compare against my
hybrid quant. Made with the same version of llm-compressor and compressed-tensors, using the same calibration data, to isolate the variables as much as possible.
Four Over Six (4/6) is a modification to the block-scaled NVFP4 quantization algorithm that yields reduced quantization error. Unlike integer formats, floating point formats have non-uniform step sizes which create larger quantization error on larger values. 4/6 takes advantage of this by adaptively scaling some blocks to smaller FP4 values, making the distribution of representable values more uniform and reducing quantization error for near-maximal values.
1@misc{cook2025sixaccuratenvfp4quantization,
2 title={Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling},
3 author={Jack Cook and Junxian Guo and Guangxuan Xiao and Yujun Lin and Song Han},
4 year={2025},
5 eprint={2512.02010},
6 archivePrefix={arXiv},
7 primaryClass={cs.CL},
8 url={https://arxiv.org/abs/2512.02010},
9}
