ISTA-DASLab/

Llama-3.1-8B-Instruct-FPQuant-QAT-NVFP4

This is the official QAT FP-Quant checkpoint of meta-llama/Llama-3.1-8B-Instruct, produced as described in the "Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization" paper.

This model can be run on Blackwell-generation NVIDIA GPUs via QuTLASS and FP-Quant in either transformers or vLLM.

The approximate recipe for training this model (up to local batch size and LR) is available here.

This checkpoint has the following performance relative to the original model and the RTN quantization: