This is a quantized version of deepseek-ai/DeepSeek-V4-Pro with MoE layers quantized to NVFP4 and attention layers quantized to FP8 block

This model is intended for deployment with vLLM and requires the following branch: https://github.com/vllm-project/vllm/pull/41276. You can serve the model using

This model was created using LLM Compressor. The example script can be found in examples/quantizing_moe/deepseek_v4_pro_example.py [DSV4] DeepSeekV4 Pro. Quantizing the model with data parallelism and 6xA100 takes about 3 hours.