vllm serve RedHatAI/Hy3-NVFP4-FP8 \
    --tensor-parallel-size 4 \
    --tool-call-parser hy_v3 \
    --enable-auto-tool-choice \
    --reasoning-parser hy_v3 \
    --port 8089

May require https://github.com/vllm-project/vllm/pull/48769 as a prerequisite

Creation Process

https://github.com/vllm-project/llm-compressor/pull/2928

Evaluation

Observed 100% recovery on a limited sample from

inspect eval hf/Idavidrein/gpqa/diamond   --model vllm/RedHatAI/Hy3-NVFP4-FP8 --reasoning-effort high --model-base-url http://localhost:8089/v1