You can deploy this model and start earning money today!

ealexeev/
Mistral-Small-24B-NVFP4

Name: Mistral-Small-24B-NVFP4
Availability: InStock

Views

No views yet

Model Card for ealexeev/Mistral-Small-24B-NVFP4

This is a compressed version of Mistral Small 24B Instruct, quantized to NVFP4 using llm-compressor.

This model is optimized for NVIDIA Blackwell/Hopper GPUs (H100, B200) and vLLM.

Metric	Base Model (FP16)	This Model (NVFP4)	Delta
HellaSwag (Logic)	83.47%	83.20%	-0.27%
IFEval (Strict)	71.46%	70.50%	-0.96%
Throughput	712 tok/s	1344 tok/s	+1.88x

⚠️ This model requires vLLM. It will NOT work with GGUF/llama.cpp/Ollama.

bash

vllm serve ealexeev/Mistral-Small-24B-NVFP4 \
    --tensor-parallel-size 1 \
    --gpu-memory-utilization 0.8 \
    --enforce-eager