GPTQ INT8 W8A8 quantized mistralai/Devstral-Small-2505 calibrated with a sequence len of 4096 and 128 samples of TokenBender/code_instructions_122k_alpaca_style, glaiveai/glaive-code-assistant-v2, google/code_x_glue_ct_code_to_text for a total sample size of 1024.

Follow the Devstral-Small-2505 docs for running with vllm.