You can deploy this model and start earning money today!

NouraAlqasim/
qwen2.5-7b-fp8-mixed

Name: qwen2.5-7b-fp8-mixed
Availability: InStock

Qwen/Qwen2.5-7B-Instruct

quantized

Views

No views yet

qwen2.5-7b · FP8 · calibrated on 64 MSA + 64 Gulf

Post-training W8A8 quantization with NVIDIA ModelOpt (FP8_DEFAULT_CFG).


base model	`Qwen/Qwen2.5-7B-Instruct`
base dtype	float16
precision	FP8 (W8A8)
calibration variety	64 MSA + 64 Gulf
calibration file	`calib3_mixed.txt`
calibration samples	128 dialogues, truncated at 128 tokens
calibration source	`Almheiri/ArabCulture-Dialogue` rev `9acd60cbbb4f`, seed 1448
weight MSE	2.050e-07
activation quantizers calibrated	196/196
environment	torch 2.8.0+cu128 · transformers 4.57.6 · modelopt 0.45.0

FP8 weight scales are data-free. What the calibration variety changes is the static per-tensor activation scales, exported as input_scale. Sibling checkpoints -fp8-msa, -fp8-gulf and -fp8-mixed differ in exactly that and nothing else.

Not loadable by plain transformers — config.json declares quantization type modelopt.

vllm serve NouraAlqasim/qwen2.5-7b-fp8-mixed --quantization modelopt