You can deploy this model and start earning money today!

NouraAlqasim/
llama3.1-8b-fp8-mixed

Name: llama3.1-8b-fp8-mixed
Availability: InStock

meta-llama/Llama-3.1-8B-Instruct

quantized

Views

No views yet

llama3.1-8b · FP8 · calibrated on 64 MSA + 64 Gulf

Post-training W8A8 quantization with NVIDIA ModelOpt (FP8_DEFAULT_CFG).


base model	`meta-llama/Llama-3.1-8B-Instruct`
base dtype	float16
precision	FP8 (W8A8)
calibration variety	64 MSA + 64 Gulf
calibration file	`calib3_mixed.txt`
calibration samples	128 dialogues, truncated at 128 tokens
calibration source	`Almheiri/ArabCulture-Dialogue` rev `9acd60cbbb4f`, seed 1448
weight MSE	1.786e-07
activation quantizers calibrated	224/224
environment	torch 2.8.0+cu128 · transformers 4.57.6 · modelopt 0.45.0

FP8 weight scales are data-free. What the calibration variety changes is the static per-tensor activation scales, exported as input_scale. Sibling checkpoints -fp8-msa, -fp8-gulf and -fp8-mixed differ in exactly that and nothing else.

Not loadable by plain transformers — config.json declares quantization type modelopt.

vllm serve NouraAlqasim/llama3.1-8b-fp8-mixed --quantization modelopt