You can deploy this model and start earning money today!

NouraAlqasim/
aya-8b-fp8-gulf

Name: aya-8b-fp8-gulf
Availability: InStock

CohereLabs/aya-expanse-8b

quantized

Views

No views yet

aya-8b · FP8 · calibrated on Gulf dialect

Post-training W8A8 quantization with NVIDIA ModelOpt (FP8_DEFAULT_CFG).


base model	`CohereLabs/aya-expanse-8b`
base dtype	float16
precision	FP8 (W8A8)
calibration variety	Gulf dialect
calibration file	`calib2_gulf.txt`
calibration samples	128 dialogues, truncated at 128 tokens
calibration source	`Almheiri/ArabCulture-Dialogue` rev `9acd60cbbb4f`, seed 1448
weight MSE	2.930e-07
activation quantizers calibrated	224/224
environment	torch 2.8.0+cu128 · transformers 4.57.6 · modelopt 0.45.0

FP8 weight scales are data-free. What the calibration variety changes is the static per-tensor activation scales, exported as input_scale. Sibling checkpoints -fp8-msa, -fp8-gulf and -fp8-mixed differ in exactly that and nothing else.

Not loadable by plain transformers — config.json declares quantization type modelopt.

vllm serve NouraAlqasim/aya-8b-fp8-gulf --quantization modelopt