Views
No views yet
FP8_DEFAULT_CFG); activation quantizers were
disabled after calibration, so the exported model is weight-only.| base model | google/gemma-2-9b-it |
| base dtype | float16 |
| mode | W8A16 — FP8 weights (E4M3, per-tensor scale), activations FP16 |
| calibration | calib3_mixed.txt, 512 samples, 512 tokens, variety mixed |
| calibration reaches weights | no — see below |
| weight MSE | 3.163557e-08 |
| weight quantizers | 294 |
| environment | torch 2.8.0+cu128 · transformers 4.57.6 · modelopt 0.46.0 |
calib_stats.json only.transformers — config.json declares quantization type modelopt.vllm serve Azzam15/gemma2-9b-fp8-w8a16-mixed --quantization modelopt