FP8 (E4M3) post-training quantization of GPT-OSS 20B, produced with block-wise GPTQ (Hessian-based, second-order weight quantization) on a parallel-Hessian calibration pass. Quantized from unsloth/gpt-oss-20b-BF16.

Method

Algorithm: block-wise GPTQ, blocksize 128, percdamp 0.01
Calibration: C4, 512 sequences × 2048 tokens (parallel Hessian collection)
Format: FP8 E4M3, per-channel. The quantization error is baked into the weights, which are stored in bfloat16 (simulated FP8), so the file size is ~40 GB and the model loads with a standard from_pretrained call.

Results (FP8 vs BF16, seq_len 2048)

Dataset	BF16 ppl	FP8 ppl	Δppl
WikiText-2	229.4992	228.1501	−1.3490
C4	719.8874	720.1364	+0.2489

FP8 matches the BF16 baseline within threshold on both datasets.

The absolute perplexity is inflated by a long-context degradation in GPT-OSS's 2048-token windowed evaluation. This affects the BF16 and models identically, so Δppl is the meaningful metric — the absolute values are not comparable across models.

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("anirudmohan26/gpt-oss-20
tok   = AutoTokenizer.from_pretrained("anirudmohan26/gpt-oss-20B-FP8")