You can deploy this model and start earning money today!

inference-optimization/
ctest-Qwen3-8B-speculator.dflash-fp8ablation-fp8

Name: ctest-Qwen3-8B-speculator.dflash-fp8ablation-fp8
Availability: InStock

Views

No views yet

ctest: Qwen3-8B DFlash speculator — FP8 hidden-states ablation (fp8)

Test checkpoint — not a production release. Trained to validate the FP8-quantized hidden-states transfer backend added in vllm-project/speculators#1028 (draft, reviving #491).

Verifier: Qwen/Qwen3-8B
Algorithm: dflash
Hidden-states precision during training: fp8 (new FP8 backend, per-token float8_e4m3fn quantization)
Training data: 5K magpie + 5K ultrachat samples from inference-optimization/Dataset-Qwen3-235B-Instruct

This is one of a matched pair (...-fp8ablation-bf16 / ...-fp8ablation-fp8) trained identically except for the hidden-states transfer precision, to isolate the effect of FP8 quantization on speculator quality. See the sibling repo for the other precision.

Validation metrics (final epoch)

loss_epoch: 0.669430
full_acc_epoch: 0.190271
position_1_acc_epoch: 0.672740
position_2_acc_epoch: 0.472040
position_3_acc_epoch: 0.345522
position_4_acc_epoch: 0.260192
position_5_acc_epoch: 0.199753
position_6_acc_epoch: 0.156152
position_7_acc_epoch: 0.124998
position_8_acc_epoch: 0.103757
position_9_acc_epoch: 0.089780
position_10_acc_epoch: 0.080512
position_11_acc_epoch: 0.074648
position_12_acc_epoch: 0.071026
position_13_acc_epoch: 0.067589
position_14_acc_epoch: 0.064986
position_15_acc_epoch: 0.062941
eal_epoch: 1.169300

guidellm serving eval

See acceptance.csv in this repo for the full per-subset guidellm breakdown (9 subset rows).

Full ablation writeup, all 6 checkpoints' results, exact commands, and code: see the results package referenced from PR #1028.

inference-optimization/ctest-Qwen3-8B-speculator.dflash-fp8ablation-fp8

ctest: Qwen3-8B DFlash speculator — FP8 hidden-states ablation (fp8)

Validation metrics (final epoch)

guidellm serving eval

inference-optimization/
ctest-Qwen3-8B-speculator.dflash-fp8ablation-fp8