You can deploy this model and start earning money today!

amanwalksdownthestreet/
Llama-3.3-70B-Instruct-MXFP8-KV8

Name: Llama-3.3-70B-Instruct-MXFP8-KV8
Availability: InStock

MXFP8 quantization of meta-llama/Llama-3.3-70B-Instruct (FP8 KV cache)

Quantized with llm-compressor 0.13.1.dev2+g20499a97b.d20260813 (MXFP8 W8A8, lm_head and embeddings retained at original precision).

8-bit (FP8) KV cache, calibrated against 2048 sequences of 2048 tokens from HuggingFaceH4/ultrachat_200k.

Requires Blackwell (SM100+). Older GPUs such as Ada, Hopper, etc. cannot run this model — MXFP8 matmul needs Blackwell's native microscaling support.

Serving with vLLM

bash

vllm serve amanwalksdownthestreet/Llama-3.3-70B-Instruct-MXFP8-KV8 \
    --max-model-len 82352

Weights occupy ~70 GiB. On a single 96 GB Blackwell card, the FP8 KV cache supports a context length of up to 82352 tokens — double the headroom of the BF16-KV variant (the model's native 131072 remains multi-GPU territory).