FP8 (W8A8 dynamic) quantization of Qwen/Qwen3-4B, produced with llm-compressor in the compressed-tensors format for efficient serving with vLLM and SGLang.

The quantization scheme is declared in config.json and detected automatically — no extra flags are required. Compatible with any engine that reads compressed-tensors (vLLM, SGLang, …).

config.json 中已声明量化方案，引擎自动识别，无需额外参数。兼容一切支持 compressed-tensors 的推理框架。

<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

English — This checkpoint reproduces the base model's behavior at reduced precision, shrinking memory footprint and increasing throughput while keeping quality close to the original. It is a drop-in replacement for the base model in vLLM/SGLang deployments.

中文 — 本权重是原始模型的低精度量化版本，在保持接近原模型质量的同时，显著降低显存占用、提升吞吐。 在 vLLM/SGLang 部署中可直接替换原始模型使用。