Views
No views yet
Qwen/Qwen3-Embedding-0.6B.llm-compressor using FP8
dynamic activation quantization for the Qwen3 embedding backbone.Qwen/Qwen3-Embedding-0.6Bllm-compressorcompressed-tensorsFP8_DYNAMICLinear1from llmcompressor import oneshot
2from llmcompressor.modifiers.quantization import QuantizationModifier
3
4recipe = QuantizationModifier(
5 targets="Linear",
6 scheme="FP8_DYNAMIC",
7)
8
9oneshot(model=model, recipe=recipe)
10
11model.save_pretrained("Qwen3-Embedding-0.6B-FP8")
12tokenizer.save_pretrained("Qwen3-Embedding-0.6B-FP8")