You can deploy this model and start earning money today!

dolfsai/
Qwen3-Reranker-0.6B-seq-cls-vllm-W8A8

Original Model: tomaarsen/Qwen3-Reranker-0.6B-seq-cls
Quantization Method: GPTQ
Compression Libraries: <a href="https://github.com/vllm-project/llm-compressor" target="_blank" rel="noopener noreferrer" class="text-blue-400 hover:text-blue-300 underline underline-offset-2 transition-colors">llm-compressor
Calibration Dataset: ultrachat_200k (2048 samples)
Optimized For: Inference with vLLM
License: same as original model

Name: Qwen3-Reranker-0.6B-seq-cls-vllm-W8A8
Availability: InStock

Views

No views yet

prudant/Qwen3-Reranker-0.6B-seq-cls-W8A8

This is a compressed version of tomaarsen/Qwen3-Reranker-0.6B-seq-cls using llm-compressor with the following scheme: W8A8

python3 -m vllm.entrypoints.openai.api_server --model 'dolfsai/Qwen3-Reranker-0.6B-seq-cls-vllm-W8A8' --task classify

Important: You MUST read the following guide for correct usage of this model here Guide