Views
No views yet
| Variant | File | Size | Description |
|---|---|---|---|
| INT8 | onnx/model_quantized.onnx | 573 MB | Dynamic INT8 quantization (default) |
| Q4F16 | onnx/model_q4f16.onnx | 517 MB | INT4 weights + FP16 activations |
| YesNo INT8 | onnx/model_yesno_quantized.onnx | 572 MB | YES/NO-only logits (last token, 150x smaller output) |
| YesNo Q4F16 | onnx/model_yesno_q4f16.onnx | 517 MB | YES/NO-only logits + INT4 weights |
1# INT8 (default)
2from qwen3_embed import TextCrossEncoder
3model = TextCrossEncoder("Qwen/Qwen3-Reranker-0.6B")
4
5# Q4F16 (smaller, slightly less accurate)
6model = TextCrossEncoder("Qwen/Qwen3-Reranker-0.6B-Q4F16")onnxruntime.quantization.quantize_dynamic (QInt8)MatMulNBitsQuantizer (block_size=128, symmetric) + FP16 cast