You can deploy this model and start earning money today!

danielchalef/
Qwen3-Reranker-4B-seq-cls-vllm-fixed

Name: Qwen3-Reranker-4B-seq-cls-vllm-fixed
Availability: InStock

sequence-classification

vllm

text-classification

Qwen/Qwen3-Reranker-4B

finetune

apache-2.0

Views

No views yet

Qwen3-Reranker-4B-seq-cls-vllm-fixed

This is a fixed version of the Qwen3-Reranker-4B model converted to sequence classification format, optimized for use with vLLM.

Model Description

This model is a pre-converted version of Qwen/Qwen3-Reranker-4B that:

Has been converted from CausalLM to SequenceClassification architecture
Includes proper configuration for vLLM compatibility
Provides ~75,000x reduction in classification head size
Offers ~150,000x fewer operations per token compared to using the full LM head

Key Improvements

The original converted model (tomaarsen/Qwen3-Reranker-4B-seq-cls) was missing critical vLLM configuration attributes. This version adds:

json

{
  "classifier_from_token": ["no", "yes"],
  "method": "from_2_way_softmax",
  "use_pad_token": false,
  "is_original_qwen3_reranker": false
}

These configurations are essential for vLLM to properly handle the pre-converted weights.

Usage with vLLM

bash

vllm serve danielchalef/Qwen3-Reranker-4B-seq-cls-vllm-fixed \
    --task score \
    --served-model-name qwen3-reranker-4b \
    --disable-log-requests

Python Example

python

from vllm import LLM

llm = LLM(
    model="danielchalef/Qwen3-Reranker-4B-seq-cls-vllm-fixed",
    task="score"
)

queries = ["What is the capital of France?"]
documents = ["Paris is the capital of France."]

outputs = llm.score(queries, documents)
scores = [output.outputs.score for output in outputs]
print(scores)

Performance

This model performs identically to the original Qwen3-Reranker-4B when used with proper configuration, while providing significant efficiency improvements:

Memory: ~600MB → ~8KB for classification head
Compute: 151,936 logits → 1 logit per forward pass
Speed: Faster inference due to reduced computation

Technical Details

Architecture: Qwen3ForSequenceClassification
Base Model: Qwen/Qwen3-Reranker-4B
Conversion Method: from_2_way_softmax (yes_logit - no_logit)
Model Size: 4B parameters
Task: Reranking/Scoring

Citation

If you use this model, please cite the original Qwen3-Reranker:

bibtex

@misc{qwen3reranker2024,
  title={Qwen3-Reranker},
  author={Qwen Team},
  year={2024},
  publisher={Hugging Face}
}

License

Apache 2.0 (inherited from the base model)