You can deploy this model and start earning money today!

ksukrit/
qwen2-vl-2b-4bit

Name: qwen2-vl-2b-4bit
Availability: InStock

Views

No views yet

Qwen2-VL-2B-Instruct 4-bit Quantized

This is a 4-bit quantized version of the Qwen2-VL-2B-Instruct model.

Model Description

Original Model: Qwen/Qwen2-VL-2B-Instruct
Quantization: 4-bit quantization using bitsandbytes
Usage: This model is optimized for memory efficiency while maintaining performance
License: Same as original model

Usage

python

from transformers import Qwen2VLModel, AutoTokenizer
import torch

model = Qwen2VLModel.from_pretrained("ksukrit/qwen2-vl-2b-4bit", trust_remote_code=True, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("ksukrit/qwen2-vl-2b-4bit", trust_remote_code=True)

Quantization Details

Quantization Method: bitsandbytes 4-bit quantization
Compute dtype: float16
Uses double quantization: True
Quantization type: nf4