Views
No views yet
Qwen2.5-VL-3B-Instruct 的多模态座舱助手模型，采用 GPTQ INT8 量化，面向端侧或轻量部署场景。Qwen2.5-VL-3B-Instruct and quantized with GPTQ INT8 for lightweight or on-device deployment.content 数组一起传入。content array.1{
2 "messages": [
3 {
4 "role": "user",
5 "content": [
6 {"type": "image"},
7 {"type": "text", "text": "请根据图像描述驾驶员状态并给出功能建议。"}
8 ]
9 }
10 ]
11}AutoProcessor 和 Qwen2_5_VLForConditionalGeneration。AutoProcessor and Qwen2_5_VLForConditionalGeneration.1from PIL import Image
2from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration
3
4repo_id = "qualcomm-ai-hub-community/OpenSparX-3b-cabin-sft-v2-gptq-int8"
5processor = AutoProcessor.from_pretrained(repo_id, trust_remote_code=True)
6model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
7 repo_id,
8 torch_dtype="auto",
9 device_map="auto",
10 trust_remote_code=True,
11)
12
13image = Image.open("/path/to/your/image.jpg").convert("RGB")
14messages = [
15 {
16 "role": "user",
17 "content": [
18 {"type": "image"},
19 {"type": "text", "text": "Describe the driver state and suggest actions."},
20 ],
21 }
22]
23
24text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
25inputs = processor(text=[text], images=[image], return_tensors="pt")
26inputs = inputs.to(model.device)
27
28generated_ids = model.generate(**inputs, max_new_tokens=128)
29generated_ids = [
30 output_ids[len(input_ids):]
31 for input_ids, output_ids in zip(inputs.input_ids, generated_ids)
32]
33print(processor.batch_decode(generated_ids, skip_special_tokens=False)[0])model.safetensors：量化后的单文件权重。config.json：模型结构配置，基座为 Qwen2.5-VL-3B-Instruct。quantize_config.json：GPTQ INT8 量化配置。preprocessor_config.json：视觉预处理配置。chat_template.json：对话模板。model.safetensors: quantized single-file model weights.config.json: model architecture config based on Qwen2.5-VL-3B-Instruct.quantize_config.json: GPTQ INT8 quantization settings.preprocessor_config.json: visual preprocessing config.chat_template.json: chat template used for prompt construction.