You can deploy this model and start earning money today!

onurulu17/
Qwen2.5-VL-7B-CXR

Name: Qwen2.5-VL-7B-CXR
Availability: InStock

safetensors

Views

No views yet

Qwen2.5-VL-7B-CXR

Paper: Link to PDF

Model Description

This is a Vision-Language model (VLM) based on Qwen2.5-VL-7B-Instruct, fine-tuned on MIMIC-CXR chest X-ray dataset using QLoRA PEFT. The model can analyze chest X-ray images and provide findings and impressions in a clear, concise, and medically relevant manner.

Base model: Qwen/Qwen2.5-VL-7B-Instruct
Fine-tuned with: LoRA PEFT on MIMIC-CXR dataset
License: Apache 2.0

Intended Use

Automatic chest X-ray report generation
Radiology VQA and clinical decision support
Research purposes for medical image-language tasks

Caution: Use under supervision of medical professionals; outputs are for informational purposes and not a substitute for professional medical advice.

How to Use

python

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from peft import PeftModel
from qwen_vl_utils import process_vision_info
import torch

base_model_id = "Qwen/Qwen2.5-VL-7B-Instruct"
adapter_id = "onurulu17/Qwen2.5-VL-7B-CXR"

# Load base model
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    base_model_id,
    device_map="auto",
    torch_dtype=torch.bfloat16,
)

# Load LoRA adapter
model = PeftModel.from_pretrained(model, adapter_id)

# Processor
processor = AutoProcessor.from_pretrained(base_model_id)

# Example inference
def generate_text_from_sample(model, processor, sample, max_new_tokens=1024, device="cuda"):
    text_input = processor.apply_chat_template(sample[:1], tokenize=False, add_generation_prompt=True)
    image_inputs, _ = process_vision_info(sample)
    model_inputs = processor(text=[text_input], images=image_inputs, return_tensors="pt").to(device)
    generated_ids = model.generate(**model_inputs, max_new_tokens=max_new_tokens)
    trimmed_generated_ids = [out_ids[len(in_ids):] for in_ids, out_ids in zip(model_inputs.input_ids, generated_ids)]
    output_text = processor.batch_decode(trimmed_generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)
    return output_text[0]

# Example usage
sample = [
{'role': 'user',
  'content': [{'type': 'image',
    'image': "./chest_xray_image.jpg"},
   {'type': 'text',
    'text': 'Please analyze this chest X-ray and provide the findings and impression.'}]},
 ]

output = generate_text_from_sample(model, processor, sample)
print(output)