Uploaded model
- Developed by: Yash983487
- License: apache-2.0
- Finetuned from model : unsloth/llama-3.2-11b-vision-instruct-unsloth-bnb-4bit
This mllama model was trained 2x faster with
UnslothLlama-3.2-11B-Vision-Radiology-LoRA
This is a fine-tuned version of Llama-3.2-11B-Vision-Instruct specialized for medical radiography. It was trained using Unsloth and LoRA (Low-Rank Adaptation) to analyze X-ray images and generate descriptive medical reports.
🚀 How to Use (Avoid Errors)
Because this is a LoRA Adapter, the standard "Use in Transformers" button on Hugging Face will not work directly. You must load the adapter onto the base model using the code below.
- Install Required Libraries
Bash
pip install -U unsloth bitsandbytes transformers accelerate
- Run Inference in Python
Python
from unsloth import FastVisionModel
import torch
from PIL import Image
from transformers import TextStreamer
Load the Base Model + Your Radiology Adapter
model, tokenizer = FastVisionModel.from_pretrained(
model_name = "Yash983487/Llama-3.2-11B-Vision-Radiology-LoRA",
load_in_4bit = True,
)
FastVisionModel.for_inference(model)
Example: Ask the model to describe an image
(Upload an image as 'image.png' first)
image = Image.open("image.png")
instruction = "You are an expert radiographer. Describe accurately what you see in this image."
messages = [
{"role": "user", "content": [{"type": "text", "text": instruction}, {"type": "image", "image": image}]}
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt = True,
tokenize = True,
return_tensors = "pt"
).to("cuda")
text_streamer = TextStreamer(tokenizer, skip_prompt = True)
_ = model.generate(input_ids = inputs, streamer = text_streamer, max_new_tokens = 128)
🧠 Training Details
Architecture: Llama-3.2-11B-Vision (4-bit quantization).
Method: LoRA fine-tuning on vision and language modules.
Hardware: Trained on a single NVIDIA T4 GPU (Google Colab).
Optimization: Solved Python 3.12 recursion limits and VRAM constraints for efficient training.
⚠️ Medical Disclaimer
This model is for educational and research purposes only. It is not a substitute for professional medical advice, diagnosis, or treatment. Always seek the advice of a qualified radiologist or physician regarding medical imaging.
Developed by Yash983487
