A LoRA adapter for Qwen2-VL-2B-Instruct that grades diabetic retinopathy (0-4) and produces a structured clinical explanation (grade, lesions, reasoning, recommendation). Fine-tuned with 4-bit QLoRA (Unsloth) on the IDRiD-VQA dataset, entirely on a single 6 GB consumer GPU (NVIDIA RTX 4050).

⚠️ Research use only. Not a medical device. Not for clinical use.

Results (103-image IDRiD test split)

Model	Accuracy	QWK	ROUGE-L	BERTScore	s/img
Qwen2-VL-2B (zero-shot)	31.1%	0.00	0.081	0.000	10.6
+ QLoRA (this model)	66.0%	0.76	0.253	0.309	5.6

Fine-tuning lifts quadratic weighted kappa (the standard DR metric) from no agreement to substantial agreement (0.76), ~3x better explanations, and ~2x faster. Input resolution (768 px) was the biggest single lever.

Usage

python

from unsloth import FastVisionModel
from PIL import Image

model, tok = FastVisionModel.from_pretrained("vermayuvraj/idrid-qwen2vl-2b-qlora", load_in_4bit=True)
FastVisionModel.for_inference(model)

img = Image.open("fundus.jpg").convert("RGB")
q = "What is the diabetic retinopathy severity grade (0-4) shown in this fundus image?"
msgs = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": q}]}]
text = tok.apply_chat_template(msgs, add_generation_prompt=True)
inputs = tok(img, text, add_special_tokens=False, return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Training

QLoRA r=16, alpha=16 on all attention + MLP projections (language + vision towers), batch size 1 x grad-accum 8, max_seq_length=1280, images at 768 px, bf16, 4 epochs. Full run: ~90 min, peak 3.52 GB VRAM on a 6 GB RTX 4050.

Limitations

Trained/evaluated on IDRiD only (single-centre, 516 images). Proliferative DR (grade 4) is the hardest class because lesion flags derive from grade-conditioned priors. Grade and reasoning come from separate generations and can occasionally disagree. See the paper for details.

Citation

bibtex

@misc{verma2026idridvqa,
  title  = {IDRiD-VQA: Fine-Tuning a 2B Vision-Language Model for Explainable
            Diabetic Retinopathy Grading on Consumer Hardware},
  author = {Verma, Yuvraj},
  year   = {2026}
}

Built on IDRiD (Porwal et al., 2018, CC BY 4.0).