A 2B-parameter medical vision-language model, trained for medical image understanding, radiology report assistance, clinical visual question answering, and medical text reasoning.
This release introduces Dynamic LoRA Scaling, a lightweight calibration technique that reduces adapter dominance while preserving the medical knowledge learned during fine-tuning. The objective is to improve reliability, reduce hallucinations, and mitigate common diagnostic confusion patterns observed in earlier releases of madrisight/MadriMed-VL-2B.
This model was trained and calibrated using bfloat16 (BF16) precision for best performance and reproducibility on Pytorch MPS
)
1import torch
2import re
3from transformers import AutoProcessor, Qwen3VLForConditionalGeneration
4from PIL import Image
5
6BASE_MODEL_ID = "madrisight/MadriMed-VL-2B-enc"
7
8model = Qwen3VLForConditionalGeneration.from_pretrained(
9 BASE_MODEL_ID,
10 device_map="cuda",
11 trust_remote_code=True,
12)
13
14model.eval()
15
16processor = AutoProcessor.from_pretrained(BASE_MODEL_ID, trust_remote_code=True)
17
18def load_direct_image(path: str) -> Image.Image:
19 with Image.open(path) as raw:
20 img = raw.convert("RGB")
21 return img
22
23
24# 5. Formulate the Query
25prompt = """Choose the correct option for the question
26
27Question:
28Examine the mammogram image shown above. Which of the following findings is most evident?
29
30Options
31A. Well-circumscribed round mass with benign features
32B. Clustered microcalcifications within an area of irregular density
33C. Fat-containing lesion consistent with lipoma
34D. Diffuse bilateral breast edema
35"""
36
37img = load_direct_image("/content/MM-1-a.png")
38
39
40messages = [
41 {
42 "role": "system",
43 "content": "You are an expert medical AI. You must deeply analyze the question and provide the final answer."
44 },
45 {
46 "role": "user",
47 "content": [
48 {"type": "image"},
49 {"type": "text", "text": prompt}
50 ]
51 }
52]
53stop_token_id = processor.tokenizer.convert_tokens_to_ids("<|im_end|>")
54
55with torch.inference_mode():
56
57 text = processor.apply_chat_template(
58
59 messages,
60 tokenize=False,
61 add_generation_prompt=True
62 )
63
64 inputs = processor(
65 text=text,
66 images=[img],
67 return_tensors="pt",
68 ).to("cuda")
69
70 generated_ids = model.generate(
71 **inputs,
72 max_new_tokens=1024, # tight control (prevents drift)
73 do_sample=False, # deterministic output
74 pad_token_id=processor.tokenizer.pad_token_id,
75 eos_token_id=stop_token_id
76 )
77
78 output_text = processor.batch_decode(
79 generated_ids[:, inputs.input_ids.shape[1]:],
80 skip_special_tokens=True
81 )[0]
82
83
84print(output_text.strip())
85
1So, let's analyze the mammogram. The image shows a breast with some irregularities.
2Looking at the options: A is about a well-circumscribed round mass, but the image doesn't show a clear mass.
3B mentions clustered microcalcifications in an irregular density area. In mammograms, microcalcifications are often seen as small white spots, and irregular density might be a pattern.
4C is a fat-containing lesion like a lipoma, but the image doesn't show fat density.
5D is bilateral breast edema, which isn't visible here.
6So B seems to fit because microcalcifications are a key finding in mammograms, especially when clustered.
7</think>
8
9B. Clustered microcalcifications within an area of irregular density
10
1@software{madrimedvl2b,
2 title = {MadriMed-VL-2B: A Compact Multimodal Medical Vision-Language Model},
3 author = {Madrisight},
4 year = {2026},
5 url = {https://huggingface.co/madrisight/MadriMed-VL-2B}
6}
