This is a fine-tuned vision-language model designed for Medical Visual Question Answering (VQA) and multimodal document retrieval. It is optimized to process both clinical text and medical imaging queries within a Retrieval-Augmented Generation (RAG) architecture.

This model acts as the core reasoning engine for a highly specialized medical assistant. Fine-tuned over 8 hours on institute DGX hardware, it merges the vision-language capabilities of Qwen3-VL with domain-specific biomedical knowledge from PubMed. It is built to operate within a FAISS-backed RAG pipeline, allowing it to ground its diagnostic reasoning and medical image analysis in retrieved clinical literature.

The model is designed to be used via an API endpoint (e.g., LMDeploy or vLLM) as part of a larger medical RAG application. It directly takes user queries combining text and uploaded medical images (such as scans or diagrams), processes context retrieved from a vector database, and generates clinical summaries or answers.

This model is a research and educational tool. It is strictly not intended for real-world clinical decision-making, patient diagnosis, or as a replacement for professional medical advice. The model may hallucinate or misinterpret complex medical imaging.

While fine-tuned on PubMed, the model's knowledge relies on the biases present in the retrieved literature. It may struggle with highly ambiguous imaging or rare medical conditions not well-represented in the fine-tuning dataset. Users should exercise extreme caution and verify all outputs against professional medical standards.

Users should implement robust guardrails and human-in-the-loop verification when deploying this model in any health-adjacent context.

This model is intended to be served using lmdeploy or vLLM due to its multimodal architecture and RAG integration.

To serve the merged adapter weights locally: