A Mistral 7B model fine-tuned on 5,000 medical question-answer pairs using QLoRA (Quantized Low-Rank Adaptation). The model is trained to respond to patient descriptions with clinically informed responses in the style of a consulting physician.

This model is for research purposes only. It does not provide medical advice and should never be used as a substitute for professional medical consultation.

Training details

Parameter	Value
Base model	Mistral 7B v0.1
Dataset	ChatDoctor-HealthCareMagic-100k
Training examples	5,000
Technique	QLoRA (4-bit quantization + LoRA)
LoRA rank	16
LoRA alpha	32
Trainable parameters	13.6M (0.19% of total)
Training steps	625
Final training loss	2.03
Hardware	Google Colab T4 GPU (free tier)
Training time	~96 minutes

Why QLoRA

Fine-tuning Mistral 7B normally requires ~28GB of GPU memory. QLoRA reduces this to ~5GB by quantizing the base model to 4-bit precision and only training small adapter matrices (LoRA). Only 0.19% of the model's parameters were updated during training — 13.6 million out of 7.2 billion.

This makes large model fine-tuning accessible on free hardware.

Example usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-v0.1",
    torch_dtype=torch.float16,
    device_map="auto"
)

model = PeftModel.from_pretrained(base_model, "Umer797/medqa-mistral-7b-lora")
tokenizer = AutoTokenizer.from_pretrained("Umer797/medqa-mistral-7b-lora")

prompt = """<s>[INST] If you are a doctor, please answer the medical questions 
based on the patient's description.

Patient: I have been experiencing severe headaches on one side of my head 
with sensitivity to light and nausea. [/INST]

Doctor:"""

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Ethical notice

This model is a research demonstration only
It does not provide medical advice
Outputs must not be used for clinical decision making
Always consult a qualified healthcare professional

Built by

Umer Mahmood Khan — AI Research Engineer at NCAI Pakistan

Part of a medical AI portfolio:

RadiOCR: https://cfxf4o325p6lwux8me8ugz.streamlit.app
MedRAG: https://medrag-hck3ynxroca5zi7fkj9vapp.streamlit.app
ClinicalMind: https://clinicalmind-i8ut2tunwca5vz5rpq23re.streamlit.app