This model card describes the MediTalk300-Mistral-7B model, which was trained on MediTalk300 dataset with corpus of 1725 doctor-patient role played dialogue pairs for 300 most commonly occurring diseases in India. Each conversation averages 300 tokens as mentioned in Table I below..

The MediTalk300-Mistral-7B model was trained on a large multicenter dataset with a large corpus of 1.2B words of diverse diseases we constructed.

To fine-tune MediTalk300 efficiently on limited hardware, we combined three key techniques.

We used PEFT to update only a small portion of the model’s parameters, keeping the rest frozen. This reduced training time and resource usage. Then, with QLoRA, we applied low-rank adapters along with 4-bit/8-bit quantization, allowing us to train the model on consumer GPUs without sacrificing accuracy. Finally, we applied Instruction Tuning by structuring our dataset as clear instruction-response pairs, helping the model understand user intent and generate medically accurate replies.

These methods allowed us to optimize both performance and efficiency, making MediTalk300 suitable for real-world use and local deployment.

We used a batch size of 32, a maximum sequence length of 256, and a learning rate of 5e-5 for pre-training our models.

Load the model via the transformers library:

Please cite these articles: Uploading Soon....

base_model: unsloth/mistral-7b-instruct-v0.3-bnb-4bit tags: