A QLoRA (4-bit NF4 quantization + LoRA) fine-tune of mistralai/Mistral-7B-Instruct-v0.2 on the ChatDoctor-HealthCareMagic-100k dataset, adapting the base instruction-tuned model to respond in the style of a medical Q&A assistant.

⚠️ Disclaimer: This is a research/portfolio project. It is not a substitute for professional medical advice, diagnosis, or treatment. Do not use this model for real medical decision-making. Always consult a qualified healthcare provider for medical concerns.

Model Details

Model Description

Developed by: Diwash Adhikari (d1wash)
Model type: Causal language model, LoRA adapter (PEFT)
Language(s): English
License: Apache 2.0
Finetuned from model: mistralai/Mistral-7B-Instruct-v0.2

Model Sources

Base model: mistralai/Mistral-7B-Instruct-v0.2
Training dataset: ChatDoctor-HealthCareMagic-100k
GitHub: Diwash17

Uses

Direct Use

This adapter is intended to be loaded on top of the Mistral-7B-Instruct-v0.2 base model to generate conversational responses in a medical Q&A style, for research, demonstration, and educational purposes.

Out-of-Scope Use

Not for clinical or diagnostic use. This model has not been validated for real-world medical decision-making and should never be used to diagnose, treat, or advise on actual patient care.
Not intended for use in any high-stakes setting without human expert review.

Bias, Risks, and Limitations

The model was fine-tuned on a public, community-sourced medical Q&A dataset (ChatDoctor-HealthCareMagic), which may contain inaccuracies, inconsistent medical advice, or outdated information.
Like all LLMs, the model can hallucinate plausible-sounding but incorrect medical claims.
Training was done on a subset of the full dataset (15,000 of ~110,000 examples) under a fixed compute/time budget, so coverage of rarer conditions or edge cases is limited.
No clinical validation, bias auditing, or safety red-teaming has been performed on this model.

Recommendations

Users should treat all outputs as illustrative only, verify any medical claims with a qualified professional, and not rely on this model for real health decisions.

How to Get Started with the Model

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

base_model_name = "mistralai/Mistral-7B-Instruct-v0.2"
adapter_repo = "d1wash/mistral-7b-healthcare-qlora"

tokenizer = AutoTokenizer.from_pretrained(base_model_name)
tokenizer.pad_token = tokenizer.eos_token

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)

base_model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    quantization_config=bnb_config,
    device_map="auto",
)

model = PeftModel.from_pretrained(base_model, adapter_repo)

prompt = """<s>[INST] You are a helpful medical assistant.
I have been experiencing persistent headaches for the past week. What could be the cause?

Patient: I wake up with a throbbing headache every morning. [/INST]

Doctor:"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(
    **inputs,
    max_new_tokens=200,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
    repetition_penalty=1.3,
    no_repeat_ngram_size=3,
    pad_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Training Details

Training Data

A filtered, subsampled subset of ChatDoctor-HealthCareMagic-100k:

Original dataset size: ~112,000 examples
After quality filtering (removing very short/long examples): ~109,800 examples
Subsampled to 15,000 training examples and 1,500 evaluation examples to fit within a fixed compute/time budget on a single T4 GPU

Training Procedure

Preprocessing

Prompts formatted in Mistral's instruction format: <s>[INST] ... [/INST] ...
Tokenized with a max sequence length of 256 tokens (truncated/padded)

Training Hyperparameters

Quantization: 4-bit NF4 (bitsandbytes), double quantization enabled
Compute dtype: float16
LoRA rank (r): 8
LoRA alpha: 16
LoRA dropout: 0.05
Target modules: q_proj, v_proj
Optimizer: paged_adamw_8bit
Learning rate: 2e-4 (cosine schedule)
Effective batch size: 16 (per-device batch size 8 × gradient accumulation 2)
Total steps: 900 (full run completed — checkpoints saved through step 900)
Training regime: fp16 mixed precision, gradient checkpointing enabled

Speeds, Sizes, Times

Hardware: 1x NVIDIA T4 (16GB), Lightning AI Studio (free tier)
Training time: ~3 hours for the full 900-step run
Adapter size: ~13.6 MB (LoRA adapter only, does not include base model weights)

Evaluation

Metrics

Training and validation loss decreased consistently across checkpoints:

Step	Training Loss	Validation Loss
150	2.004	2.011
300	1.900	1.907
450	1.880	1.885
600	1.903	1.870

Validation loss tracked closely with training loss throughout, indicating the model generalized to the held-out set rather than memorizing the training data. Training completed the full 900-step run with the best checkpoint (by evaluation loss) restored at the end.

d1wash/
mistral-7b-healthcare-qlora

Mistral-7B-Instruct QLoRA — Medical Q&A Assistant