A fine-tuned Mistral-7B model specialized for Indian healthcare assistance. SwasthAI provides accurate, culturally-appropriate medical information for Indian patients.

Model Details

Property	Value
Base Model	unsloth/mistral-7b-v0.3-bnb-4bit
Fine-tuning Method	QLoRA (4-bit quantization + LoRA adapters)
Training Framework	Unsloth + TRL
Trainable Parameters	~42M (LoRA adapters)
Context Length	2048 tokens
License	Apache 2.0

Training Data

Fine-tuned on 1,223 curated medical Q&A pairs covering:

General medicine and primary care
Chronic disease management (diabetes, hypertension, thyroid)
Maternal and child health
Mental health and wellness
Nutrition and lifestyle guidance
Ayurveda and traditional Indian medicine
Healthcare navigation in India
Emergency care guidance

Quick Start

With Unsloth (Recommended - Fastest)

python

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="sri2000us/swasthai-medical-7b",
    max_seq_length=2048,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

def ask_swasthai(question):
    prompt = f"""### Instruction:
You are SwasthAI, an expert Indian healthcare assistant. Answer the medical question accurately.

### Input:
{question}

### Response:
"""
    inputs = tokenizer([prompt], return_tensors="pt").to("cuda")
    outputs = model.generate(**inputs, max_new_tokens=256, use_cache=True)
    response = tokenizer.batch_decode(outputs)[0]
    return response.split("### Response:")[-1].strip()

# Example
print(ask_swasthai("What are the symptoms of diabetes?"))

With Transformers + PEFT

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mistral-7B-v0.3",
    load_in_4bit=True,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-v0.3")

# Load LoRA adapters
model = PeftModel.from_pretrained(base_model, "sri2000us/swasthai-medical-7b")

Prompt Format

This model uses the Alpaca prompt format:

### Instruction:
You are SwasthAI, an expert Indian healthcare assistant. Answer the medical question accurately.

### Input:
{user_question}

### Response:
{model_response}

Example Outputs

Q: What are the symptoms of Type 2 diabetes?

Common symptoms include increased thirst and frequent urination, unexplained weight loss, fatigue, blurred vision, slow-healing wounds, and numbness in extremities. If you experience these, consult a doctor for HbA1c and fasting blood sugar tests.

Q: How can I manage high blood pressure naturally?

Reduce sodium intake, follow DASH diet, exercise 30 minutes daily, maintain healthy weight, limit alcohol, practice yoga/meditation, and include potassium-rich foods. Always continue prescribed medications.

Limitations

Not a replacement for professional medical advice - Always consult qualified healthcare providers
Information currency - Medical knowledge evolves; verify critical information
Emergency situations - Call 102/108 in India for medical emergencies
Individual variation - Responses are general guidance

Training Configuration

python

# LoRA Configuration
r = 16
lora_alpha = 16
lora_dropout = 0
target_modules = ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]

# Training Arguments
learning_rate = 2e-4
batch_size = 2
gradient_accumulation_steps = 4
num_epochs = 1
optimizer = "adamw_8bit"

Environmental Impact

Hardware: Google Colab T4 GPU (16GB VRAM)
Training Time: ~25 minutes
Cloud Provider: Google Colab

Citation

bibtex

@misc{swasthai-medical-7b,
  author = {SwasthAI Team},
  title = {SwasthAI Medical 7B: Fine-tuned Mistral for Indian Healthcare},
  year = {2025},
  publisher = {HuggingFace},
  url = {https://huggingface.co/sri2000us/swasthai-medical-7b}
}

Acknowledgments

Unsloth for efficient fine-tuning
Mistral AI for the base model
Hugging Face for model hosting

Framework versions

PEFT 0.18.0
Unsloth 2024.12
TRL 0.12.0