You can deploy this model and start earning money today!

Prajwol699/
fined-tuned-medical-chatbot

Name: fined-tuned-medical-chatbot
Availability: InStock

safetensors

Views

No views yet

🏥 Medical LLM Fine-tuning Pipeline

A complete 3-stage Large Language Model fine-tuning pipeline specialized for the medical domain, trained entirely on free Google Colab (T4 GPU) using QLoRA and DPO alignment.

📌 Project Overview

This project demonstrates how to fine-tune a large language model from scratch using a professional 3-stage training pipeline on the medical domain — all within the constraints of free cloud compute. It covers domain adaptation, instruction fine-tuning, and RLHF-style preference alignment using Direct Preference Optimization (DPO).

The final model is a medical conversational assistant capable of answering health-related questions using evidence-based responses.

🗂️ Repository Structure

fine tune model/
└── finetuned medical chatbot/
    ├── adapter_model.safetensors   # Trained LoRA adapter weights
    ├── adapter_config.json         # LoRA configuration (r, alpha, target modules)
    ├── tokenizer.json              # Tokenizer vocabulary and rules
    └── tokenizer_config.json       # Tokenizer settings and special tokens

Note: This repository contains only the LoRA adapter weights (~50MB), not the full model weights (~2GB). You need to load the base TinyLlama model separately and apply the adapter on top. See How to Use below.

🧠 3-Stage Training Pipeline

Stage 1 — Domain Adaptation

Teaches the model medical vocabulary, terminology, drug names, disease descriptions, and clinical language before any instruction formatting.

Property	Value
Dataset	`medalpaca/medical_meadow_wikidoc`
Samples	5,000 medical Wikipedia articles
Format	Raw text (no Q&A structure)
Loss	Next-token prediction
Learning rate	3e-4

Stage 2 — Instruction Fine-tuning

Teaches the model to follow medical instructions and respond to patient questions in a helpful, structured format.

Property	Value
Dataset	`lavita/ChatDoctor-HealthCareMagic-100k`
Samples	3,000 real doctor-patient Q&A pairs
Format	ChatML (system / user / assistant)
Loss	Cross-entropy on response tokens only
Learning rate	2e-4

Stage 3 — DPO Alignment (RLHF Alternative)

Teaches the model to prefer safe, accurate, evidence-based responses over hallucinated or dangerous ones using human preference pairs.

Property	Value
Dataset	`argilla/ultrafeedback-binarized-preferences-cleaned`
Samples	800 filtered medical preference pairs
Format	Prompt + chosen response vs rejected response
Loss	DPO objective (β = 0.1)
Learning rate	5e-5

⚙️ Technical Stack

Component	Details
Base model	`TinyLlama/TinyLlama-1.1B-Chat-v1.0`
Quantization	QLoRA — 4-bit NF4 via `bitsandbytes`
Adapter method	LoRA (r=16, alpha=32)
Target modules	q_proj, k_proj, v_proj, o_proj
Precision	bf16 (bfloat16)
Framework	HuggingFace `transformers` + `trl` 0.29.1
Hardware	Google Colab Free Tier — NVIDIA T4 (16GB VRAM)
Total training time	~75 minutes

🚀 How to Use

Step 1 — Install dependencies

pip install transformers peft bitsandbytes accelerate torch

Step 2 — Load the model

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

# Load base model with 4-bit quantization
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

base_model = AutoModelForCausalLM.from_pretrained(
    "TinyLlama/TinyLlama-1.1B-Chat-v1.0",
    quantization_config=bnb_config,
    device_map="auto",
)

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("finetuned medical chatbot/")
tokenizer.pad_token = tokenizer.eos_token

# Apply LoRA adapter
model = PeftModel.from_pretrained(base_model, "finetuned medical chatbot/")
model.eval()
print("Model ready!")

Step 3 — Run inference

python

def medical_chat(question):
    prompt = (
        f"<|system|>\nYou are a helpful and accurate medical assistant. "
        f"Provide evidence-based information and always recommend consulting "
        f"a qualified doctor for personal medical advice.</s>\n"
        f"<|user|>\n{question}</s>\n"
        f"<|assistant|>\n"
    )
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=250,
            temperature=0.3,
            do_sample=True,
            top_p=0.9,
            repetition_penalty=1.2,
        )
    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
    return response.split("<|assistant|>")[-1].strip()

# Try it
print(medical_chat("What are the symptoms of diabetes?"))
print(medical_chat("How is hypertension treated?"))

📊 Training Configuration Summary

python

# Stage 1 & 2 — SFTConfig
SFTConfig(
    num_train_epochs=1,
    per_device_train_batch_size=8,
    gradient_accumulation_steps=2,
    gradient_checkpointing=True,
    bf16=True,
    max_length=256,
    packing=False,
)

# Stage 3 — DPOConfig
DPOConfig(
    num_train_epochs=1,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,
    beta=0.1,
    max_length=256,
    disable_dropout=True,
    bf16=True,
)

📈 What I Learned

How domain adaptation differs from instruction fine-tuning — raw text vs structured Q&A
Why QLoRA + bf16 is required on T4 GPUs to avoid gradient scaler conflicts
How DPO works as a simpler, more compute-efficient alternative to full PPO-based RLHF
Real quality tradeoffs between model size, dataset size, and training time
TRL 0.29.1 breaking API changes — SFTConfig, processing_class, disable_dropout
How dataset quality directly affects output quality (ChatDoctor persona bleeding)

⚠️ Limitations

TinyLlama 1.1B is too small to reliably hold complex medical facts
Reduced dataset sizes (for Colab time limits) affect output quality
The model may produce incorrect or hallucinated medical information
Not suitable for real clinical or medical use — experimental only
Outputs may reflect biases present in the training datasets

🔮 Future Improvements

Upgrade base model to Mistral-7B or LLaMA-3 with Colab Pro
Train on larger datasets — 20k+ samples per stage
Add evaluation metrics (ROUGE, BERTScore, MedQA accuracy)
Push LoRA adapters to HuggingFace Hub for easy sharing
Add a Gradio web UI for interactive demo
Filter ChatDoctor persona from training data for cleaner outputs

📦 Datasets Used

Stage	Dataset	Link
Domain Adaptation	medalpaca/medical_meadow_wikidoc	HuggingFace
Instruction Tuning	lavita/ChatDoctor-HealthCareMagic-100k	HuggingFace
DPO Alignment	argilla/ultrafeedback-binarized-preferences-cleaned	HuggingFace

All datasets are publicly available on HuggingFace — no login required.

📄 License

This project is for educational and research purposes only. The trained model should not be used for real medical diagnosis or treatment decisions. Always consult a qualified healthcare professional for medical advice.

🙏 Acknowledgements

HuggingFace — transformers, peft, trl, datasets
TinyLlama — base model
MedAlpaca — medical domain dataset
Google Colab — free GPU compute

Built with ❤️ on free Google Colab T4

Prajwol699/fined-tuned-medical-chatbot

🏥 Medical LLM Fine-tuning Pipeline

📌 Project Overview

🗂️ Repository Structure

🧠 3-Stage Training Pipeline

Stage 1 — Domain Adaptation

Stage 2 — Instruction Fine-tuning

Stage 3 — DPO Alignment (RLHF Alternative)

⚙️ Technical Stack

🚀 How to Use

Step 1 — Install dependencies

Step 2 — Load the model

Step 3 — Run inference

📊 Training Configuration Summary

📈 What I Learned

⚠️ Limitations

🔮 Future Improvements

📦 Datasets Used

📄 License

🙏 Acknowledgements

Prajwol699/
fined-tuned-medical-chatbot