You can deploy this model and start earning money today!

Yash983/
medgemma-2b-sft

Name: medgemma-2b-sft
Availability: InStock

safetensors

Views

No views yet

🏥 MedGemma-2B-SFT

Base Model: google/gemma-2b
Fine-tuning: QLoRA (Supervised Fine-Tuning)
Domain: Medical QA, Clinical reasoning

📌 Overview

MedGemma-2B-SFT is a domain-adapted medical language model built by fine-tuning Gemma-2B on a curated multi-source dataset:

PubMedQA (scientific QA)
MedMCQA (clinical MCQs)
MedAlpaca (instruction-style medical data)

The model is optimized for:

Medical question answering
Clinical reasoning
Structured explanations

📊 Evaluation Results

Aggregate Metrics

Metric	Base Gemma-2B	SFT	Δ
Perplexity	46.88	17.11	↓ 29.77
ROUGE-1	14.95	25.16	+10.21
ROUGE-2	3.81	9.30	+5.49
ROUGE-L	12.78	21.52	+8.74
BERTScore-F1	69.63	75.40	+5.77

Detailed Evaluation

Perplexity: 8.02
ROUGE-1: 0.2137
ROUGE-2: 0.0511
ROUGE-L: 0.1345
Win Rate: 98.67%

🧪 Observations

✅ Strengths

Strong improvement in factual grounding vs base model
Better structured medical explanations
Improved QA performance across datasets

⚠️ Limitations

Occasional repetition in long-form answers
Some hallucinations in rare medical conditions
Still not reliable for clinical decision-making

⚙️ Training Details

Method: QLoRA
Rank (r): 16
Target modules: q_proj, v_proj
Batch size: 1
Gradient accumulation: 8
Max sequence length: 512
Optimizer: paged_adamw_8bit

📦 Dataset

Combined dataset (~190K samples after filtering):

PubMedQA
MedMCQA
MedAlpaca

Processing:

Deduplication
Length filtering (>80 chars)
80/10/10 split

🚀 Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel
import torch

MODEL_ID = "google/gemma-2b"
ADAPTER  = "Yash983/medgemma-2b-sft"

# ── 4-bit quantization — without this, 2B params load as float32 (~16GB RAM) ──
bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
)

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
tokenizer.pad_token = tokenizer.eos_token

model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    quantization_config=bnb,
    device_map="auto",
    torch_dtype=torch.float16,
)
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()                                        # disable dropout

prompt = "Explain symptoms of diabetes"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)  # must match model device

with torch.no_grad():                               # no gradient tracking at inference
    outputs = model.generate(
        **inputs,
        max_new_tokens=200,
        do_sample=True,                             # required when using temperature
        temperature=0.7,
        top_p=0.9,                                  # pair with temperature for quality
        pad_token_id=tokenizer.eos_token_id,
        eos_token_id=tokenizer.eos_token_id,
    )

# strip prompt tokens — otherwise the prompt is printed again at the start
generated_ids = outputs[0][inputs["input_ids"].shape[1]:]
print(tokenizer.decode(generated_ids, skip_special_tokens=True))

📄 Citation

bibtex

@misc{medgemma2b-sft,
  title   = {MedGemma-2B-SFT: Supervised fine-tuned medical language model},
  author  = {Yash},
  year    = {2025},
  base    = {google/gemma-2b},
  url     = {https://huggingface.co/Yash983/medgemma-2b-sft}
}