This model is specialized for biomedical question answering. Given a research context and a medical question, it answers yes, no, or maybe with a detailed explanation.

Training Details

Parameter	Value
Base model	Llama 3.2 3B Instruct
Dataset	PubMedQA (pqa_labeled)
Training samples	1,000
Technique	QLoRA (4-bit quantization + LoRA)
LoRA rank	16
Epochs	3
Learning rate	2e-4
Final training loss	1.56
Training time	~7 minutes (A100)
Trainable parameters	~13M / 3B (0.42%)

How to Use

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "Yashwanth-Pulimi/llama3-medical-qa"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16)

prompt = """<|begin_of_text|><|start_header_id|>system<|end_header_id|>
You are an expert medical AI assistant.<|eot_id|>
<|start_header_id|>user<|end_header_id|>
Context: A randomized trial of 200 diabetic patients showed metformin 
reduced HbA1c by 2.1% vs 1.2% with lifestyle changes alone.

Question: Does metformin reduce HbA1c more than lifestyle changes alone?<|eot_id|>
<|start_header_id|>assistant<|end_header_id|>"""

inputs  = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=150, temperature=0.1, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Dataset

PubMedQA is a biomedical question answering dataset collected from PubMed abstracts. Questions are answered with yes, no, or maybe based on research context.

Dataset: https://huggingface.co/datasets/qiaojin/PubMedQA
Paper: https://arxiv.org/abs/1909.06146

Intended Use

Biomedical research assistance
Medical Q&A applications
Healthcare decision support prototypes

Limitations

Trained on 1,000 samples — not for clinical use
Struggles with ambiguous maybe cases due to class imbalance
Not a substitute for medical professionals

Portfolio Project

Built as part of an ML Engineer portfolio demonstrating:

LLM fine-tuning with QLoRA
Parameter efficient training (0.42% of weights)
HuggingFace ecosystem (datasets, hub, transformers)
Experiment tracking with W&B

GitHub: https://github.com/YashPulimi/llama3-medical-qa