This Llama 3.1 8B Instruct Model was obtained using the Unsloth library. It is fine-tuned using the LoRA PEFT approach to give better responses to medical questions.
Model Details
Model Description
MedLam explores the fine-tuning of LLMs using a Medical QA dataset to improve their performance for domain-specific tasks. By combining state-of-the-art natural language processing techniques and medical data, MedLam aims to deliver an effective and intuitive medical assistant.
Developed by: Avishek Choudhury
Model type: Transformer based auto-regressive language model
Language(s) (NLP): English
License: MIT
Finetuned from model Meta Llama 3.1 8B Instruct: unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit
This model was developed only for research and learning purposes and is not meant for commercialization as a final product.
Assisting medical students in understanding complex topics.
Supporting professors in teaching and curriculum design.
Helping doctors with quick and accurate responses to medical questions.
How to Get Started with the Model
Use the code below to get started with the model.
from peft import PeftModel
from transformers import AutoModelForCausalLM
base_model = AutoModelForCausalLM.from_pretrained("unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit")
model = PeftModel.from_pretrained(base_model, "aviici4cs/MedLam")
Training Procedure
Data Preparation:
Preprocessed Medical QA datasets for training and evaluation.
Model Fine-Tuning:
Applied LoRA PEFT (Parameter-Efficient Fine-Tuning) to train LLMs on domain-specific data.
Performance Validation:
Analyzed and compared models across multiple configurations to determine optimal hyperparameters.
Deployment (Future Scope):
Aimed at integrating the fine-tuned model into a user-friendly medical assistant platform.
Hardware
Google Colab
Software
Programming Languages: Python
Frameworks: Unsloth Library, PyTorch, Hugging Face Transformers
Machine Learning Techniques: LoRA PEFT, Deep Learning
