You can deploy this model and start earning money today!

maherghanem86/
PharmaCompass

Name: PharmaCompass
Availability: InStock

safetensors

Views

No views yet

Llama2-7B-MIMIC-iii-Extraction-v1

Model Description

This model is a fine-tuned version of Llama-2-7b-chat-hf designed for Structured Clinical Information Extraction. It has been specifically trained to process unstructured clinical notes (discharge summaries) from the MIMIC-III database and transform them into a structured JSON format.

The model can identify and extract key medical entities such as:

Drug names
Dosages
Frequency of administration
Indications/Reasons for treatment

Training Procedure

The model was fine-tuned using QLoRA (4-bit quantization) to ensure efficiency and high performance.

Training Hyperparameters:

Base Model: NousResearch/Llama-2-7b-chat-hf
Method: LoRA (Low-Rank Adaptation)
Max Sequence Length: 2048 tokens
Learning Rate: 2e-4
Batch Size: 1 (with 4 gradient accumulation steps)
Optimizer: paged_adamw_32bit
Precision: 4-bit (bitsandbytes)

LoRA Configuration:

r (Rank): 16
lora_alpha: 32
Target Modules: q_proj, v_proj, k_proj, o_proj (Attention layers)
lora_dropout: 0.05

Intended Use

This model is intended for researchers and developers working on clinical natural language processing (NLP). It is designed to assist in converting medical narratives into machine-readable data.

How to use:

To use this model, you need to load it as a PEFT (Adapter) on top of the base Llama-2-7b-chat-hf model.

python

from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model_name = "NousResearch/Llama-2-7b-chat-hf"
adapter_model_name = "maherghanem86/PharmaCompass"

model = AutoModelForCausalLM.from_pretrained(base_model_name)
model = PeftModel.from_pretrained(model, adapter_model_name)
tokenizer = AutoTokenizer.from_pretrained(base_model_name)