You can deploy this model and start earning money today!

Pharmamapllm/
MedLLaMA3-DDI-QLoRA-V1

Name: MedLLaMA3-DDI-QLoRA-V1
Availability: InStock

safetensors

Views

No views yet

🧬 MedLLaMA3-DDI-QLoRA-V1

PharmaMap-LLM: A QLoRA Fine-Tuned MedLLaMA-3 Model for Drug-Drug Interaction (DDI) Prediction

This model is part of the MSc research project PharmaMap-LLM, developed to enable memory-efficient biomedical relation extraction using instruction-tuned Large Language Models (LLMs). It fine-tunes johnsnowlabs/JSL-MedLlama-3-8B-v2.0 using QLoRA on DrugBank-based datasets for DDI classification and reasoning.

📌 Model Summary

Base Model: johnsnowlabs/JSL-MedLlama-3-8B-v2.0
Tuning Method: QLoRA (4-bit quantization + LoRA adapters)
Task: Drug-drug interaction classification + explanation
Frameworks: Hugging Face Transformers, PEFT, Datasets
Prompt Format: Instruction-style (Vicuna-like) with reasoning support
Instruction Accuracy: 91%
Memory Reduction: ~60% compared to full-parameter fine-tuning

🧠 Example Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load base + LoRA adapter
base_model = AutoModelForCausalLM.from_pretrained("johnsnowlabs/JSL-MedLlama-3-8B-v2.0", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("Pharmamapllm/MedLLaMA3-DDI-QLoRA-V1")
model = PeftModel.from_pretrained(base_model, "Pharmamapllm/MedLLaMA3-DDI-QLoRA-V1")

prompt = """Drug1: Ranolazine
SMILES for drug1: COC1=CC=CC=C1OCC(O)CN1CCN(CC(=O)NC2=C(C)C=CC=C2C)CC1
Organism targeted by drug1: Unknown
Genes targeted by drug1: P00747, P35498, Q14500, Q13936, P35348, P08588
Drug2: Capreomycin
SMILES for drug2: ...
Organism targeted by drug2: Unknown
Genes targeted by drug2: P9WJ63

CLASSIFICATION:"""

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=150)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

📊 Performance

Metric	Value
F1-Score (DDI 2013)	0.87
Precision	0.89
Recall	0.85
Instruction-following	91%
Latent clustering (UMAP)	✅ Well-separated classes
Arcee LLM-as-Judge score	High agreement

🧬 Data & Labeling

Positive Samples: DrugBank annotated interactions
Negative Samples: Random drug pairs with no known interactions
Enhancement: Some positive samples injected with reasoning generated by Arcee.ai
Schema:
- Drug names
- SMILES structures
- Organism targets
- Gene targets
Prompt-based format: "CLASSIFICATION:" triggers generation

🧪 Training Configuration

Setting	Value
Method	QLoRA + PEFT (LoRA)
LoRA Rank	8
LoRA Dropout	0.05
Learning Rate	1e-5
Epochs	3
Batch Size	15 (with grad acc: 2)
Quantization	4-bit (nf4, double quant)
Device	Colab A100 (16-bit bfloat)

📁 Files Included

adapter_model.safetensors — LoRA adapter weights
adapter_config.json — PEFT adapter configuration
tokenizer.model, tokenizer_config.json — Tokenizer files
special_tokens_map.json — Special token mapping
README.md — This file!

📄 Citation

bibtex

@misc{raveendran2025pharmamap,
  title={PharmaMap-LLM: Fine-tuning Large Language Models for Drug-Drug Interaction (DDI) Analysis},
  author={Raveendran, Kithusshand},
  year={2025},
  institution={University of Westminster, Informatics Institute of Technology},
  howpublished={\url{https://huggingface.co/Pharmamapllm/MedLLaMA3-DDI-QLoRA-V1}}
}

🙋‍♂️ Author

Kithusshand Raveendran
MSc in Advanced Software Engineering
University of Westminster / Informatics Institute of Technology
🔗 LinkedIn Profile

📢 Disclaimer

This model is for research and educational purposes only. It is not approved for clinical use. Always consult licensed medical professionals before using automated predictions for any real-world treatment.

🧠 Acknowledgements

johnsnowlabs/JSL-MedLlama-3-8B-v2.0
DrugBank & DDIExtraction 2013
Hugging Face Transformers & PEFT
Arcee.ai for reasoning generation
University of Westminster & IIT Sri Lanka