🏥 LLaMA 2 7B Medical QLoRA Adapter
This repository contains a QLoRA fine-tuned adapter for medical domain instruction-following and QA tasks.
Model Info
- Base Model: meta-llama/Llama-2-7b-hf
- Fine-tuning Method: QLoRA (PEFT)
- Domain: Medical QA / Instructions
Training Setup
- Total Epochs: 3
- Batch Size: 16
- Gradient Accumulation: 4
- Scheduler: Cosine
- Optimizer: paged_adamw_32bit
- Precision: bf16
Learning Rate Strategy
Training was performed in two phases:
- Epoch 1 → Learning Rate:
2e-4
- Epoch 2,3 → Learning Rate:
1e-4
This staged LR reduction helps stabilize training and improve convergence.
LoRA Configuration
Files Included
adapter_model.safetensors → LoRA weightsadapter_config.json → LoRA configtokenizer.json → Tokenizertokenizer_config.json → Tokenizer configchat_template.jinja → Prompt format
Important Notes
- This repository contains only LoRA adapter weights, not the full model
- You must download the base model separately
- Requires access to LLaMA 2 weights from Hugging Face
Intended Use
- Medical question answering
- Educational and research purposes
- NOT intended for clinical or real-world medical decision-making
License
- Base model license: Meta LLaMA 2 License
- Adapter: Open for research use
Acknowledgements
- Hugging Face Transformers
- PEFT (Parameter-Efficient Fine-Tuning)
- TRL (Transformer Reinforcement Learning)
