This repository contains a Parameter-Efficient Fine-Tuned (PEFT) version of the Llama-2-7B Chat model using the QLoRA approach. The model was instruction-tuned using the Guanaco LLaMA2 1K dataset with the Hugging Face TRL SFTTrainer.
The objective of this project was to understand and implement an end-to-end Large Language Model (LLM) fine-tuning workflow using modern open-source tools, including Transformers, PEFT, BitsAndBytes, and TRL.
Model Details
Item
Value
Base Model
NousResearch/Llama-2-7b-chat-hf
Fine-Tuning Method
PEFT
Technique
QLoRA
Training Method
Supervised Fine-Tuning (SFT)
Framework
Hugging Face Transformers
Trainer
TRL SFTTrainer
Quantization
4-bit
Dataset
mlabonne/guanaco-llama2-1k
Dataset
This model was fine-tuned using the mlabonne/guanaco-llama2-1k instruction dataset available on Hugging Face.
The dataset contains approximately 1,000 instruction-response pairs covering a wide range of tasks, including:
Question Answering
General Knowledge
Reasoning
Coding
Writing Assistance
Conversational AI
Instruction Following
The dataset is designed for supervised instruction tuning of chat-based Large Language Models.
Training Pipeline
The complete training workflow consisted of the following stages:
Load the Llama-2-7B Chat base model.
Load the tokenizer.
Quantize the base model to 4-bit using BitsAndBytes.
Configure PEFT using LoRA adapters.
Prepare the Guanaco instruction dataset.
Perform Supervised Fine-Tuning (SFT) using TRL's SFTTrainer.
Save the trained LoRA adapter.
Push the trained adapter to the Hugging Face Hub.
Training Configuration
Parameter
Value
Epochs
1
Batch Size
1
Gradient Accumulation Steps
2
Learning Rate
2e-4
Optimizer
paged_adamw_32bit
Weight Decay
0.001
Max Gradient Norm
0.3
Warmup Ratio
0.03
Quantization
4-bit
LoRA Rank (r)
64
LoRA Dropout
0.1
Why PEFT?
Parameter-Efficient Fine-Tuning (PEFT) enables efficient adaptation of Large Language Models by training only a small set of additional parameters instead of updating the entire model. This significantly reduces GPU memory consumption, training time, and storage requirements while maintaining competitive performance.
Why QLoRA?
QLoRA combines Low-Rank Adaptation (LoRA) with 4-bit quantization using the BitsAndBytes library. By loading the frozen base model in 4-bit precision and training only the LoRA adapter weights, QLoRA enables fine-tuning of billion-parameter language models on limited GPU resources.
This repository contains only the LoRA adapter weights and not the original Llama-2-7B base model.
The model was fine-tuned on the public mlabonne/guanaco-llama2-1k instruction dataset containing approximately 1,000 instruction-response pairs. It is intended to demonstrate the end-to-end PEFT/QLoRA fine-tuning workflow and should be considered an educational portfolio project rather than a production-ready conversational assistant.
Additional fine-tuning, evaluation, and safety testing would be required before deploying it in production.
Future Work
Future enhancements include:
Fine-tuning on larger instruction datasets
Hyperparameter optimization
Multi-epoch training
Domain-specific instruction tuning
Retrieval-Augmented Generation (RAG) integration
Evaluation using standard LLM benchmarks
Merging LoRA adapters with the base model for deployment
Acknowledgements
Hugging Face Transformers
Hugging Face TRL
Hugging Face PEFT
BitsAndBytes
NousResearch Llama-2-7B Chat
mlabonne Guanaco Dataset
Author
Nilesh Kendre
AI Engineer | Generative AI | LLMs | NLP | RAG | Agentic AI
This repository showcases hands-on implementation of PEFT, QLoRA, and Supervised Fine-Tuning (SFT) using the Hugging Face ecosystem.
