This repository contains a Parameter-Efficient Fine-Tuned (PEFT) version of Qwen2.5-3B-Instruct using the QLoRA (Quantized Low-Rank Adaptation) technique.
The objective of this project was to implement an end-to-end Large Language Model (LLM) fine-tuning pipeline using the Hugging Face ecosystem. The model was instruction-tuned on the Guanaco LLaMA2 1K dataset using TRL's SFTTrainer, 4-bit quantization, and LoRA adapters.
This project demonstrates practical experience with modern LLM fine-tuning techniques, efficient GPU utilization, and model publishing on Hugging Face Hub.
Model Details
Item
Value
Base Model
Qwen/Qwen2.5-3B-Instruct
Model Size
3 Billion Parameters
Fine-Tuning Method
PEFT
Technique
QLoRA
Training Method
Supervised Fine-Tuning (SFT)
Framework
Hugging Face Transformers
Trainer
TRL SFTTrainer
Quantization
4-bit NF4
Dataset
mlabonne/guanaco-llama2-1k
Hardware
NVIDIA Tesla T4 GPU
Project Objective
The primary objective of this project was to gain hands-on experience with parameter-efficient fine-tuning of Large Language Models while minimizing GPU memory consumption.
The project covers the complete workflow:
Loading a pretrained instruction-tuned LLM
Preparing conversational datasets
Applying chat templates
Configuring 4-bit quantization
Applying LoRA adapters
Fine-tuning using TRL
Saving adapter weights
Publishing the trained model on Hugging Face Hub
Dataset
The model was fine-tuned using the public mlabonne/guanaco-llama2-1k instruction dataset.
Dataset characteristics:
Approximately 1,000 instruction-response samples
Multi-domain conversational tasks
Instruction-following examples
Question Answering
Writing Assistance
General Knowledge
Reasoning
Coding
Chat-style responses
Training Pipeline
The end-to-end workflow consisted of the following steps:
Load the Qwen2.5-3B-Instruct base model
Load the tokenizer
Apply Qwen chat template formatting
Configure 4-bit NF4 quantization using BitsAndBytes
Prepare the model for QLoRA training
Configure LoRA adapters
Fine-tune using TRL SFTTrainer
Save LoRA adapter weights
Upload the adapter to Hugging Face Hub
Training Configuration
Parameter
Value
Epochs
1
Batch Size
1
Gradient Accumulation
2
Learning Rate
2e-4
Optimizer
paged_adamw_8bit
Precision
BF16
Quantization
4-bit NF4
Double Quantization
Enabled
Max Sequence Length
1024
LoRA Rank (r)
64
LoRA Alpha
16
LoRA Dropout
0.1
LoRA Target Modules
LoRA adapters were applied to the following transformer projection layers:
q_proj
k_proj
v_proj
o_proj
gate_proj
up_proj
down_proj
Trainable Parameters
Metric
Value
Total Parameters
3.21 Billion
Trainable Parameters
119.7 Million
Trainable Percentage
3.74%
Only LoRA adapter weights were updated during training while the original Qwen base model remained frozen.
Why PEFT?
Parameter-Efficient Fine-Tuning (PEFT) enables efficient adaptation of Large Language Models by training only a small subset of additional parameters instead of updating the entire model.
Benefits include:
Lower GPU memory usage
Faster training
Smaller model checkpoints
Efficient storage
Easy adapter sharing
Why QLoRA?
QLoRA combines Low-Rank Adaptation (LoRA) with 4-bit quantization.
Advantages include:
Fine-tuning billion-parameter models on consumer GPUs
This repository contains only the LoRA adapter weights and does not include the original Qwen2.5-3B-Instruct base model.
The model was trained on the public mlabonne/guanaco-llama2-1k dataset containing approximately 1,000 instruction-response pairs. It is intended as an educational and portfolio project demonstrating the complete QLoRA fine-tuning workflow.
Additional domain-specific fine-tuning, evaluation, and safety testing would be required before production deployment.
Future Improvements
Possible future enhancements include:
Fine-tuning on larger instruction datasets
Multi-epoch training
Hyperparameter optimization
Domain-specific adaptation
Model evaluation using standard LLM benchmarks
LoRA adapter merging
Retrieval-Augmented Generation (RAG) integration
Quantitative benchmarking against the base model
Acknowledgements
This project was built using the following open-source technologies:
Hugging Face Transformers
Hugging Face TRL
Hugging Face PEFT
BitsAndBytes
Qwen Team
mlabonne Guanaco Dataset
Author
Nilesh Kendre
AI / ML Engineer | Generative AI | LLMs | Agentic AI | RAG | NLP
This repository demonstrates hands-on implementation of modern Large Language Model fine-tuning using QLoRA, PEFT, TRL, and the Hugging Face ecosystem. The project showcases practical experience in parameter-efficient training, adapter-based fine-tuning, and model deployment workflows.
