Llama 3.2 3B Instruct — SAMSum Dialogue Summarization (QLoRA)
This model is a QLoRA fine-tuned adapter for meta-llama/Llama-3.2-3B-Instruct, trained for dialogue summarization using the SAMSum dataset .
The model was fine-tuned using 4-bit NF4 quantization and LoRA adapters , allowing efficient fine-tuning of a 3B-parameter language model on a single NVIDIA Tesla T4 GPU.
Model Details
Base model: meta-llama/Llama-3.2-3B-InstructTask: Dialogue SummarizationFine-tuning method: QLoRA / LoRAQuantization: 4-bit NF4Dataset: SAMSumHardware: NVIDIA Tesla T4 (16 GB VRAM)Training samples: 14,731Validation samples: 818Training epochs: 1Trainable parameters: 9,175,040Total parameters: 3,221,924,864Trainable percentage: 0.2848%
Intended Use
The adapter is intended for generating concise summaries of conversational text, particularly dialogue-style inputs similar to the SAMSum dataset.
It can be used for:
Conversation summarization
Meeting/dialogue summarization
Chat summarization
NLP research and experimentation
Educational and research projects involving parameter-efficient fine-tuning
Training
Dataset
The model was fine-tuned on the SAMSum dataset , which contains conversations paired with human-written summaries.
The training data was formatted as instruction-style prompts for Llama 3.2 Instruct.
LoRA Configuration
1 r = 16
2 lora_alpha = 32
3 lora_dropout = 0.05
4 bias = none
5 task_type = CAUSAL_LM
6 target_modules = q_proj, k_proj, v_proj, o_proj
Training Configuration
Epochs: 1Training steps: 1,842Final training loss: 2.07446Mean token accuracy: 0.5798Training time: approximately 2 hours 58 minutesTraining hardware: NVIDIA Tesla T4 16 GB
Evaluation
The model was evaluated on 100 SAMSum examples using ROUGE metrics.
Base Model vs Fine-Tuned Model
Metric Base Model Fine-Tuned Model Improvement ROUGE-1 0.3273 0.5287 +59.7% ROUGE-2 0.1078 0.2787 +158.3% ROUGE-L 0.2420 0.4486 +85.4%
The results show a substantial improvement in dialogue summarization performance after fine-tuning.
Results
The fine-tuned model achieved:
ROUGE-1: 0.5287ROUGE-2: 0.2787ROUGE-L: 0.4486
Compared with the base model, the largest relative improvement was observed in ROUGE-2 , increasing by approximately 158% .
Adapter Size
This repository contains the LoRA adapter weights , not the full base model.
The adapter is approximately 36.7 MB , making it significantly smaller than storing the complete 3B-parameter model.
To use the adapter, the original Llama-3.2-3B-Instruct base model must also be available.
How to Use
Install the required packages:
pip install transformers peft accelerate bitsandbytes torchThen load the base model and attach the adapter:
1 import torch
2 from transformers import AutoTokenizer , AutoModelForCausalLM , BitsAndBytesConfig
3 from peft import PeftModel
4
5 base_model = "meta-llama/Llama-3.2-3B-Instruct"
6 adapter_model = "YOUR_HUGGINGFACE_USERNAME/YOUR_ADAPTER_REPOSITORY"
7
8 quantization_config = BitsAndBytesConfig (
9 load_in_4bit = True ,
10 bnb_4bit_quant_type = "nf4" ,
11 bnb_4bit_compute_dtype = torch . float16
12 )
13
14 tokenizer = AutoTokenizer . from_pretrained ( base_model )
15
16 model = AutoModelForCausalLM . from_pretrained (
17 base_model ,
18 quantization_config = quantization_config ,
19 device_map = "auto"
20 )
21
22 model = PeftModel . from_pretrained (
23 model ,
24 adapter_model
25 )
26
27 prompt = """Summarize the following conversation:
28
29 Person 1: Hey, are you coming to the meeting?
30 Person 2: Yes, I'll be there at 3 PM.
31 Person 1: Great, see you then.
32 """
33
34 inputs = tokenizer ( prompt , return_tensors = "pt" ) . to ( model . device )
35
36 with torch . no_grad ( ) :
37 outputs = model . generate (
38 ** inputs ,
39 max_new_tokens = 100 ,
40 temperature = 0.7
41 )
42
43 print ( tokenizer . decode ( outputs [ 0 ] , skip_special_tokens = True ) )
Replace:
YOUR_HUGGINGFACE_USERNAME/YOUR_ADAPTER_REPOSITORYwith the Hugging Face repository containing this adapter.
Limitations
The model was trained specifically on dialogue summarization data and may not perform equally well on unrelated tasks.
Evaluation was performed on a limited sample of 100 examples.
ROUGE scores do not fully capture summary quality, factuality, or readability.
The adapter requires the original Llama 3.2 3B base model.
The model may occasionally produce inaccurate, incomplete, or overly detailed summaries.
Performance may vary for conversations significantly different from the SAMSum dataset.
Technical Specifications
Architecture
The underlying model is Llama 3.2 3B Instruct .
Only a small set of attention projection layers were adapted using LoRA:
1 q_proj
2 k_proj
3 v_proj
4 o_proj
Software
The fine-tuning workflow used:
Python
PyTorch
Transformers
PEFT
TRL
BitsAndBytes
Accelerate
Compute
GPU: NVIDIA Tesla T4VRAM: 16 GBTraining duration: ~2h 58mQuantization: 4-bit NF4Fine-tuning: QLoRA
Model Format
This repository provides the LoRA adapter and tokenizer files.
The full base model is not included because it is available separately through the original Llama 3.2 3B model repository.
Acknowledgements
This project was completed as part of an NLP internship project at Elevvo Pathways .
The project demonstrates parameter-efficient fine-tuning of a modern instruction-tuned language model for dialogue summarization using QLoRA.
License
The underlying base model is subject to the license and terms provided by Meta for Llama 3.2 .
Please review the base model's license and acceptable-use terms before using or redistributing the model.
Citation
If you use this adapter in your project, please reference the original Llama 3.2 model and the SAMSum dataset.
Author
Abdelrahman Mohamed
NLP / AI Student
Fine-tuning, NLP, Deep Learning & Computer Vision
