This is a Llama 2 Fine Tuned Model with 7 Billion Parameters on the Dataset from mlabonne/guanaco-llama2. The training data is basically a Chat between a Human and an Assistant where the Human poses some queries and the Assistant responds to those queries in a suitable fashion.
In the case of Llama 2, the following Chat Template is used for the chat models:
(s)[INST] ((sys))
SYSTEM PROMPT
((/sys))
User Prompt [/INST] Model Answer (/s)
System Prompt (optional) --> to guide the model
User prompt (required) --> to give the instruction / User Query
Model Answer (required)
Training Data
The Instruction Dataset is reformated to follow the above Llama 2 template.
To drastically reduce the VRAM usage, we must fine-tune the model in 4-bit precision, which is why we’ll use QLoRA here and the GPU on which the model was fined tuned on was L4 (Google Colab Pro)
Process
Load the dataset as defined.
Configure bitsandbytes for 4-bit quantization.
Load the Llama 2 model in 4-bit precision on a GPU (L4 - Google Colab Pro) with the corresponding tokenizer.
Loading configurations for QLoRA, regular training parameters, and pass everything to the SFTTrainer.
