Large language models have achieved groundbreaking success in the field of natural language processing (NLP). However, since these models are generally trained for general-purpose tasks, they may not perform optimally for specific tasks. Therefore, fine-tuning these large models for specific tasks is a common practice. In this article, we will delve into the process of fine-tuning and adapting the Zephyr-7B-alpha-GPTQ, a large language model, for a particular task.
Training hyperparameters
The following hyperparameters were used during training:
learning_rate: 0.0002
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
