Supervised fine-tuning refers to a machine learning technique where a pre-trained model is further trained on a specific task or dataset with labeled examples (supervised learning). The process involves taking a model that has been pre-trained on a large general dataset and then adapting it to a more focused task by continuing the training using task-specific data.

This model is test version, alignment-tuned model.

We utilize state-of-the-art instruction fine-tuning methods including direct preference optimization (DPO).