This is a fine-tuned version of the Llama model trained using the ORPO (Optimized Ranked Preference Ordering) dataset (mlabonne/orpo-dpo-mix-40k) to enhance conversational and preference-based response generation.
The model uses the LoRA (Low-Rank Adaptation) technique to achieve efficient adaptation with minimal additional parameters, allowing it to learn task-specific knowledge without extensive computational demands.
Hyperparameters
LoRA Configuration: r=8,
lora_alpha=16,
lora_dropout=0.1
ORPO Trainer Configuration:
Learning Rate: 1e-5
Max Length: 2048
Batch Size: 1
Epochs: 1
Model Performance
The model was evaluated on the hellaswag task, yielding the following metrics:
