LoRA rank: 8
LoRA alpha: 16
LoRA dropout: 0
Rank-stabilized LoRA: Yes
Number of epochs: 3
Learning rate: 1e-5
Batch size: 2
Gradient accumulation steps: 4
Weight decay: 0.01
Target modules:

  - Query projection (`q_proj`)
  - Key projection (`k_proj`)
  - Value projection (`v_proj`)
  - Output projection (`o_proj`)
  - Gate projection (`gate_proj`)
  - Up projection (`up_proj`)
  - Down projection (`down_proj`)

sayhan/OpenHermes-2.5-Strix-Philosophy-Mistral-7B-LoRA

Finetuned by: <a href="https://huggingface.co/sayhan" target="_blank" rel="noopener noreferrer" class="text-blue-400 hover:text-blue-300 underline underline-offset-2 transition-colors">sayhan
License: <a href="https://choosealicense.com/licenses/apache-2.0/" target="_blank" rel="noopener noreferrer" class="text-blue-400 hover:text-blue-300 underline underline-offset-2 transition-colors">apache-2.0
Finetuned from model: <a href="https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B" target="_blank" rel="noopener noreferrer" class="text-blue-400 hover:text-blue-300 underline underline-offset-2 transition-colors">teknium/OpenHermes-2.5-Mistral-7B
Dataset: <a href="https://huggingface.co/datasets/sayhan/strix-philosophy-qa" target="_blank" rel="noopener noreferrer" class="text-blue-400 hover:text-blue-300 underline underline-offset-2 transition-colors">sayhan/strix-philosophy-qa

OpenHermes 2.5 Stix Philosophy Mistral 7B

sayhan/
OpenHermes-2.5-Strix-Philosophy-Mistral-7B-LoRA