Views
No views yet
adapter: qlora
load_in_4bit: true
lora_r: 32
lora_alpha: 16
lora_dropout: 0.001
lora_target_linear: true
relora_steps: 150
relora_warmup_steps: 10
gradient_accumulation_steps: 2
micro_batch_size: 3" ***System:", " ***Query:", or " ***Response:", paying attention to whitespace.| Metric | Value |
|---|---|
| Avg. | 52.21 |
| ARC (25-shot) | 57.68 |
| HellaSwag (10-shot) | 82.44 |
| MMLU (5-shot) | 55.33 |
| TruthfulQA (0-shot) | 43.61 |
| Winogrande (5-shot) | 77.35 |
| GSM8K (5-shot) | 6.6 |
| DROP (3-shot) | 42.46 |
