Views
No views yet
Slerp merge method from the best models on 14th Dec on the
OpenLLM Leaderboard:1slices:
2 - sources:
3 - model: viethq188/LeoScorpius-7B-Chat-DPO
4 layer_range: [0, 32]
5 - model: GreenNode/GreenNodeLM-7B-v1olet
6 layer_range: [0, 32]
7merge_method: slerp
8base_model: GreenNode/GreenNodeLM-7B-v1olet
9parameters:
10 t:
11 - filter: lm_head
12 value: [0.55]
13 - filter: embed_tokens
14 value: [0.7]
15 - filter: self_attn
16 value: [0.65, 0.35]
17 - filter: mlp
18 value: [0.35, 0.65]
19 - filter: layernorm
20 value: [0.4, 0.6]
21 - filter: modelnorm
22 value: [0.6]
23 - value: 0.5 # fallback for rest of tensors
24dtype: bfloat16{system_message}
### Instruction:
{prompt}
### Response:
1337 with OpenAI compatible endpoints
| Metric | Value |
|---|---|
| Avg. | 74.8 |
| ARC (25-shot) | 72.27 |
| HellaSwag (10-shot) | 88.36 |
| MMLU (5-shot) | 65.2 |
| TruthfulQA (0-shot) | 69.31 |
| Winogrande (5-shot) | 82 |
| GSM8K (5-shot) | 71.65 |