Experiment26-7B is a merge of the following models using
mergekit:
1slices:
2 - sources:
3 - model: yam-peleg/Experiment26-7B
4 layer_range: [0, 32]
5 - model: EmbeddedLLM/Mistral-7B-Merge-14-v0.1
6 layer_range: [0, 32]
7merge_method: slerp
8base_model: yam-peleg/Experiment26-7B
9parameters:
10 t:
11 - filter: self_attn
12 value: [0, 0.5, 0.3, 0.7, 1]
13 - filter: mlp
14 value: [1, 0.5, 0.7, 0.3, 0]
15 - value: 0.5
16dtype: bfloat16
17# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
18
19
20| Metric |Value|
21|---------------------------------|----:|
22|Avg. |74.96|
23|AI2 Reasoning Challenge (25-Shot)|71.84|
24|HellaSwag (10-Shot) |88.09|
25|MMLU (5-Shot) |65.07|
26|TruthfulQA (0-shot) |72.16|
27|Winogrande (5-shot) |82.32|
28|GSM8k (5-shot) |70.28|
