Views
No views yet
1base_model: unsloth/Meta-Llama-3.1-8B-Instruct
2dtype: bfloat16
3merge_method: dare_ties
4parameters:
5 int8_mask: 1.0
6 normalize: 1.0
7 random_seed: 145.0
8slices:
9- sources:
10 - layer_range: [0, 32]
11 model: unsloth/Llama-3.1-Storm-8B
12 parameters:
13 density: 0.94
14 weight: 0.35
15 - layer_range: [0, 32]
16 model: arcee-ai/Llama-3.1-SuperNova-Lite
17 parameters:
18 density: 0.92
19 weight: 0.2
20 - layer_range: [0, 32]
21 model: VAGOsolutions/Llama-3.1-SauerkrautLM-8b-Instruct
22 parameters:
23 density: 0.91
24 weight: 0.2
25 - layer_range: [0, 32]
26 model: Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2
27 parameters:
28 density: 0.93
29 weight: 0.19
30 - layer_range: [0, 32]
31 model: allenai/Llama-3.1-Tulu-3-8B
32 parameters:
33 density: 0.92
34 weight: 0.03
35 - layer_range: [0, 32]
36 model: FreedomIntelligence/HuatuoGPT-o1-8B
37 parameters:
38 density: 0.92
39 weight: 0.03
40 - layer_range: [0, 32]
41 model: unsloth/Meta-Llama-3.1-8B-Instruct
42tokenizer:
43 tokens:
44 <pad>:
45 source:
46 kind: model_token
47 model: unsloth/Meta-Llama-3.1-8B-Instruct
48 token: <|finetune_right_pad_id|>
49 <|begin_of_text|>:
50 force: true
51 source: unsloth/Meta-Llama-3.1-8B-Instruct
52 <|eot_id|>:
53 force: true
54 source: unsloth/Meta-Llama-3.1-8B-Instruct
55 <|finetune_right_pad_id|>:
56 force: true
57 source: unsloth/Meta-Llama-3.1-8B-Instruct| Metric | Value (%) |
|---|---|
| Average | 28.20 |
| IFEval (0-Shot) | 76.21 |
| BBH (3-Shot) | 31.47 |
| MATH Lvl 5 (4-Shot) | 16.77 |
| GPQA (0-shot) | 7.94 |
| MuSR (0-shot) | 7.19 |
| MMLU-PRO (5-shot) | 29.62 |