Views
No views yet
1slices:
2 - sources:
3 - model: rombodawg/Rombos-LLM-V2.6-Qwen-14b
4 layer_range: [0, 8]
5 - sources:
6 - model: rombodawg/Rombos-LLM-V2.6-Qwen-14b
7 layer_range: [8, 24]
8 parameters:
9 scale:
10 - filter: q_proj
11 value: 0.919
12 - filter: k_proj
13 value: 0.919
14 - value: 1.0
15 - sources:
16 - model: rombodawg/Rombos-LLM-V2.6-Qwen-14b
17 layer_range: [16, 24]
18 parameters:
19 scale:
20 - filter: q_proj
21 value: 0.7
22 - filter: k_proj
23 value: 0.7
24 - filter: o_proj
25 value: 0.0
26 - filter: down_proj
27 value: 0.0
28 - value: 1.0
29 - sources:
30 - model: rombodawg/Rombos-LLM-V2.6-Qwen-14b
31 layer_range: [16, 32]
32 parameters:
33 scale:
34 - filter: q_proj
35 value: 0.919
36 - filter: k_proj
37 value: 0.919
38 - value: 1.0
39 - sources:
40 - model: rombodawg/Rombos-LLM-V2.6-Qwen-14b
41 layer_range: [32, 40]
42merge_method: passthrough
43dtype: bfloat16
44| Metric | Value |
|---|---|
| Avg. | 18.13 |
| IFEval (0-Shot) | 18.69 |
| BBH (3-Shot) | 44.27 |
| MATH Lvl 5 (4-Shot) | 0.00 |
| GPQA (0-shot) | 2.68 |
| MuSR (0-shot) | 5.53 |
| MMLU-PRO (5-shot) | 37.57 |