Views
No views yet
1name: tempesthenno-nuslerp-0124
2merge_method: nuslerp
3tokenizer:
4 source: union
5chat_template: "chatml"
6dtype: float32
7out_dtype: bfloat16
8parameters:
9 int8_mask: true
10 normalize: true
11 rescale: false
12slices:
13 - sources:
14 - model: /Users/sthenno/models/tempesthenno--converge-dtask
15 layer_range: [0, 8]
16 parameters:
17 weight: 0.65
18 nuslerp_flatten: false
19 nuslerp_row_wise: true
20 - model: /Users/sthenno/models/tempesthenno--converge-breadcrumbs
21 layer_range: [0, 8]
22 parameters:
23 weight: 0.35
24 nuslerp_flatten: false
25 nuslerp_row_wise: true
26 - sources:
27 - model: /Users/sthenno/models/tempesthenno--converge-dtask
28 layer_range: [8, 16]
29 parameters:
30 weight: 0.60
31 nuslerp_flatten: false
32 nuslerp_row_wise: true
33 - model: /Users/sthenno/models/tempesthenno--converge-breadcrumbs
34 layer_range: [8, 16]
35 parameters:
36 weight: 0.40
37 nuslerp_flatten: false
38 nuslerp_row_wise: true
39 - sources:
40 - model: /Users/sthenno/models/tempesthenno--converge-dtask
41 layer_range: [16, 24]
42 parameters:
43 weight: 0.55
44 nuslerp_flatten: false
45 nuslerp_row_wise: false
46 - model: /Users/sthenno/models/tempesthenno--converge-breadcrumbs
47 layer_range: [16, 24]
48 parameters:
49 weight: 0.45
50 nuslerp_flatten: false
51 nuslerp_row_wise: false
52 - sources:
53 - model: /Users/sthenno/models/tempesthenno--converge-dtask
54 layer_range: [24, 32]
55 parameters:
56 weight: 0.50
57 nuslerp_flatten: false
58 nuslerp_row_wise: false
59 - model: /Users/sthenno/models/tempesthenno--converge-breadcrumbs
60 layer_range: [24, 32]
61 parameters:
62 weight: 0.50
63 nuslerp_flatten: false
64 nuslerp_row_wise: false
65 - sources:
66 - model: /Users/sthenno/models/tempesthenno--converge-dtask
67 layer_range: [32, 40]
68 parameters:
69 weight: 0.45
70 nuslerp_flatten: true
71 - model: /Users/sthenno/models/tempesthenno--converge-breadcrumbs
72 layer_range: [32, 40]
73 parameters:
74 weight: 0.55
75 nuslerp_flatten: true
76 - sources:
77 - model: /Users/sthenno/models/tempesthenno--converge-dtask
78 layer_range: [40, 48]
79 parameters:
80 weight: 0.40
81 nuslerp_flatten: true
82 - model: /Users/sthenno/models/tempesthenno--converge-breadcrumbs
83 layer_range: [40, 48]
84 parameters:
85 weight: 0.60
86 nuslerp_flatten: true
87| Metric | Value |
|---|---|
| Avg. | 40.97 |
| IFEval (0-Shot) | 70.04 |
| BBH (3-Shot) | 49.28 |
| MATH Lvl 5 (4-Shot) | 39.27 |
| GPQA (0-shot) | 18.68 |
| MuSR (0-shot) | 20.21 |
| MMLU-PRO (5-shot) | 48.36 |