Views
No views yet
1slices:
2 - sources:
3 - model: bamec66557/MISCHIEVOUS-12B-Mix_0.4v
4 layer_range: [0, 15]
5 - model: bamec66557/MISCHIEVOUS-12B-Mix_III_ex_V
6 layer_range: [0, 15]
7 parameters:
8 t:
9 - value: 0.8
10 - sources:
11 - model: bamec66557/MISCHIEVOUS-12B-Mix_0.4v
12 layer_range: [15, 30]
13 - model: bamec66557/MISCHIEVOUS-12B-Mix_III_ex_V
14 layer_range: [15, 30]
15 parameters:
16 t:
17 - value: [0.8, 1.0, 0.8]
18 - sources:
19 - model: bamec66557/MISCHIEVOUS-12B-Mix_0.4v
20 layer_range: [30, 40]
21 - model: bamec66557/MISCHIEVOUS-12B-Mix_III_ex_V
22 layer_range: [30, 40]
23 parameters:
24 t:
25 - value: 1.0
26 - filter: self_attn
27 value: [0.9, 1.0, 1.1]
28merge_method: slerp
29base_model: bamec66557/MISCHIEVOUS-12B-Mix_0.4v
30dtype: bfloat16
31regularization:
32 - method: weight_clipping
33 clip_range: [-0.1, 0.1]
34postprocessing:
35 - operation: gaussian_smoothing
36 sigma: 1.2
37 - operation: normalize| Metric | Value |
|---|---|
| Avg. | 23.22 |
| IFEval (0-Shot) | 40.31 |
| BBH (3-Shot) | 34.85 |
| MATH Lvl 5 (4-Shot) | 12.84 |
| GPQA (0-shot) | 9.40 |
| MuSR (0-shot) | 12.34 |
| MMLU-PRO (5-shot) | 29.60 |