Views
No views yet
| Metric | Value |
|---|---|
| GSM8k (zero-shot) | 90.06 |
| HellaSwag (zero-Shot) | 82.77 |
| MBPP (zero-shot) | 62.21 |
1base_model: Qwen/Qwen2.5-7B
2dtype: bfloat16
3merge_method: breadcrumbs
4parameters:
5 lambda: 0.9075603207928135
6 normalize: 1.0
7slices:
8- sources:
9 - layer_range: [0, 28]
10 model: Qwen/Qwen2.5-7B
11 - layer_range: [0, 28]
12 model: Qwen/Qwen2.5-Math-7B
13 parameters:
14 density: 0.11722197443445775
15 gamma: 0.07547691839721048
16 weight: 0.17267293536872041
17 - layer_range: [0, 28]
18 model: Qwen/Qwen2.5-Coder-7B
19 parameters:
20 density: 0.48352747334554935
21 gamma: 0.0753405327865558
22 weight: 0.11164770709858211
23 - layer_range: [0, 28]
24 model: Qwen/Qwen2.5-7B-Instruct
25 parameters:
26 density: 0.8190520808683315
27 gamma: 0.022307694128235696
28 weight: 0.7626295102691242