Views
No views yet
| Model | Size | Coding | Extraction | Humanities | Math | Reasoning | Roleplay | STEM | Writing | avg_score |
|---|---|---|---|---|---|---|---|---|---|---|
| Mixtral-8x7B-Instruct-v0.1 | 8x7B | 5.3 | 8.5 | 9.9 | 6.8 | 6.0 | 9.1 | 9.55 | 8.9 | 8.00625 |
| This model | around 8x12B? | 6.3 | 8.4 | 9.9 | 5.4 | 7.7 | 9.2 | 9.75 | 9.8 | 8.30625 |
| Model | Size | Coding | Extraction | Humanities | Math | Reasoning | Roleplay | STEM | Writing | avg_score |
|---|---|---|---|---|---|---|---|---|---|---|
| Mixtral-8x7B-Instruct-v0.1 | 8x7B | 4.1 | 8.4 | 9.8 | 4.7 | 5.6 | 9.0 | 9.2 | 9.5 | 7.5375 |
| This model | around 8x12B? | 4.2 | 7.4 | 9.9 | 4.0 | 5.2 | 9.5 | 8.7 | 8.0 | 7.1125 |
1merge_method: passthrough
2slices:
3 - sources:
4 - model: mistralai/Mixtral-8x7B-Instruct-v0.1
5 layer_range: [0, 8]
6 - sources:
7 - model: mistralai/Mixtral-8x7B-Instruct-v0.1
8 layer_range: [4, 12]
9 - sources:
10 - model: mistralai/Mixtral-8x7B-Instruct-v0.1
11 layer_range: [8, 16]
12 - sources:
13 - model: mistralai/Mixtral-8x7B-Instruct-v0.1
14 layer_range: [12, 20]
15 - sources:
16 - model: mistralai/Mixtral-8x7B-Instruct-v0.1
17 layer_range: [16, 24]
18 - sources:
19 - model: mistralai/Mixtral-8x7B-Instruct-v0.1
20 layer_range: [20, 28]
21 - sources:
22 - model: mistralai/Mixtral-8x7B-Instruct-v0.1
23 layer_range: [24, 32]
24dtype: bfloat16
25tokenizer_source: base
26