This is a merge of pre-trained language models created using
mergekit.
This model was merged using the
TIES merge method using
Qwen/Qwen2.5-7B-Instruct as a base.
1base_model: Qwen/Qwen2.5-7B-Instruct
2dtype: bfloat16
3merge_method: ties
4parameters:
5 int8_mask: 1.0
6 normalize: 1.0
7slices:
8- sources:
9 - layer_range: [0, 28]
10 model: Qwen/Qwen2.5-7B-Instruct
11 parameters:
12 density: 0.7
13 weight: 0.4
14 - layer_range: [0, 28]
15 model: DHMATH/Qwen-7B-Instruct
16 parameters:
17 density: 1.0
18 weight: 0.6
19tokenizer_source: DHMATH/Qwen-7B-Instruct
