This is a merge of pre-trained language models created using
mergekit.
This model was merged using the
TIES merge method using
Qwen/Qwen2.5-1.5B-Instruct as a base.
1base_model: Qwen/Qwen2.5-1.5B-Instruct
2dtype: bfloat16
3merge_method: ties
4models:
5- model: Qwen/Qwen2.5-1.5B-Instruct
6 parameters:
7 density: 0.5
8 weight: 0.4
9- model: Qwen/Qwen2.5-Coder-1.5B-Instruct
10 parameters:
11 density: 0.5
12 weight: 0.3
13- model: Qwen/Qwen2.5-1.5B
14 parameters:
15 density: 0.5
16 weight: 0.3
17parameters:
18 int8_mask: true
19 normalize: true
20tokenizer_source: base
21
