This is an agentic model created by merging multiple fine-tuned models based on
Qwen3.5-4B using a version of Mergekit forked to support Qwen3.5.
Compared to Mephisto-4B, v2 involved a more meaningful merging process; rather than completing it in a single step, a multi-stage merging approach was used.
1slices:
2 - sources:
3 - model: Jackrong/Qwopus3.5-4B-v3
4 layer_range: [0, 32]
5 - model: Jackrong/Qwopus3.5-4B-Coder
6 layer_range: [0, 32]
7merge_method: slerp
8base_model: Jackrong/Qwopus3.5-4B-v3
9parameters:
10 t:
11 - filter: self_attn
12 value: [0.2, 0.3, 0.4, 0.5, 0.6, 0.6, 0.5]
13 - filter: mlp
14 value: [0.5, 0.5, 0.5, 0.5, 0.5]
15 - value: 0.5
16dtype: bfloat16
1slices:
2 - sources:
3 - model: BAAI/AREX-Turbo
4 layer_range: [0, 32]
5 weight: 0.5
6 - model: InternScience/Agents-A1-4B
7 layer_range: [0, 32]
8 weight: 0.5
9merge_method: slerp
10base_model: BAAI/AREX-Turbo
11parameters:
12 t:
13 - value: 0.5
14dtype: bfloat16
1merge_method: dare_ties
2base_model: /path/to/Qwen/Qwen3.5-4B
3
4models:
5 - model: /home/CloudGoat/llms_merge/Qwents-4B
6 parameters:
7 weight: 1.0
8 density: 1.0
9
10 - model: /home/CloudGoat/llms_merge/Qwopus3.5-4B-Fusion
11 parameters:
12 weight: 0.7
13 density: 0.3
14
15parameters:
16 int8_mask: true
17 normalize: false
18
19dtype: bfloat16
