This is a merge of pre-trained language models created using
mergekit.
This model was merged using the
task arithmetic merge method using
meta-llama/Meta-Llama-3.1-8B-Instruct as a base.
1models:
2 - model: meta-llama/Meta-Llama-3-8B-Instruct
3 parameters:
4 weight: 0.2
5 - model: meta-llama/Meta-Llama-3.1-8B-Instruct
6 parameters:
7 weight: 0.8
8
9base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
10merge_method: task_arithmetic
11parameters:
12 normalize: true
13 int8_mask: true
14
15dtype: bfloat16
16
17
Detailed results can be found
here
