Using
PruneMe to find minimal average distance. Thank you for awesome toolkit @arcee-ai !
distanceI haven't done any post-training (called 'healing' process as the
paper suggests), will do it later but no guarantee at all.
This is a merge of pre-trained language models created using
mergekit.
This model was merged using the passthrough merge method.
1dtype: bfloat16
2merge_method: passthrough
3slices:
4- sources:
5 - layer_range: [0, 21]
6 model:
7 model:
8 path: meta-llama/Meta-Llama-3-8B-Instruct
9- sources:
10 - layer_range: [29, 32]
11 model:
12 model:
13 path: meta-llama/Meta-Llama-3-8B-Instruct
