This is a merge of pre-trained language models created using
mergekit.
This model was merged using the della merge method using
meta-llama/Llama-3.1-8B-Instruct as a base.
1merge_method: della
2dtype: bfloat16
3parameters:
4 normalize: true
5models:
6 - model: ValiantLabs/Llama3.1-8B-ShiningValiant2
7 parameters:
8 density: 0.5
9 weight: 0.3
10 - model: ValiantLabs/Llama3.1-8B-Enigma
11 parameters:
12 density: 0.5
13 weight: 0.25
14base_model: meta-llama/Llama-3.1-8B-Instruct
15
Detailed results can be found
here
