This is a merge of pre-trained language models created using
mergekit.
This model was merged using the
Linear DELLA merge method using
VAGOsolutions/Llama-3.1-SauerkrautLM-70b-Instruct as a base.
1models:
2 - model: Sao10K/L3.3-70B-Euryale-v2.3
3 parameters:
4 weight: 0.5
5 density: 0.7
6 epsilon: 0.2
7 lambda: 1.1
8 - model: VAGOsolutions/Llama-3.1-SauerkrautLM-70b-Instruct
9 parameters:
10 weight: 0.5
11 density: 0.7
12 epsilon: 0.1
13 lambda: 1
14base_model: VAGOsolutions/Llama-3.1-SauerkrautLM-70b-Instruct
15merge_method: della_linear
16parameters:
17 normalize: false
18 int8_mask: true
19chat_template: llama3
20tokenizer:
21 source: union
22dtype: float32
23out_dtype: bfloat16
