Views
No views yet
1# DARE + Average Merging
2# random seed: 42
3# Each parameter refers to DARE [https://arxiv.org/abs/2311.03099]
4base_model: mistralai/Mistral-7B-v0.1
5models:
6 - model: mistralai/Mistral-7B-Instruct-v0.2
7 parameters:
8 density: 0.2
9 weight: 1.0
10 - model: TIGER-Lab/MAmmoTH2-7B
11 parameters:
12 density: 0.2
13 weight: 1.0
14 - model: Nondzu/Mistral-7B-codealpaca-lora
15 parameters:
16 density: 0.2
17 weight: 1.0
18merge_method: dare_linear
19parameters:
20 normalize: true
21 lambda: 1.0
22 rescale: true
23dtype: float16
24tokenizer:
25 source: union
26