Views
No views yet
1dtype: float16
2merge_method: linear
3slices:
4 - sources:
5 - layer_range: [0, 16] # Assuming the first half of the model is more general and can be reduced more
6 model: mistralai/Mistral-7B-Instruct-v0.2
7 parameters:
8 weight: 0.5 # Reduce the weight of the first half to make room for the second half
9 - layer_range: [16, 32] # Assuming the second half of the model is more specialized and can be reduced less
10 model: mistralai/Mistral-7B-Instruct-v0.2
11 parameters:
12 weight: 0.5 # Maintain the weight of the second half