Phi and Mistral works with interesting results.. I quite like it with my settings.
This is a merge of pre-trained language models created using
mergekit.
This model was merged using the
TIES merge method using
pot99rta/CaptainMaid-12B-VioletMell-V0.420 as a base.
1models:
2 - model: pot99rta/CaptainMaid-12B-VioletMell-V0.420
3 #no parameters necessary for base model
4 - model: pot99rta/CaptainMaid-12B-VioletMell-V0.420
5 parameters:
6 density: 0.5
7 weight: 0.5
8 - model: Nitral-AI/Captain_BMO-12B
9 parameters:
10 density: 0.5
11 weight: 0.5
12
13merge_method: ties
14base_model: pot99rta/CaptainMaid-12B-VioletMell-V0.420
15parameters:
16 normalize: false
17 int8_mask: true
18dtype: float16
