Quantization made by Richard Erkhov.
image/png
MaidenlessNoMore-7B was my first attempt at merging an LLM
I decided to use one of the first models I really enjoyed that not many people know of:
https://huggingface.co/cookinai/Valkyrie-V1 with my other favorite model that has been my fallback model for a long time:
https://huggingface.co/SanjiWatsuki/Kunoichi-7B This was more of an experiment than anything else. Hopefully this will lead to some more interesting merges and who knows what else in the future.
I mean we have to start somewhere right?
Alpaca or Alpaca roleplay is recommended.
This is a merge of pre-trained language models created using
mergekit .
This model was merged using the SLERP merge method.
1 slices :
2 - sources :
3 - model : SanjiWatsuki/Kunoichi - 7B
4 layer_range : [ 0 , 32 ]
5 - model : cookinai/Valkyrie - V1
6 layer_range : [ 0 , 32 ]
7 merge_method : slerp
8 base_model : SanjiWatsuki/Kunoichi - 7B
9 parameters :
10 t :
11 - filter : self_attn
12 value : [ 0 , 0.5 , 0.3 , 0.7 , 1 ]
13 - filter : mlp
14 value : [ 1 , 0.5 , 0.7 , 0.3 , 0 ]
15
16 - value : 0.5
17 dtype : bfloat16
