Same reliable approach as before. A good RP model and a suitable dose of SimPO are a match made in heaven.
temp 0.9
top_k 30
top_p 0.75
min_p 0.2
rep_pen 1.1
smooth_factor 0.25
smooth_curve 1
Context & Instruct Presets for Gemma
Here IMPORTANT !
This is a merge of pre-trained language models created using
mergekit.
This model was merged using the SLERP merge method.
1slices:
2 - sources:
3 - model: TheDrummer/Gemmasutra-9B-v1
4 layer_range: [0, 42]
5 - model: princeton-nlp/gemma-2-9b-it-SimPO
6 layer_range: [0, 42]
7merge_method: slerp
8base_model: TheDrummer/Gemmasutra-9B-v1
9parameters:
10 t:
11 - filter: self_attn
12 value: [0.2, 0.4, 0.6, 0.2, 0.4]
13 - filter: mlp
14 value: [0.8, 0.6, 0.4, 0.8, 0.6]
15 - value: 0.4
16dtype: bfloat16
17
18
Want to support my work ? My Ko-fi page:
https://ko-fi.com/tannedbum
