This is set of quantized GGUF files for a merge of pre-trained language models created using
mergekit.
An asymmetric gradient SLERP was used to lightly apply MN-12B-Mag-Mell-R1 to Magnolia-v3-12B.
Tested for narrative text completion with temperature=1.0 and minP=0.02. Coherence is fairly high, though there may be occasional slips. If repetition is a problem, raising temperature briefly may help. The model appears to tolerate temperature=2.0 even.
This model was merged using the
SLERP merge method.
1base_model: grimjim/Magnolia-v3-12B
2dtype: bfloat16
3merge_method: slerp
4slices:
5- sources:
6 - model: grimjim/Magnolia-v3-12B
7 layer_range: [0,40]
8 - model: inflatebot/MN-12B-Mag-Mell-R1
9 layer_range: [0,40]
10parameters:
11 t:
12 - filter: self_attn
13 value: [0.0,0.09]
14 - filter: mlp
15 value: [0.09,0.0]
16 - value: [0.0,0.09]
17
