Views
No views yet
passthrough method work, mostly the result responds with gibberish and my stupid brain thinking that
doing something random will magically create what I want. It took me 3 tries to get it right, but still... I didn't know why it work now.passthrough merge method.1
2slices:
3 - sources:
4 - model: RozGrov/NemoDori-v0.2-12B-MN-BT
5 layer_range: [0, 8]
6 - sources:
7 - model: RozGrov/NemoDori-v0.2-12B-MN-BT
8 layer_range: [8, 24]
9 parameters:
10 scale:
11 - filter: q_proj
12 value: 0.919
13 - filter: k_proj
14 value: 0.919
15 - value: 1.0
16 - sources:
17 - model: RozGrov/NemoDori-v0.2-12B-MN-BT
18 layer_range: [16, 32]
19 parameters:
20 scale:
21 - filter: q_proj
22 value: 0.919
23 - filter: k_proj
24 value: 0.919
25 - value: 1.0
26 - sources:
27 - model: RozGrov/NemoDori-v0.2-12B-MN-BT
28 layer_range: [32, 40]
29merge_method: passthrough
30dtype: bfloat16
31