This is quantized version of
Hastagaras/HALU-8B-LLAMA3-BRSLURP created suing llama.cpp
You can see the Halu 0.35 model details in
HERESo two different models with different base models...a fusion of OpenAI and MetaAI, truthful QA gonna be tough.
After some testing, I think this super duper easy merge that I did while I was half asleep is actually pretty decent.
After another testing...the Blackroot influence is way smoother than the Anjir, probably because the base models are different, so...no duplicate layers, I guess.
Works better with around 0.95-1.1 temp.
1slices:
2 - sources:
3 - model: Hastagaras/Halu-8B-Llama3-v0.35
4 layer_range: [0,32]
5 - model: Hastagaras/Halu-8B-Llama3-Blackroot
6 layer_range: [0,32]
7merge_method: slerp
8base_model: Hastagaras/Halu-8B-Llama3-v0.35
9parameters:
10 t:
11 - filter: self_attn
12 value: [0, 0.5, 0.5, 0.5, 1]
13 - filter: mlp
14 value: [1, 0.5, 0.5, 0.5, 0]
15 - value: 0.5
16dtype: bfloat16
17
