This model aims to achieve the human-like responses of the
Halu Blackroot, the no refusal tendencies of the
Halu OAS, and the smartness of the
Standard Halu.
-
Anjrit: This model is similar to my
Halu Blackroot model, but instead of using the standard version, this model uses the OAS version.
-
Anying: This model is also similar to the Halu Blackroot, but instead of using the model stock, I merged the Blackroot lora manually with a very low alpha.
Both models have downsides. The Anjrit model lacks coherency, while the Anying model lacks a human-like responses.
-
First, I compared the response from each layer of both models using the baukit notebook.
-
After comparing both, it seems that around the bottom layer, the Anjrit model is better, perhaps because it is unhinged.
-
From the bottom to the middle layer, the Anjrit is still better, but the Anying seems smarter.
-
At the middle layer, both seem equal, but again, the Anjrit is unhinged, so I prefer this one.
-
From the middle to the top layer, the Anying is better. It is smarter, and the response is more structured.
-
The top layer of the Anjrit model is better since the model itself is orthogonalized, so I prefer this one.
-
Then I performed slerp with the following configuration. I don't know if this is really how the slerp merge works, so let's just say this is an experimental merge. Maybe I will try the other merge methods for future experiments
1models:
2 - model: Hastagaras/anjrit
3 - model: Hastagaras/anying
4merge_method: slerp
5base_model: Hastagaras/anjrit
6dtype: bfloat16
7parameters:
8 t: [0.12, 0.17, 0.29, 0.44, 0.26]
9
Detailed results can be found
here
