Cassiopeia-70B is the result of an experimental multi-step SLERP merge of
Llama-3.3-70B-Instruct,
Shimamura-70B, and
Anubis-70B-v1.1. It is a coherent, unaligned model intended to be used for creative tasks such as storywriting, brainstorming, interactive roleplay, etc.
Stay close to Anubis v1.1, but step 20% closer to stock L3.3 70B.
1models:
2 - model: /opt/workspace/hf/Anubis-70B-v1.1
3 - model: /opt/workspace/hf/Llama-3.3-70B-Instruct
4merge_method: slerp
5base_model: /opt/workspace/hf/Anubis-70B-v1.1
6parameters:
7 t: 0.2
8dtype: bfloat16
Pull the middle of the intermediate model towards Shimamura, but leave the ends as they are.
1models:
2 - model: /opt/workspace/hf/Anubis-70B-v1.1-0.8x
3 - model: /opt/workspace/hf/Shimamura-70B
4merge_method: slerp
5base_model: /opt/workspace/hf/Anubis-70B-v1.1-0.8x
6parameters:
7 t: [0.0, 0.5, 1.0, 0.5, 0.0]
8dtype: bfloat16
Feedback on this merge is very welcome, good or bad! Please leave a comment in this discussion with your thoughts:
Cassiopeia-70B/discussions/1
