And merged them onto their base model.
I then did a simple linear merge between them, an experimental unreleased 8x7B model, and an unreleased model that was a intermediate step in creating
Envoid/BondBurger-8x7B
The end results are a surprisingly good model for role palying style entertainment.
At first I was disappointed with the results but ended up settling on the following sampler parameters which really bring it to life.
It responds well to [INST] do a thing [/INST] instruct style formatting (although uses the ChatML special tokens)