Hermes is very, um, Hermes-y. I wanted to dilute it so I could use it as an ingredient for other things. Sampling Hermes is a pain in the ass, it either sounds super model-esque or it loses all instructability. Hence, dilution back to the root.

This is a merge of pre-trained language models created using mergekit. We've been using this as one of the experimental ingredients to help stabilize the monkey-typewriter merges, and it's kinda okay at that.

Note that modern mergekit handles MoE just fine, now. But back in the day it did a horrible job and only the fork worked properly.

This model was merged using the DARE TIES merge method using mistralai/Mixtral-8x7B-Instruct-v0.1 as a base.

The following models were included in the merge:

The following YAML configuration was used to produce this model: