This is an experimental merge of pre-trained language models created using mergekit. All source model weights are BF16, avoiding issues arising from mixed-precision merges.

Although the zephyr beta and WizardLM 2 7B models are touted as SOTA and can generate varied prose compared to base Mistral v0.1, their relatively mediocre benchmarks under GSM-8K suggests only average reasoning capability in one-shot narrative text completion. The kuno-royale-v2 model was selected for merger because of its higher GSM-8K rating.

Native prompt format is Alpaca, although at least one of the prior models was fine-tuned to Vicuna.

Tested lightly with ChatML instruct prompts, temperature 1, and minP 0.02.

This model was merged using the SLERP merge method.

The following models were included in the merge:

The following YAML configuration was used to produce this model: