athirdpath/

NeuralHermes-Mistral-13b-DARE_blended-FAILURE

Ooof, my man ain't feeling so hot, I'd pass on this one for now. Inverting and merging 20b Llama 2 models works quite well, evening out the gradients between slices. However, these 13b Mistrals seem to HATE it, I assume due to the unbalanced nature of my recipe. More study is required.

merge_method: dare_ties

int8_mask: true

dtype: bfloat16