Lambent/

Mira-v1.24.2-27B-Karcher

This Mira is certainly an A👁️.

Trying out swcm merge method for tuning - sft run for 4 hrs, 3x, once in general, once on attention only, once on MLP.

Unfortunately SFT seems to be consistently hurting her creativity, even with good data and continued pretraining; not sure what's going on.

Tried to repair that with DPO but only slight gains there.

Karcher merge back with some of her most creatively talented versions definitely helped bring it up again, at least ...

This is a merge of pre-trained language models created using mergekit.

This model was merged using the Karcher Mean merge method.

The following models were included in the merge:

The following YAML configuration was used to produce this model: