PLEASE NOTE This version has an extra dimension, sadly. Due to the union base tokenizer +7 tokens in Trickster's tokenizer. I didn't realise it was through every single layer and that just changing the config would do nothing. This merge be broke. It won't produce output for you. It's here for my reference.

I wanted Hermes 2 Theta's qualities to be more marked, so I took it through a second merge. I have yet to find out how it worked. Will post an update as soon as I get that done.

This is a merge of pre-trained language models created using mergekit.

The SCE method I am using for all my SCE merges isn't "pure" SCE. It's an SCE-style hybrid merge.

I used merge_method: sce (so the mergekit math uses the SCE normalization and cosine weighting), but then I overrode its democracy by specifying filters and manual weights per subsystem—attn, mlp, lm_head.

That means:

I kept SCE’s normalization, top-k filtering, and prescale behaviour (so everything stays "stable and smooth" cough in theory).

But I layered on selective weighting, steering which model dominates which parts of the architecture.

In other words, I hijacked SCE’s math engine to run a directed SCE merge—a hybrid between SCE and TIES / linear weighting.

It runs like SCE - for example this one took 7 hours to merge.

This model was merged using the SCE merge method using Babsie/Trickster-Theta-4-70B as a base.

The following models were included in the merge: *NousResearch/Hermes-2-Theta-Llama-3-70B (RoPE config adapted in my own repo, preprepped)

The following YAML configuration was used to produce this model: