HauntingEcho is a passthrough merge which attempts to merge together RP-capable Llama 3 8B models with the smartest Llama 3.1 8B models.

The original aim of this merge was to use Llama 3.1 8B as a base to create a 32k context RP model with the popular feel of L3 8B RP models like EtherealRainbow. It seems like 32k context is not achievable, but 14-16k is, and that's a significant improvement over L3 8B.

The new aim of this merge strategy is to hit 16k context, to give people a slightly smarter alternative to Nemo with a Llama 3 feel and the freedom to play with the temperature that Llama 3 offers. 18B has been chosen as a very deliberate size, to offer a model that maximizes parameter count & quantization quality for popular GPU sizes: 24GB GPUs can run the 8bpw model, and 16GB GPUs can run the 5bpw model easily.

Please, please, please, give feedback on this merge. It is simply the result of a theory and minimally tested to ensure that it doesn't completely collapse. I welcome feedback in the community section here, or on Discord (name is the same as on here). I can be found on the Chub & SillyTavern Discord servers in the local LLM channels frequently too.

As this is a release candidate, I have only provided GGUFs.

This model is built on an abliterated base and as such is largely uncensored. It can generate explicit, disturbing or offensive responses. Use responsibly. I am not responsible for your use of this model.

I'm using these sampler settings:

These are by no means the perfect settings, feel free to experiment.

I'd recommend Llama-3 Instruct prompting format:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>

{input}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

{output}<|eot_id|>

Some of the models included in the merge were trained on ChatML & Alpaca so you can try those. I have not tested them.

N/A.

The following models were used to create EtherealRainbow-v0.3-8B: