This repo contains a merge of pre-trained language models created using mergekit.

For this experiment, a LoRA extracted from a merge of models trained on Llama 3 8B Instruct was applied to Llama 3.1 8B Instruct. The theoretical goal was to obtain the stylistic benefit from the LoRA and the longer context of Llama 3.1. The resulting model is generally coherent. Tested at temperature=1 with minP=0.01.

This model was merged using the passthrough merge method using grimjim/Llama-3.1-8B-Instruct-abliterated_via_adapter + grimjim/Llama-3-Instruct-Nephilim-v3-LoRA-8B as a base.

The following YAML configuration was used to produce this model: