ehartford's merge of Mistral 7B 0.1 with his Dolphin 2.1 dataset
and
LIMA RP dataset applied as a lora at 0.5 weight
Purpose of the model is to be RP-focused, smart, fast, and lightweight for users with low VRAM.
I've already built the exl2 4bpw quant (linked below), and it will run 8k ctx at around 6GB VRAM and respond to a full context at roughly 30tps (tested on my 3060) if exl2_hf loader is used with FA2 enabled.
Model has been tested by several users on the SillyTavern discord server, and run on Horde for a full day - with good results.
Mistral or ChatML context presets both possible.
