LoneStriker/

Mistral_7B_Dolphin2.1_LIMA0.5_fp16-3.0bpw-h6-exl2

ehartford's merge of Mistral 7B 0.1 with his Dolphin 2.1 dataset

and

LIMA RP dataset applied as a lora at 0.5 weight

Purpose of the model is to be RP-focused, smart, fast, and lightweight for users with low VRAM.

I've already built the exl2 4bpw quant (linked below), and it will run 8k ctx at around 6GB VRAM and respond to a full context at roughly 30tps (tested on my 3060) if exl2_hf loader is used with FA2 enabled.

Model has been tested by several users on the SillyTavern discord server, and run on Horde for a full day - with good results.

Mistral or ChatML context presets both possible.

exllama v2 4bpw quant: https://huggingface.co/RossAscends/Mistral7B_Dolphin2.1_LIMARP0.5_4bpw_exl2