This model is a fine-tuned version of
mistralai/Mixtral-8x7B-v0.1 on the dataset oasst2 (OpenAssistant).
It achieves the following results on the evaluation set:
This is a LoRA trained on OpenAssistant data.
The settings for the base model should be:
Model loader: Transformers
Compute_dtype: bfloat16
quant_type: nf4
cpu: enabled
load-in-4bit: enabled
use_double_quant: enabled
set GPU memory as high as possible unless running locally to give some space for your desktop environment
tweak CPU usage until it loads successfully
