mixtral-8x7b is a Mixture-of-Expert (MoE) model.
LLaMA2-Accessory has supported its inference and finetuning.
We host a web demo
💻here, which shows a mixtral-8x7b model finetuned on
evol-codealpaca-v1 and
ultrachat_200k, with LoRA and Bias tuning.
A detailed tutorial is available at our
document
