LoRA trained in 4-bit with 8k context using
mistralai/Mistral-Nemo-Base-2407 as the base model for 1 epoch.
Changed to ChatML since it might be confusing to use Llama 3 Instruct template on a Mistral model...
This mistral model was trained 2x faster with
Unsloth and Huggingface's TRL library.
