LoRA trained in 4-bit with 8k context using
meta-llama/Meta-Llama-3-8B-Instruct as the base model for 1 epoch.
This has been trained on the instruct model and not the base model. The model trained with the base model using the same dataset is here:
mpasila/Llama-3-LiPPA-LoRA-8BUnsloth changed assistant to gpt and user to human.
This llama model was trained 2x faster with
Unsloth and Huggingface's TRL library.
