This llama model was trained 2x faster with
Unsloth and Huggingface's TRL library.
This is a llm fine-tuned off of the dialogue of Jill Stingray from the game
Va11-Hall-A. It is based off of llama3.2:3b (which is linked below). While it does work, this llm will frequently think to itself (like how Jill often does) or may even refuse to respond (Jill tends to do that sometimes).
Overall, is it a good model? Meh. With the right system prompt, it's actually kinda nice. But if it's not role-playing as Jill, I wouldn't say so.
But does it work? Yea. And for my first fine-tuning, honestly it's better than I expected.
I had many issues with Unsloth. Training actually went smoothly, but I had issues downloading the base model (had to manually download it and load it locally), as well as saving as a gguf (which I had to resolve using llama.cpp cli manually). Anyway, I modified the instructions from their free google colab notebooks, then ran it as a jupyter notebook on my local T550 Nvidia laptop GPU.
Would I still recommend unsloth? Honestly, yes. It was the only library I used that actually worked out in the end. I bet running the notebooks on Google Colab would lead to less errors simply because its more reproducible.
-
Ran on Python3.10, EndeavourOS (linux)
-
2116.7746 seconds used for training.
-
35.28 minutes used for training.
-
Peak reserved memory = 3.33 GB.
-
Peak reserved memory for training = 0.0 GB.
-
Peak reserved memory % of max memory = 91.685 %.
-
Peak reserved memory for training % of max memory = 0.0 %.
-
Torch Version: 2.7.0+cu128
-
CUDA Available: True
-
CUDA Device: NVIDIA T550 Laptop GPU
