The model has been fine-tuned using personal resources and the help of RunPod and Axolotl. The process took around 15 hours for the training to complete 1.5 epochs given the size of the dataset (950.000 rows) and the size of the model itself with a batch size of 8, alongside some extra hours to merge the adapter to the base model. Given the lack of resources from my campus, this was all done throughout 7 months where I had to jump through multiple services and methods.
Information about base model:
Llama 3.2 Model Family: Token counts refer to pretraining data only. All model versions use Grouped-Query Attention (GQA) for improved inference scalability.
Finetuned from model : meta-llama/Llama-3.2-3B-Instruct
Completion Date: Sept 9, 2025
Intended use
The purpose of this model as the name suggest is to provide emotional support as a therapist would. In this day and age, mental health issues are rampant, An LLM can provide more consistent support. With tech like this, more lives can be saved, or at least maintained until more help is available.
