GEMMA3N Mental Health Fine-tuned Model
- Developed by: belal212
- License: Apache 2.0
- Fine-tuned from base model:
unsloth/gemma-3n-e2b-it-unsloth-bnb-4bit
- Frameworks: Unsloth, HuggingFace Transformers, TRL
Base Model: unsloth/gemma-3n-e2b-it-unsloth-bnb-4bit
Developed by: belal212
License: Apache-2.0
Language: English
Frameworks Used: Unsloth, Hugging Face Transformers, TRL
Model Overview
This model is a fine-tuned variant of Gemma3n, specifically adapted for mental health support conversations. Training was optimized using Unsloth's accelerated fine-tuning backend, enabling faster and memory-efficient training. The model incorporates LoRA for parameter-efficient fine-tuning.
Training Data
The model was trained on a combined dataset of mental health conversations sourced from four public Hugging Face datasets:
- ShivomH/Mental-Health-Conversations
- YvvonM/mental_health_data
- usham/mental-health-companion-new
- Harshallama/mental_health_alpaca_format
All datasets were unified into a consistent instruction-based format. The combined dataset contains approximately 4.37 million samples and was saved to a single file named therapist_dataset.csv.
Fine-Tuning Configuration
The model was fine-tuned using LoRA, with adaptation applied to the language layers, attention modules, and MLP modules. Vision layers were not fine-tuned. LoRA configuration used moderate rank and dropout, with no bias term adaptation. The random seed was set for reproducibility.
Supervised Fine-Tuning (SFT) was conducted using TRL's SFTTrainer. The dataset field used for text input was formatted_text. Training used a small per-device batch size with gradient accumulation, an 8-bit AdamW optimizer, linear learning rate scheduling, and evaluation every few hundred steps. The training process saved checkpoints periodically and used a fixed seed.
Training Environment
- CUDA-enabled GPU: NVIDIA RTX A6000
- Frameworks: PyTorch, Hugging Face Datasets, Transformers, TRL, Unsloth
Intended Use
This model is designed for use in:
- Mental health chatbots
- Supportive dialogue agents
- Psychological well-being applications
- Research and experimentation in empathetic NLP
⚠️ Disclaimer
This model is for research and educational purposes only. It is not a substitute for professional mental health care or medical treatment.
.
