Amod/mental_health_counseling_conversations
library_name: transformers
license: mit
Model Card Summary
This model is a fine-tuned version of gemma-2b-it for mental health counseling conversations.
It was fine-tuned on the Amod/Mental Health Counseling Conversations dataset,
which contains dialogues related to mental health counseling.
Model Details
Model Description
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
Developed by: Miguel Flores
Model type: Causal Language Model (based on transformers)
Language(s) (NLP): English
License: MIT License
Finetuned from model gemma-2b-it: gemma-2b-it, which is a base model fine-tuned for mental health-related queries.
Use Cases
Direct Use
This model is fine-tuned for generating responses related to mental health counseling tasks.
It can be used for providing suggestions, conversation starters, or follow-ups in mental health scenarios.
Downstream Use
This model can be adapted for use in more specific counseling-related tasks,
or in applications where generating mental health-related dialogue is necessary.
Out-of-Scope Use
The model is not intended to replace professional counseling.
It should not be used for real-time crisis management or any situation
requiring direct human intervention. Use in highly critical or urgent care situations is out of scope.
Bias, Risks, and Limitations
The model was trained on mental health-related dialogues, but it may still generate biased or
inappropriate responses. Users should exercise caution when interpreting or acting on the model's outputs,
particularly in sensitive scenarios.
Recommendations
The model should not be used as a replacement for professional mental health practitioners.
Users should carefully evaluate generated responses in the context of their use case.
inputs = tokenizer("How can I manage anxiety better?", return_tensors="pt") outputs = model.generate(**inputs, max_length=200) response = tokenizer.decode(outputs[0], skip_special_tokens=True) print(response)
Training Details
Training Data
The model was trained on the Amod/Mental Health Counseling Conversations dataset,
which consists of mental health dialogues focused on counseling situations.
Training Procedure
The model was fine-tuned using LoRA (Low-Rank Adaptation) with the following hyperparameters:
The model was evaluated using a split from the training data,
specifically a 10% test split of the original training dataset.
Metrics
The following metrics were used during the training and evaluation process:
Training Loss: The training loss was tracked during training to monitor how well the model was learning from the data. It decreased throughout the epochs.
Semantic Similarity: Semantic similarity was employed as the primary metric to assess the model’s ability to generate contextually relevant and meaningful responses. Since the dataset involves conversational context, particularly in the sensitive area of mental health counseling, it was crucial to evaluate how well the model understands and retains the intent and meaning behind the input rather than merely focusing on fluency or token-level prediction.
Perplexity: Perplexity was used as a metric to evaluate the model's ability to generate coherent and fluent text responses. The model was evaluated on a subset of the test data, and both non-finetuned and finetuned perplexities were compared.
