You can deploy this model and start earning money today!

tiwarinitin94/
therapy-mistral-7b-qlora

Name: therapy-mistral-7b-qlora
Availability: InStock

Views

No views yet

🧠 Mistral Mental Health Therapy v0.1

Model Overview

Mistral Mental Health Therapy v0.1 is a fine-tuned version of Mistral-7B-Instruct-v0.2, trained using QLoRA (4-bit quantization) on a mental health counseling dataset.
It is designed to produce empathetic, supportive, and structured conversational responses similar to a therapist’s tone.

Base model: Mistral-7B-Instruct-v0.2
Fine-tuning method: QLoRA (LoRA + 4-bit quantization)
Framework: Hugging Face Transformers, PEFT, BitsAndBytes
Dataset: fadodr/mental_health_therapy
Language: English
Model version: v0.1
Release date: October 2025

Intended Use

This model is intended for research, education, and experimentation in the domain of mental health dialogue systems and empathy modeling.

✅ Suitable for:

Research on conversational empathy.
Fine-tuning or continued training.
Generating supportive dialogue samples.

⚠️ Not suitable for clinical or real-world therapeutic use.
This model is not a substitute for mental health professionals.

Training Details

Dataset

Used dataset: fadodr/mental_health_therapy
Each sample follows:

Instruction: [Task or situation]
User: [User query or context]
Therapist: [Expected response]

Preprocessing

Combined all fields into a conversational format:

Instruction: <instruction>
User: <input>
Therapist: <output>

Tokenized with Mistral tokenizer (max_length = 512).
Truncated longer sequences.

Fine-Tuning Configuration

Parameter	Value
Method	QLoRA
Rank (`r`)	8
LoRA Alpha	16
Dropout	0.05
Batch Size	1
Gradient Accumulation	8
Learning Rate	2e-4
Max Steps	200
Precision	FP16
Save Steps	25

Hardware

GPU: NVIDIA RTX 4070 (8–16 GB VRAM)
Frameworks: PyTorch, PEFT, Transformers, BitsAndBytes

Example Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_name = "your-username/mistral-mental-health-v0.1"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype=torch.float16)

prompt = "I've been feeling very anxious about my studies. What can I do to calm myself down?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.7, top_p=0.9)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Evaluation (v0.1)

Observations

Generates calm, empathetic, and structured responses.
Occasionally produces repetitive or generic answers.
Handles therapy-style tone effectively within trained context.

Areas for Improvement

Broader dataset for diverse mental health topics.
Validation with real-world dialogue structures.
Add reinforcement or reward-based tuning for sensitivity and safety.

Limitations & Ethics

Not a licensed therapy model.
May produce inaccurate or emotionally insensitive outputs.
Avoid using in real-world therapeutic contexts.

Citation

If you use this model, please cite it as:

Nitin Tiwari. (2025). Mistral Mental Health Therapy v0.1. Hugging Face.  
https://huggingface.co/your-username/mistral-mental-health-v0.1

Author: Nitin Tiwari
Contact: tiwarinitin94@gmail.com
Version: v0.1
Status: Experimental Research Model

tiwarinitin94/therapy-mistral-7b-qlora