You can deploy this model and start earning money today!

help2opensource/
Qwen3-4B-Instruct-2507_mental_health_cbt

Name: Qwen3-4B-Instruct-2507_mental_health_cbt
Availability: InStock

Qwen/Qwen3-4B-Instruct-2507

Views

No views yet

🧠 Model Card: help2opensource/Qwen3-4B-Instruct-2507_mental_health_cbt

🔍 Overview

This model is a 4-bit quantized version of the Qwen/Qwen3-4B-Instruct large language model fine-tuned using LoRA (Low-Rank Adaptation) to generate CBT (Cognitive Behavioral Therapy) responses for mental health applications. The model follows the prompt format:
### Dialogue:\n{dialogue}\n### Use the CBT technique: {technique}\n### Plan: {plan}\n### Assistant:

It is designed to assist in generating structured, evidence-based therapeutic interventions for individuals working with cognitive-behavioral techniques.

🎯 Use Cases

Mental Health Support: Provide users with CBT strategies (e.g., cognitive restructuring, behavioral activation).
Therapeutic Planning: Generate actionable plans based on patient dialogues.
Clinical Training: Simulate therapist responses for training purposes.

📚 Training Data

The model is trained on the LangAGI-Lab/cactus dataset, which includes:

Dialogues: Real-world conversations between patients and therapists.
CBT Techniques: Predefined techniques (e.g., "challenging negative thoughts").
Plans: Step-by-step therapeutic plans to address specific issues.

The dataset is split into training and test sets (90/10). Each example includes:

python

{
    "dialogue": str,          # Patient-Therapist conversation
    "cbt_technique": str,     # CBT technique to apply
    "cbt_plan": str           # Step-by-step therapeutic plan
}

🧠 Model Architecture

Base Model: Qwen/Qwen3-4B-Instruct (a 4-billion parameter causal language model).
Quantization: 4-bit quantized with bitsandbytes for reduced memory usage.
LoRA Configuration: Fine-tuned using LoRA with the following parameters:
- Rank (r): 8
- Alpha (lora_alpha): 32
- Target Layers: q_proj, v_proj, k_proj, o_proj
- Dropout Rate: 0.1

🛠️ Training Process

Hardware: GPU with mixed-precision (FP16).
Batch Size: per_device_train_batch_size=5, gradient_accumulation_steps=5.
Optimization: AdamW optimizer, learning rate 2e-5, weight decay 0.01.
Early Stopping: Not included in the code (can be added via EarlyStoppingCallback).
Evaluation Metrics:
- BLEU (for n-gram overlap)
- ROUGE-L (for long-text similarity)

📌 Known Limitations

The model may struggle with highly specialized clinical cases not covered in the training data.
Generated CBT responses should be reviewed by licensed professionals before use.

🛡️ Safety & Ethics

This model is intended for educational and research purposes only.
Avoid using it for real-world therapeutic decisions without human oversight.
Ensure compliance with local laws and ethical guidelines for mental health applications.