qwen25-7b-empathy

Name: qwen25-7b-empathy
Rating: 2.5 (8 reviews)

qwen25-7b-empathy is a LoRA adapter fine-tuned from unsloth/Qwen2.5-7B-Instruct-bnb-4bit for emotionally supportive conversation. The goal is to make Qwen2.5-7B respond less like a generic daily chatbot and more like an empathetic support assistant: validating feelings first, reflecting the user's emotional state, and offering gentle next steps when appropriate.

This repository contains the PEFT adapter weights, tokenizer files, chat template, and model card. It is not a merged full-weight model.

Model Details

Developed by: JamieYCR
Model type: Qwen2.5-7B Instruct LoRA adapter
Base model: unsloth/Qwen2.5-7B-Instruct-bnb-4bit
Library: PEFT / Transformers / TRL / Unsloth
Language: English
Task: text generation, emotional-support conversation
Adapter size: about 154 MB

Intended Use

This adapter is intended for research, demos, and experiments around empathetic dialogue generation. It can be used for:

emotionally supportive chatbot responses
empathy-focused response generation
comparison with a base Qwen2.5 instruction model
classroom or portfolio demonstrations of LoRA fine-tuning

It should not be used as a replacement for professional mental health care, medical advice, legal advice, or emergency support. For crisis or self-harm situations, route users to qualified human help and local emergency resources.

Training Data

The final training mix emphasized empathy-focused and harder emotional-support examples rather than ordinary small talk. The mix included:

EmpatheticDialogues
ESConv emotional-support conversations
CounselChat-style counseling QA examples
locally generated hard-case domain examples from difficult emotional scenarios
handwritten empathy-boost examples
safety-oriented examples

Daily small-talk data was removed from the boosted run so the model would learn deeper validation and emotional reflection instead of casual chatbot replies.

Training Procedure

The adapter was trained with supervised fine-tuning using QLoRA/LoRA on Qwen2.5-7B Instruct.

Key settings from the final boosted run:

LoRA rank: 16
LoRA alpha: 16
LoRA dropout: 0.0
Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Training epochs: 3
Learning rate: 7.5e-5
PEFT version: 0.19.1
Final local adapter path: outputs/qwen25-7b-empathy-boosted/final_lora

Example Behavior

Prompt:

text

Emotion: sadness 0.62, disappointment 0.28
Message: I keep telling everyone I'm just tired, but honestly I don't remember the last time I felt excited for anything.

Example response:

That sounds really painful, like the absence of hope feels harsh and physical. It makes sense that you feel discouraged; you deserve support around how hard this has been, not just pressure to feel better quickly.

How to Load

python

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer
import torch

adapter_id = "JamieYCR/qwen25-7b-empathy"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
model = AutoPeftModelForCausalLM.from_pretrained(
    adapter_id,
    device_map="auto",
    torch_dtype=torch.bfloat16,
)

messages = [
    {
        "role": "system",
        "content": "You are a deeply empathetic emotional-support assistant. Validate feelings first, reflect the user's emotional state, and avoid generic small talk.",
    },
    {
        "role": "user",
        "content": "I keep telling everyone I'm just tired, but honestly I don't remember the last time I felt excited for anything.",
    },
]

inputs = tokenizer.apply_chat_template(
    messages,
    return_tensors="pt",
    add_generation_prompt=True,
).to(model.device)

outputs = model.generate(
    inputs,
    max_new_tokens=220,
    temperature=0.7,
    top_p=0.9,
)

print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))

Limitations and Risks

The model can sound emotionally supportive, but it does not understand a user's full real-world context.
It may over-validate, miss clinical risk, or give advice that is too general.
It may produce unsafe or incomplete answers in crisis situations.
It inherits limitations and biases from the base model and training data.
It should be evaluated carefully before use in any user-facing support setting.

Recommended Safeguards

Add crisis detection and human escalation for self-harm, abuse, violence, or emergency situations.
Make it clear to users that the system is not a therapist or medical professional.
Test responses across different emotional states and user backgrounds.
Monitor for generic, dismissive, overly directive, or unsafe responses.

Citation

If you use this adapter, cite the base Qwen2.5 model and the datasets used in your training/evaluation notes where applicable.

Contact

For questions about this fine-tuned adapter, contact the repository owner on Hugging Face: JamieYCR.