qwen25-7b-empathy-new-weighted is a PEFT LoRA adapter fine-tuned from
unsloth/Qwen2.5-7B-Instruct-bnb-4bit
for empathetic, emotionally validating conversational responses.
The adapter is intended to be loaded together with the base Qwen2.5 7B Instruct model. It is not a standalone merged model.
Model Details
Model Description
This model was supervised fine-tuned to respond to user messages with emotional validation first, then gentle support. The prompt format includes an explicit detected emotion profile, optional conversation history, and the current user message.
Developed by: ychenrui
Shared by: ychenrui
Model type: PEFT LoRA adapter for causal language modeling
Use this adapter for empathetic conversational response generation, especially when an upstream emotion classifier or application has already identified the user's likely emotional state.
Good fit examples:
Supportive chatbot responses
Empathy-style conversation demos
Research prototypes for affect-aware response generation
Controlled local experiments with emotional support phrasing
Downstream Use
This adapter can be integrated into a larger assistant that supplies:
A short emotion profile
Recent conversation history
The user's current message
Additional safety and escalation logic
Out-of-Scope Use
This model should not be used as a replacement for professional mental health care, emergency support, medical advice, legal advice, diagnosis, or crisis intervention. It should not be used to manipulate users, impersonate clinicians, make high-stakes decisions, or encourage dependency on an AI system.
For crisis or self-harm contexts, applications should use dedicated safety handling and route users to immediate support. In the United States, users in crisis can call or text 988.
Bias, Risks, and Limitations
The model can still hallucinate, misunderstand emotional context, over-validate harmful beliefs, miss crisis signals, or produce generic comfort. It may reflect biases present in the base model and in public conversational datasets. It was trained for supportive wording, not clinical reasoning.
Known limitations:
No formal clinical validation
No claim of therapeutic efficacy
Limited evaluation beyond validation loss and local smoke tests
May perform worse outside English or outside everyday support conversations
May be sensitive to prompt format changes
Recommendations
Use this model with application-level safeguards, human review for sensitive deployments, and separate crisis detection. Downstream systems should clearly disclose that responses are AI-generated and should not be treated as professional advice.
1from unsloth import FastLanguageModel
2import torch
34BASE_MODEL ="unsloth/Qwen2.5-7B-Instruct-bnb-4bit"5ADAPTER_MODEL ="YOUR_USERNAME/qwen25-7b-empathy-new-weighted"67SYSTEM_PROMPT =(8"You are a warm and empathetic AI assistant. "9"Always respond with emotional support first. "10"Acknowledge and validate the user's feelings before asking questions. "11"Keep responses 2-4 sentences."12)1314defbuild_user_content(emotion, message, history=None):15 history = history or[]16 history_text ="\n".join(17f"{turn['role']}: {turn['content']}"for turn in history
18)19 history_block =f"Conversation history:\n{history_text}\n\n"if history_text else""20return(21f"Detected emotion profile: {emotion}\n\n"22+ history_block
23+f"User message:\n<user>{message}</user>\n\n"24+"Respond with emotional validation first, then gentle support. "25"If multiple emotions are present, acknowledge the main emotion "26"and reflect the others naturally."27)2829model, tokenizer = FastLanguageModel.from_pretrained(30 model_name=BASE_MODEL,31 max_seq_length=2048,32 dtype=torch.bfloat16,33 load_in_4bit=True,34)35model.load_adapter(ADAPTER_MODEL)36FastLanguageModel.for_inference(model)3738messages =[39{"role":"system","content": SYSTEM_PROMPT},40{41"role":"user",42"content": build_user_content(43"sadness 0.46, worry 0.35, helplessness 0.11",44"I just found out my dog is sick.",45),46},47]4849prompt = tokenizer.apply_chat_template(50 messages,51 tokenize=False,52 add_generation_prompt=True,53)54inputs = tokenizer([prompt], return_tensors="pt").to(model.device)5556outputs = model.generate(57**inputs,58 max_new_tokens=260,59 temperature=0.7,60 top_p=0.9,61 do_sample=True,62 pad_token_id=tokenizer.eos_token_id,63)6465response = tokenizer.decode(66 outputs[0][inputs["input_ids"].shape[-1]:],67 skip_special_tokens=True,68)69print(response.strip())
If you upload this adapter under a different Hugging Face username or organization, replace YOUR_USERNAME/qwen25-7b-empathy-new-weighted with that repository ID.
Training Details
Training Data
The local processed dataset contains 15,000 examples:
Split
Examples
Train
14,250
Validation
450
Test
300
Source mix used in the processed data:
Source
Examples
EmpatheticDialogues
8,250
ESConv
3,000
DailyDialog
1,500
CounselChat
1,500
Hand-written safety examples
750
The training data was converted into chat-style supervised fine-tuning records. Each record contains a system message, a user message with the detected emotion profile and optional history, and an assistant response.
Training Procedure
The adapter was trained with QLoRA using Unsloth and TRL supervised fine-tuning.
Preprocessing
The preprocessing pipeline:
Normalized user and assistant text
Converted multiple conversational datasets into one chat format
Preserved recent conversation history when available
Filtered very short user and assistant turns
Added hand-written safety examples for crisis-response anchoring
Hardware and wall-clock training time were not recorded in the model card metadata.
Evaluation
Testing Data, Factors & Metrics
Testing Data
Evaluation used the held-out validation split generated by the local preprocessing pipeline. A separate 300-example test split was also produced, but no formal benchmark results are reported here.
Factors
No disaggregated evaluation by demographic group, emotion label, dataset source, or crisis category has been completed.
Metrics
Validation loss was tracked during training. Local qualitative smoke tests were also used for empathy-style responses, but they should not be interpreted as a formal safety or clinical evaluation.
Results
Step
Validation loss
250
2.1326
500
2.0806
750
2.0574
1000
2.0429
1250
2.0347
1500
2.0259
1750
2.0206
2000
2.0604
2250
2.0632
2500
2.0655
2673
2.0661
Best observed validation loss in the recorded training log was 2.0206 at step 1750. The final adapter was saved after 3 epochs at step 2673.
Summary
The model was optimized for warmer, more emotionally validating responses. It still requires robust downstream safety checks for crisis, medical, legal, or other high-stakes use.
Environmental Impact
Carbon emissions were not measured for this run.
Hardware Type: Not recorded; training config was designed for a single RTX 4090-class GPU
Hours used: Not recorded
Cloud Provider: Not recorded
Compute Region: Not recorded
Carbon Emitted: Not measured
Technical Specifications
Model Architecture and Objective
This is a LoRA adapter for Qwen2.5 7B Instruct. It was trained with a causal language modeling objective on chat-formatted supervised examples. The adapter targets the attention and MLP projection modules.
Compute Infrastructure
Hardware
Not recorded in the model metadata.
Software
PEFT: 0.19.1
TRL: 0.24.0
Transformers: 5.5.0
PyTorch: 2.10.0+cu126
Datasets: 4.3.0
Tokenizers: 0.22.2
Unsloth
Citation
If you use this adapter, cite the base model and the training libraries where appropriate.
Qwen2.5:
bibtex
1@misc{qwen2.5,
2 title = {Qwen2.5: A Party of Foundation Models},
3 url = {https://qwenlm.github.io/blog/qwen2.5/},
4 author = {Qwen Team},
5 month = {September},
6 year = {2024}
7}
TRL:
bibtex
1@misc{vonwerra2022trl,
2 title = {{TRL: Transformer Reinforcement Learning}},
3 author = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouedec},
4 year = {2020},
5 publisher = {GitHub},
6 howpublished = {\url{https://github.com/huggingface/trl}}
7}
