Model Card for Fine-Tuned LLaMA Empathy

Model Summary

Model Details

Model Description

• Developed by: AI Medical in collaboration with Ruslanmv.com
• Funded by:
• Shared by: AI Medical
• Model type: Fine-tuned Meta-Llama-3.1-8B-Instruct
• Language(s) (NLP): English
• License: Creative Commons Attribution 4.0 International License (CC BY 4.0)
• Fine-tuned from model: meta-llama/Meta-Llama-3.1-8B-Instruct

Model Sources

• Repository: Hugging Face Model Repository

Uses

Direct Use

• Psychology & Therapy: Assisting professionals in understanding and responding empathetically to patient emotions.
• Conflict Resolution: Helping mediators decode emotional expressions and address underlying needs.
• Human-Computer Interaction: Enhancing chatbots and virtual assistants with emotionally aware responses.
• Social Media Moderation: Reducing toxicity and improving online discourse through need-based responses.
• Education: Supporting emotional intelligence training and communication skill development.

Downstream Use

• Fine-tuning for specialized applications in mental health, conflict resolution, or AI-driven assistance.
• Integration into virtual therapists, mental health applications, and online support systems.

Out-of-Scope Use

• Not a substitute for professional psychological evaluation or medical treatment.
• Not suitable for high-risk applications requiring absolute accuracy in emotional interpretation.

Bias, Risks, and Limitations

• Bias: As with any NLP model, biases may exist due to the dataset and training methodology. LLaMA models, in particular, have shown biases.
• Risk of Misinterpretation: Emotional expressions are subjective and may be misclassified in complex scenarios.
• Generalization Limitations: May not fully capture cultural and contextual variations in emotional expressions.

Recommendations

How to Get Started with the Model

Training Details

Training Data

• Dataset: Annotated dataset mapping evaluative expressions to emotions and needs.
• Annotations: 1,500+ labeled examples linking expressions to emotional states and corresponding needs.

Training Procedure

Preprocessing

• Tokenized using Hugging Face transformers library.
• Augmented with synonym variations and paraphrased sentences.

Training Hyperparameters

• Training regime: Mixed precision training using LoRA.
• Batch size: 32
• Learning rate: 2e-5
• Training steps: 1k
• Hardware: 1x A100 GPU using DeepSpeed ZeRO-3

Evaluation

Testing Data, Factors & Metrics

Testing Data

• Held-out dataset containing unseen evaluative expressions.

Factors

• Performance across different emotional expression categories.
• Sensitivity to nuanced phrasing and variations.

Metrics

• Accuracy: Measures correct classification of emotions and needs.
• Precision & Recall: Evaluates the balance between capturing true emotions and avoiding false positives.
• F1-Score: Measures the balance between precision and recall.

Results

• Accuracy: 89.5%
• F1-Score: 87.2%
• Latency: <500ms response time

Environmental Impact

• Hardware Type: A100 GPUs
• Training Time: hours
• Carbon Emitted: Estimated using Machine Learning Impact calculator.

Technical Specifications

Model Architecture and Objective

• Base Model: meta-llama/Meta-Llama-3.1-8B-Instruct
• Fine-tuned using LoRA for parameter-efficient training. Key LoRA parameters: r=8, lora_alpha=16, lora_dropout=0.2, target_modules=["v_proj", "q_proj"]

Compute Infrastructure

• Hardware: AWS spot instances (1x A100 GPUs)
• Software: Hugging Face transformers, PEFT, PyTorch

Citation

More Information

• Model Card Authors: AI Medical Team, ruslanmv.com
• Framework Versions: PEFT 0.14.0