SinMood: Emotional Distress Detection in Singlish Social Media
Project Overview
SinMood is a machine learning project aimed at developing a multi-class classifier to detect emotional states (Normal, Sad, Depressed, Anxious) in romanized Singlish social media posts. This model is designed to assist in identifying early signs of emotional distress in a linguistically unique context.
Model Architecture
The SinMood model employs a hybrid feature representation combined with an ensemble classifier:
- XLM-RoBERTa Contextual Embeddings: Utilizes
xlm-roberta-base to extract 768-dimensional contextual embeddings for each post, capturing semantic meaning and context.
- Character N-grams: TF-IDF features for character n-grams (range 3-5, max 15K features) to handle spelling variations common in informal Singlish.
- Word N-grams: TF-IDF features for word n-grams (range 1-2, max 5K features) to capture common phrases and word associations.
- Domain-specific Keyword Features: A set of 24 handcrafted features based on emotion keywords, crisis markers, and linguistic patterns relevant to emotional distress in Singlish.
These diverse feature sets are concatenated to form a rich representation for each social media post.
Ensemble Classifier
The final model is an ensemble classifier combining three powerful algorithms through soft voting (averaging probabilities):
- GradientBoostingClassifier
- XGBoostClassifier
- RandomForestClassifier
Class Weighting: Crisis classes (Depressed, Anxious) are given higher weights during training to prioritize their detection, reflecting the critical nature of these emotional states.
Data and Preprocessing
- Dataset: 1,093 manually annotated Singlish posts (874 for training, 219 for testing).
- Preprocessing: Includes URL removal, @mention removal, emoji-to-text conversion, character repetition normalization (preserving intensity), punctuation normalization (preserving emphasis), and all-caps marking. This approach was chosen to preserve crucial emotional signals.
- Class Balancing: SMOTE (Synthetic Minority Over-sampling Technique) was applied to the training data to address class imbalance and improve the model's ability to detect minority classes.
Performance
Overall Metrics (4-class classification):
- Accuracy: 62.6%
- F1-Score (weighted): 0.618
- F2-Score (recall-prioritized weighted): 0.620
The model demonstrates a good baseline for a challenging multi-class classification task in a low-resource, code-mixed language.
How to Use
To use this model for inference, you can leverage the inference.py script provided in this repository. The init() function loads all necessary model artifacts (ensemble model, vectorizers, and XLM-RoBERTa components), and the inference() function takes a dictionary with a "text" key and returns the predicted emotion and probabilities.
Example Usage (programmatic)
1from huggingface_hub import HfApi, snapshot_download
2import os
3import sys
4import json
5
6# Assuming inference.py, model files are locally available or downloaded
7# For Hugging Face Spaces, these are automatically handled.
8
9# --- Minimal setup to run inference locally after downloading repo contents ---
10# from inference import init, inference # If inference.py is in current directory
11
12# If running directly from Hub, use the API
13# api = HfApi()
14# model_path = snapshot_download(repo_id="mzmuzni/sinmood-emotion-detection")
15# sys.path.append(model_path)
16# from inference import init, inference
17
18# init() # Initialize the model once
19
20# text_input = {"text": "mata jeewithe nethi krgnna hithenwa hamadema epa wela"}
21# result = inference(text_input)
22# print(result)
23# Output example:
24# {
25# 'prediction': 'Depressed',
26# 'confidence': 0.870,
27# 'probabilities': {'Depressed': 0.870, 'Sad': 0.057, 'Normal': 0.044, 'Anxious': 0.029}
28# }
29
30# This model is primarily for research and educational purposes.
31# Always consult with mental health professionals for actual diagnosis and treatment.
Files in this Repository
sinmood_model_compatible.pkl: The pickled ensemble classifier.char_vectorizer_compatible.pkl: The pickled TF-IDF vectorizer for character n-grams.word_vectorizer_compatible.pkl: The pickled TF-IDF vectorizer for word n-grams.model_metadata.json: Metadata about the model, features, and performance.confusion_matrix.png: Visualization of the model's confusion matrix.inference.py: Python script for performing inference with the loaded model.requirements.txt: List of Python dependencies required to run the model.X_train_xlmr.npy, X_test_xlmr.npy: Pre-computed XLM-RoBERTa embeddings for train and test sets (for reproducibility/reference).
Acknowledgements
This project was developed by M.M.M. Maznavi (20212086) at the Informatics Institute of Technology & University of Westminster. Special thanks to the contributors of the XLM-RoBERTa model and the open-source community.
Disclaimer: This model is for research and experimental purposes only. It is not a substitute for professional medical advice, diagnosis, or treatment. If you or someone you know is experiencing emotional distress, please seek help from a qualified mental health professional or crisis hotline.
