PersonaPath Emotion Detection (MobileNetV2 - FER2013)

Overview

This repository contains a fine-tuned MobileNetV2 model trained on the FER-2013 dataset for facial emotion classification.

Task

• Problem Type: Multi-class classification
• Input: Facial images (224 × 224 RGB)
• Output: 7 emotion classes

• Angry
• Disgust
• Fear
• Happy
• Sad
• Surprise
• Neutral

Model Details

• Base Model: MobileNetV2 (pretrained on ImageNet)
• Framework: TensorFlow / Keras
• Strategy: Transfer Learning + Fine-Tuning

Fine-Tuning Approach

• Initial training with frozen base layers (~52% validation accuracy)
• Progressive unfreezing of top layers:
  • Top 30 layers → ~60.66% accuracy
  • Top 50 layers → further improvement (ongoing experimentation)
• Lower layers retained for general feature extraction
• Higher layers adapted for emotion-specific patterns

Dataset

• FER-2013
• Source: Hugging Face
• 35,000+ grayscale facial images

Preprocessing

• Resized to 224×224
• Converted to RGB
• Data augmentation:
  • Horizontal flip
  • Rotation (±10°)
  • Zoom (±10%)

Challenge

• Severe class imbalance (e.g., Disgust class underrepresented)
• Handled using class weighting

Training Details

• Loss: Sparse Categorical Crossentropy
• Optimizer: Adam
  • Base training: lr = 1e-4
  • Fine-tuning: lr = 1e-5
• Callbacks:
  • EarlyStopping
  • ReduceLROnPlateau
  • ModelCheckpoint

Results

Usage

Role in PersonaPath

• Extract frames from input video at 3 FPS
• Perform emotion classification on each frame
• Aggregate predictions across frames to estimate an overall confidence score

Limitations

• Moderate performance (~60% validation accuracy), affected by:
  • Class imbalance in FER-2013
  • Low-resolution facial images
• Generalization may drop in real-world conditions (lighting, pose variations)
• Emotion predictions are used as a proxy for confidence, not a direct measurement

Future Improvements

• Explore deeper fine-tuning or alternative architectures
• Train on higher-quality datasets (e.g., RAF-DB, AffectNet)
• Integrate with audio-based features (e.g., LSTM on MFCC) for multi-modal confidence estimation