NeuroFeel is a lightweight NLP model built on NeuroBERT, fine-tuned for short-text emotion detection on edge and IoT devices. With a quantized size of ~25MB and ~7M parameters, it classifies text into 13 nuanced emotional categories (e.g., Happiness, Sadness, Anger, Love) with high precision. Optimized for low-latency and offline operation, NeuroFeel is perfect for privacy-focused applications like chatbots, social media sentiment analysis, mental health monitoring, and contextual AI in resource-constrained environments such as wearables, smart home devices, and mobile apps.
Parameters: ~7M, significantly fewer than standard BERT models.
Quantization: INT8 quantization for minimal memory usage and fast inference.
Vocabulary Size: 30,522 tokens, compatible with NeuroBERT’s tokenizer.
Max Sequence Length: 64 tokens, ideal for short-text inputs like social media posts or chatbot messages.
This architecture ensures NeuroFeel delivers high accuracy for emotion detection while maintaining compatibility with resource-constrained devices like Raspberry Pi, ESP32, or mobile NPUs.
Installation
Install the required dependencies:
pip install transformers torch
Ensure your environment supports Python 3.6+ and has ~25MB of storage for model weights.
1Text: i love you
2Predicted Emotion: Love ❤️
3Confidence: 85.63%
Note: Fine-tune the model for domain-specific tasks to boost accuracy.
NeuroFeel excels in classifying a wide range of emotions in short texts, particularly in IoT, social media, and mental health contexts. Fine-tuning enhances performance on subtle emotions like Sarcasm or Shame.
Evaluation Metrics
Metric
Value (Approx.)
✅ Accuracy
~92–96% on 13-class emotion tasks
🎯 F1 Score
Balanced for multi-class classification
⚡ Latency
<40ms on Raspberry Pi 4
📏 Recall
Competitive for lightweight models
Note: Metrics depend on hardware and fine-tuning. Test on your target device for precise results.
Use Cases
NeuroFeel is tailored for edge and IoT scenarios requiring real-time emotion detection for short texts. Key applications include:
Chatbot Emotion Understanding: Detect user emotions, e.g., “I love you” (predicts “Love ❤️”) to tailor responses.
Social Media Sentiment Tagging: Analyze posts, e.g., “This is disgusting!” (predicts “Disgust 🤢”) for moderation or trend analysis.
Mental Health Context Detection: Monitor mood, e.g., “I feel so alone” (predicts “Sadness 😢”) for wellness apps or crisis alerts.
Smart Replies and Reactions: Suggest replies, e.g., “I’m so happy!” (predicts “Happiness 😄”) for positive emojis or animations.
Emotional Tone Analysis: Adjust IoT settings, e.g., “I’m terrified!” (predicts “Fear 😱”) to dim lights or play calming music.
Voice Assistants: Local emotion-aware parsing, e.g., “Why does it break?” (predicts “Anger 😠”) to prioritize fixes.
Toy Robotics: Emotion-driven interactions, e.g., “I really want that!” (predicts “Desire 🔥”) for engaging animations.
Smart Home Automation: Contextual responses, e.g., “I’m so tired” (predicts “Sadness 😢”) to adjust lighting or music.
Customer Support Bots: Detect frustration, e.g., “This is ridiculous!” (predicts “Anger 😠”) to escalate to human agents.
Educational Tools: Analyze student feedback, e.g., “I don’t get it” (predicts “Confusion 😕”) to offer tailored explanations.
Hardware Requirements
Processors: CPUs, mobile NPUs, or microcontrollers (e.g., ESP32-S3, Raspberry Pi 4, Snapdragon NPUs)
Storage: ~25MB for model weights (quantized, Safetensors format)
Memory: ~70MB RAM for inference
Environment: Offline or low-connectivity settings
Quantization ensures efficient memory usage, making NeuroFeel ideal for resource-constrained devices.
Training Details
NeuroFeel was fine-tuned on a custom emotion dataset augmented with ChatGPT-generated data to enhance diversity and robustness. Key training details:
Dataset:
Custom Emotion Dataset: ~10,000 labeled short-text samples covering 13 emotions (e.g., Happiness, Sadness, Love). Sourced from social media posts, IoT user feedback, and chatbot interactions.
ChatGPT-Augmented Data: Synthetic samples generated to balance underrepresented emotions (e.g., Sarcasm, Shame) and improve generalization.
Preprocessing: Lowercasing, emoji removal, and tokenization with NeuroBERT’s tokenizer (max length: 64 tokens).
Training Process:
Base Model: NeuroBERT, pre-trained on general English text for masked language modeling.
Fine-Tuning: Supervised training for 13-class emotion classification using cross-entropy loss.
Hyperparameters:
Epochs: 5
Batch Size: 16
Learning Rate: 2e-5
Optimizer: AdamW
Scheduler: Linear warmup (10% of steps)
Hardware: Fine-tuned on a single NVIDIA A100 GPU, but inference optimized for edge devices.
Quantization: Post-training INT8 quantization to reduce model size to ~25MB and improve inference speed.
Data Augmentation:
Synonym replacement and back-translation to enhance robustness.
Synthetic negative sampling to improve detection of nuanced emotions like Guilt or Confusion.
Validation:
Split: 80% train, 10% validation, 10% test.
Validation F1 score: ~0.93 across 13 classes.
Fine-tuning on domain-specific data is recommended to optimize performance for specific use cases (e.g., mental health apps or smart home devices).
Fine-Tuning Guide
To adapt NeuroFeel for custom emotion detection tasks:
Prepare Dataset: Collect labeled data with 13 emotion categories.
Fine-Tune with Hugging Face:
python
1import pandas as pd
2from transformers import BertTokenizer, BertForSequenceClassification, Trainer, TrainingArguments
3from sklearn.model_selection import train_test_split
4import torch
5from torch.utils.data import Dataset
67# === 1. Load and preprocess data ===8 dataset_path ='/content/dataset.csv'9 df = pd.read_csv(dataset_path)10# Use the correct original column name 'Label' in dropna11 df = df.dropna(subset=['Label'])# Ensure no missing labels12 df.columns =['text','label']# Normalize column names1314# === 2. Encode labels ===15 labels =sorted(df["label"].unique())16 label_to_id ={label: idx for idx, label inenumerate(labels)}17 id_to_label ={idx: label for label, idx in label_to_id.items()}18 df['label']= df['label'].map(label_to_id)1920# === 3. Train/val split ===21 train_texts, val_texts, train_labels, val_labels = train_test_split(22 df['text'].tolist(), df['label'].tolist(), test_size=0.2, random_state=4223)2425# === 4. Tokenizer ===26 tokenizer = BertTokenizer.from_pretrained("boltuix/NeuroBERT-Pro")2728# === 5. Dataset class ===29classSentimentDataset(Dataset):30def__init__(self, texts, labels, tokenizer, max_length=128):31 self.texts = texts
32 self.labels = labels
33 self.tokenizer = tokenizer
34 self.max_length = max_length
3536def__len__(self):37returnlen(self.texts)3839def__getitem__(self, idx):40 encoding = self.tokenizer(41 self.texts[idx],42 padding='max_length',43 truncation=True,44 max_length=self.max_length,45 return_tensors='pt'46)47return{48'input_ids': encoding['input_ids'].squeeze(0),49'attention_mask': encoding['attention_mask'].squeeze(0),50'labels': torch.tensor(self.labels[idx], dtype=torch.long)51}5253# === 6. Load datasets ===54 train_dataset = SentimentDataset(train_texts, train_labels, tokenizer)55 val_dataset = SentimentDataset(val_texts, val_labels, tokenizer)5657# === 7. Load model ===58 model = BertForSequenceClassification.from_pretrained(59"boltuix/NeuroBERT-Pro",60 num_labels=len(label_to_id)61)6263# Optional: Ensure tensor layout is contiguous64for param in model.parameters():65 param.data = param.data.contiguous()6667# === 8. Training arguments ===68 training_args = TrainingArguments(69 output_dir='./results',70 run_name="NeuroFeel",71 num_train_epochs=5,72 per_device_train_batch_size=16,73 per_device_eval_batch_size=16,74 warmup_steps=500,75 weight_decay=0.01,76 logging_dir='./logs',77 logging_steps=10,78 eval_strategy="epoch",79 report_to="none"80)8182# === 9. Trainer setup ===83 trainer = Trainer(84 model=model,85 args=training_args,86 train_dataset=train_dataset,87 eval_dataset=val_dataset
88)8990# === 10. Train and evaluate ===91 trainer.train()92 trainer.evaluate()9394# === 11. Save model and label mappings ===95 model.config.label2id = label_to_id
96 model.config.id2label = id_to_label
97 model.config.num_labels =len(label_to_id)9899 model.save_pretrained("./neuro-feel")100 tokenizer.save_pretrained("./neuro-feel")101102print("✅ Training complete. Model and tokenizer saved to ./neuro-feel")
Deploy: Export to ONNX or TensorFlow Lite for edge devices.
Comparison to Other Models
Model
Parameters
Size
Edge/IoT Focus
Tasks Supported
NeuroFeel
~7M
~25MB
High
Emotion Detection, Classification
NeuroBERT
~7M
~30MB
High
MLM, NER, Classification
BERT-Lite
~2M
~10MB
High
MLM, NER, Classification
DistilBERT
~66M
~200MB
Moderate
MLM, NER, Classification, Sentiment
NeuroFeel is specialized for 13-class emotion detection, offering superior performance for short-text sentiment analysis on edge devices compared to general-purpose models like NeuroBERT, while being far more efficient than DistilBERT.
.jpg)
