You can deploy this model and start earning money today!

metanovus/
ruroberta-ecom-tech-best

Name: ruroberta-ecom-tech-best
Availability: InStock

ai-forever/ruRoberta-large

finetune

apache-2.0

Views

No views yet

Food Delivery Feedback Multi-Label Classification Model

This model was developed for multi-label classification of customer feedback in the food delivery domain. It can identify up to 50 different aspects/issues from user reviews and feedback messages.

Model Description

The model is designed to analyze customer feedback on Russian and automatically categorize it into multiple relevant labels, helping businesses quickly identify and address various aspects of their food delivery service.

Key Features:

Multi-label classification across 50 distinct categories
Specialized for food delivery domain feedback
Handles various aspects including delivery speed, food quality, packaging, customer service, etc.
Supports Russian language feedback processing

Use Cases:

Automated feedback categorization
Customer satisfaction analysis
Service quality monitoring
Issue identification and prioritization
Operational improvement insights

Performance:

Trained on about 3000 customer feedback messages
Achieves Full Match Accuracy on test set
Optimized for real-world food delivery feedback scenarios

Evaluation Metric: Full Match Accuracy

The model is evaluated using a strict full match accuracy metric that requires exact matching between predicted and target labels.

Metric Description:

Type: Exact Match / Perfect Match Accuracy
Range: 0 to 1 (or 0% to 100%)
Format: Labels are represented as space-separated indices (e.g., "0 1 10")

Example:

python

# Correct prediction (100% accuracy):
Target:     "0 1 10"
Prediction: "0 1 10"

# Incorrect predictions (0% accuracy):
Target:     "0 1 10"
Prediction: "0 1"      # Partial match is considered incorrect
Target:     "0 1 10"
Prediction: "0 1 10 2" # Extra label is considered incorrect

Labels Include:

Delivery time issues
Food quality concerns
Order accuracy
Packaging condition
Courier behavior
App/website functionality
Payment issues etc.

-> Full labels see here

Usage

The model can be used for:

Real-time feedback classification
Batch processing of historical feedback
Customer support automation
Service quality analytics

Training

The model was trained as part of an NLP competition focused on improving customer feedback analysis in the food delivery industry. It uses state-of-the-art transformer architecture optimized for multi-label classification tasks.

Limitations

Primarily optimized for Russian language feedback
May require fine-tuning for specific regional contexts
Best suited for food delivery domain specifically

How to use with PyTorch and Transfomers

python

import torch
from transformers import RobertaTokenizer, RobertaForSequenceClassification


model_name = 'ai-forever/ruRoberta-large'
device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')

tokenizer = RobertaTokenizer.from_pretrained(model_name)
class_descriptions = pd.read_csv('files/trends_description.csv')

class BERTClass(torch.nn.Module):
    def __init__(self):
        super(BERTClass, self).__init__()
        self.bert_model = RobertaForSequenceClassification.from_pretrained(
            'metanovus/ruroberta-ecom-tech-best',
            return_dict=True,
            problem_type='multi_label_classification',
            num_labels=50
        )

    def forward(self, input_ids, attn_mask, token_type_ids):
        output = self.bert_model(
            input_ids,
            attention_mask=attn_mask,
            token_type_ids=token_type_ids
        )
        return output.logits

model = BERTClass().to(device)