A multi-label text classification model fine-tuned on RoBERTa-base Wechsel Swahili for classifying Swahili text into 8 predefined topics. The model can identify multiple applicable topics for a given text, providing confidence scores for each topic.

Developed by: NeboTech
Model type: Transformer-based (RoBERTa)
Language(s): Swahili (Kiswahili)
License: Apache 2.0
Finetuned from: RoBERTa-base Wechsel Swahili
Model version: v2.0 (Multi-label Classification)

Model Architecture

Base Model: RoBERTa-base Wechsel Swahili
Task: Multi-label Sequence Classification
Problem Type: multi_label_classification
Number of Labels: 8
Activation Function: Sigmoid (for multi-label)
Loss Function: BCEWithLogitsLoss
Output Format: Binary vectors [batch_size, num_labels]

Model Variants

v2.0 (Current): Multi-label classification - Returns multiple topics with confidence scores
v1.0 (Legacy): Single-label classification - Returns single topic (available at revision="v1.0-single-label")

Intended Use

Primary Use Cases

Content Classification: Categorize Swahili text messages, reports, or documents
Case Management: Automatically tag and route cases to appropriate departments
Content Moderation: Identify topics requiring attention (e.g., health emergencies, violence)
Data Analytics: Analyze trends and patterns in Swahili text data
Information Routing: Direct messages to relevant stakeholders based on topics

Out-of-Scope Uses

Not suitable for: Languages other than Swahili
Not suitable for: Very short text (< 5 words) or very long text (> 512 tokens)
Not suitable for: Real-time critical decision making without human oversight
Not suitable for: Medical diagnosis or legal advice

Training Details

Training Data

Dataset: Custom Swahili text dataset
Language: Swahili (Kiswahili)
Data Collection: U-Report platform messages and related Swahili text
Preprocessing: Text cleaning, normalization, and tokenization
Data Balance: Dataset balanced across 8 topics

Training Procedure

Training Type: Fine-tuning from pre-trained RoBERTa-base Wechsel Swahili
Optimizer: AdamW
Learning Rate: 2e-5
Batch Size: Variable (with gradient accumulation)
Epochs: 3
Gradient Accumulation: 4 steps
Weight Decay: 0.01
Mixed Precision: Enabled (FP16)
Early Stopping: Enabled (patience=2)

Training Hyperparametersl

learning_rate: 2e-5 per_device_train_batch_size: 4 gradient_accumulation_steps: 4 num_train_epochs: 3 weight_decay: 0.01 warmup_steps: 0 max_grad_norm: 1.0 fp16: true## Evaluation

Testing Data, Factors & Metrics

Evaluation Dataset: Held-out test set from balanced dataset
Evaluation Metrics:
- F1 Score (Micro): Aggregated across all labels
- F1 Score (Macro): Average per-label F1
- F1 Score (Samples): Average per-sample F1
- Precision (Micro/Macro): Classification precision
- Recall (Micro/Macro): Classification recall
- Hamming Loss: Fraction of incorrectly predicted labels
- Subset Accuracy: Exact match accuracy

Results

Metric	Score
F1 Score (Micro)	0.96
F1 Score (Macro)	0.96
F1 Score (Samples)	0.96
Precision (Micro)	0.96
Recall (Micro)	0.96
Hamming Loss	0.009054
Subset Accuracy	0.962

Model Performance Characteristics

Strengths

Multi-label Capability: Can identify multiple topics in a single text
Confidence Scores: Provides probability scores for each topic
Swahili Language Support: Specifically fine-tuned for Swahili text
Efficient Inference: ONNX format available for fast CPU inference
Balanced Performance: Trained on balanced dataset across all topics

Limitations

Language Specific: Only works with Swahili text
Topic Coverage: Limited to 8 predefined topics
Context Dependency: Performance may vary with text length and context
Dialect Variations: May not handle all Swahili dialects equally well
Threshold Sensitivity: Requires careful threshold tuning for optimal performance

Known Biases

Training Data Bias: Model reflects biases present in training data
Geographic Bias: May perform better on texts from regions in training data
Topic Imbalance: Some topics may have better representation in training data
Cultural Context: May not capture all cultural nuances in Swahili communication

How to Get Started with the Model

Using Transformers (PyTorch)

from transformers import AutoModelForSequenceClassification, AutoTokenizer import torch

Load model

model = AutoModelForSequenceClassification.from_pretrained( "NeboTech/swahili-text-classifier", problem_type="multi_label_classification" # CRITICAL for multi-label ) tokenizer = AutoTokenizer.from_pretrained("NeboTech/swahili-text-classifier")

Prepare input

text = "Nataka kujua dalili za COVID-19 na jinsi ya kujilinda" inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True, max_length=256)

Get predictions

model.eval() with torch.no_grad(): outputs = model(**inputs) logits = outputs.logits # Shape: [1, 8]

Apply sigmoid for multi-label

probs = torch.sigmoid(logits)

Apply threshold

threshold = 0.5 predictions = (probs > threshold).float()

Get applicable topics

applicable_topics = torch.where(predictions[0] == 1)[0].tolist() print(f"Applicable topics: {applicable_topics}") print(f"Probabilities: {probs[0].tolist()}")### Using ONNX Runtime

import onnxruntime as ort import numpy as np from transformers import AutoTokenizer

Load tokenizer

tokenizer = AutoTokenizer.from_pretrained("NeboTech/swahili-text-classifier")

Load ONNX model

session = ort.InferenceSession("swahili_classifier.onnx")

Prepare input

text = "Nataka kujua dalili za COVID-19" inputs = tokenizer(text, return_tensors="np", padding="max_length", truncation=True, max_length=256)

Run inference

outputs = session.run( None, { "input_ids": inputs["input_ids"].astype(np.int64), "attention_mask": inputs["attention_mask"].astype(np.int64) } )

logits = outputs[0] # Shape: [1, 8]

Apply sigmoid

probs = 1 / (1 + np.exp(-logits))

Apply threshold

threshold = 0.5 predictions = (probs > threshold).astype(float)

Get topics

applicable_topics = np.where(predictions[0] == 1)[0] print(f"Applicable topics: {applicable_topics}")## Topics (Label Mapping)

ID	Topic	Description
0	COVID	COVID-19 related topics, symptoms, prevention
1	EDUCATION	Educational content, school-related topics
2	HEALTH	General health topics, medical information
3	HIV/AIDS	HIV/AIDS related information and support
4	MENSTRUAL HYGIENE	Menstrual health and hygiene topics
5	NUTRITION	Nutrition, food, and dietary information
6	U-REPORT	U-Report platform related content
7	VIOLENCE AGAINST CHILDREN	Child protection and violence prevention

Ethical Considerations

Ethical Use

Human Oversight: Always include human review for critical decisions
Privacy: Respect user privacy when processing text data
Transparency: Inform users when automated classification is used
Fairness: Monitor for biased outcomes across different user groups

Potential Risks

Misclassification: Incorrect topic assignment could misroute important messages
False Positives/Negatives: May miss urgent cases or flag non-urgent content
Privacy Concerns: Processing sensitive health and personal information
Cultural Sensitivity: May not fully capture cultural context and nuances

Recommendations

Regular Monitoring: Continuously monitor model performance in production
Human Review: Implement human review for high-stakes classifications
Feedback Loop: Collect and incorporate user feedback for improvements
Bias Auditing: Regularly audit for biases and fairness issues
Threshold Tuning: Adjust thresholds based on use case requirements

Citation

@misc{swahili-topic-classifier-multilabel, title={Swahili Topic Classifier - Multi-label Classification}, author={NeboTech}, year={2024}, publisher={Hugging Face}, howpublished={\url{https://huggingface.co/NeboTech/swahili-text-classifier}}, note={Version 2.0 - Multi-label Classification} }## Additional Information

Model Files

config.json: Model configuration
pytorch_model.bin or model.safetensors: Model weights
tokenizer.json: Tokenizer model
tokenizer_config.json: Tokenizer configuration
vocab.json, merges.txt: Vocabulary files
swahili_classifier.onnx: ONNX model (separate repository)

Version History

v2.0 (Current): Multi-label classification with sigmoid activation
v1.0 (Legacy): Single-label classification with softmax activation

Contact

For questions, issues, or contributions