AmanLeb
AmanLeb is a multilingual SMS classification model developed for scam and smishing detection in Lebanon.
The model is fine-tuned from:
distilbert/distilbert-base-multilingual-cased
It predicts three classes:
Final Test Performance
- Accuracy: 97.3%
- Macro F1: 0.905
- Suspicious messages predicted as HAM: 2
- Smishing messages predicted as HAM: 0 on the held-out test set
Intended Use
This model is part of the AmanLeb research prototype for detecting suspicious SMS messages in multilingual settings.
The full project, including notebooks, Streamlit application, RAG pipeline, evaluation results, and documentation, is available in the AmanLeb GitHub repository.
Limitations
This model was trained on a relatively small SMS phishing dataset and should not be treated as a production anti-fraud system.
Performance may degrade on:
- unseen scam patterns
- dialectal Arabic
- Arabizi variations
- new organizations or brands
- adversarial messages
- distribution shifts
The reported zero smishing-to-HAM errors applies only to the held-out test set and is not a guarantee for real-world messages.
Base Model
distilbert/distilbert-base-multilingual-cased
