Model card for NOS Drug-Related Text Classification on Telegram
The NOS editorial team is conducting an investigation into drug-related messages on Telegram. Thousands of Telegram messages has been labeled as drugs-related content (or not), as well including detail regarding the specific type of drugs, and delivery method. The data is utilized in order to train a model to scale it up and automatically label millions more.
Methodology
Primarily a Logistic Regression model has been trained for binary classification. Text data was converted to numeric values using the Tfidf Vectorizer, considering term frequency-inverse document frequency (TF-IDF). This transformation enables the model to learn patterns and relationships between words. The model achieved 97% accuracy on the test set.
To take tasks with multiple possible labels into consideration, a MultiOutputClassifier was employed as an extension. This addresses the complexity of associating a text message with multiple categories such as "soft drugs," "hard drugs," and "medicines”. One-Hot Encoding was used for multi-label transformation.
Performance evaluation utilized Hamming Loss, a metric suitable for multi-label classification. The model demonstrated a Hamming Loss of 0.04, indicating 96% accuracy per label.
Tools used to train the model
• Python
• scikit-learn
• pandas
• numpy
How to Get Started with the Model
Use the code below to get started with the model.
python
1from joblib import load
23# load the model4clf = load('model.joblib')56# make some predictions78text_messages =[9"""
10 Oud kleding te koop! Stuur een berichtje
11 We repareren ook!
12 """,1314"""
15 COKE/XTC
16 * 1Gram = €50
17 * 5Gram = €230
18 """]1920mapping ={0:"bezorging",1:"bulk",2:"designer",3:"drugsad",4:"geendrugsad",5:"harddrugs",6:"medicijnen",7:"pickup",8:"post",9:"softdrugs"}2122labels =[]2324for message in clf.predict(text_messages):25 label =[]26for idx, labeled inenumerate(message):27if labeled ==1:28 label.append(mapping[idx])29 labels.append(label)3031print(labels)32
Details
Shared by Dutch Public Broadcasting Foundation (NOS)
Model type: text-classification
Language: Dutch
License: Creative Commons Attribution Non Commercial No Derivatives 4.0
