RoBERTa-Based SDG Document Classification Model
Model Overview
This model is a RoBERTa-based Natural Language Processing (NLP) model designed to classify documents according to the United Nations Sustainable Development Goals (SDGs). It is trained on institutional documents from Higher Education Institutions (HEIs) in the Cordillera Administrative Region, Philippines. The model aims to support SDG alignment assessments for universities and organizations.
Model Details
- Model Type: RoBERTa-based Sequence Classification
- Frameworks: PyTorch, Hugging Face Transformers
- Libraries Used: pandas, nltk, scikit-learn, pytesseract
- Training Environment: Google colab
Dataset
The dataset consists of academic journals, news articles, and official publications from HEIs, including:
- SLU
- UPB
- UB
- UC
- IFSU
- ASC
- MPSPC
- UP
- ADMU
- DLSU
- UST
- UPLB
- MSU
- LPU
- PSU
Data Preprocessing
- OCR: Pytesseract for text extraction
- Cleaning: URL removal, non-word character removal, stopword removal, tokenization, lemmatization
- Custom Lexicon: LexDG for local language adaptation (e.g., Ilocano terms)
Methodology
Data Collection and Preparation
Documents were collected, cleaned, and labeled with relevant SDGs. LexDG was integrated to capture local linguistic nuances.
Model Development
- Vectorization: TF-IDF with scikit-learn
- Model Architecture: RoBERTa Sequence Classification
- Training: 80% training, 10% validation, 10% testing splits
Evaluation Metrics
- Precision: Accuracy of positive predictions
- Recall: Ability to identify all relevant instances
- F1-Score: Harmonic mean of Precision and Recall
Implementation and Deployment
The trained model is deployed as part of a web application hosted on a cloud platform, providing API support for seamless integration. Users can upload documents, which are processed and classified against SDGs.
Sample Output
The web application displays:
- SDG Alignment Scores: Percentage alignment with each SDG
- Keyword Insights: Highlighting dominant themes and gaps
Limitations
- Regional Bias: Focus on CAR-specific documents may limit generalizability
- Data Imbalance: Uneven representation across SDGs
- Word Ambiguity: Some local terms clash with other local terms and may cause ambiguity
Future Work
- Expanding the dataset to include national and international documents
- Improving model performance for underrepresented SDGs
- Continuous updates to LexDG for better local language support
Citation
If you use this model, please cite the associated research paper:
Title: "Using Natural Language Processing to Assess Contributions to Sustainable Development Goals of Academic Institutions"
Authors: Miguel, Abella et al.
License
