You can deploy this model and start earning money today!

ilprl-docse/
SL-topic-classification-HindiBERT-25k-110M

Name: SL-topic-classification-HindiBERT-25k-110M
Availability: InStock

Views

No views yet

Benchmarking BERT-based Models for Sentence-level Topic Classification in Nepali Language

Overview

This repository contains the implementation and experiments for benchmarking various BERT-based transformer models on sentence-level topic classification in Nepali, a low-resource language.

We evaluate multilingual, Indic, Hindi, and Nepali-specific models to understand their effectiveness in capturing linguistic nuances of Nepali text.

Objectives

Benchmark multiple BERT-based models on Nepali text classification
Analyze performance differences across multilingual, Indic, and monolingual models
Establish a strong baseline for future Nepali NLP tasks
Provide insights into low-resource language modeling

Dataset

The dataset consists of 25,006 Nepali sentences categorized into five domains:

🌾 Agriculture
🏥 Health
🎓 Education & Technology
🏔️ Culture & Tourism
💬 General Communication

The dataset is balanced across all categories.

🔗 Dataset Link: https://huggingface.co/datasets/ilprl-docse/NepSen-Nepali-Categorical-Sentences-Corpus

Models Evaluated

We benchmarked the following transformer-based models:

Multilingual Models

mBERT
XLM-RoBERTa
mDeBERTa

Indic Models

MuRIL (base & large)
IndicBERT
DevBERT

Language-Specific Models

HindiBERT
NepBERTa

English Model

RoBERTa

🔗 Model Links: https://hf.co/collections/ilprl-docse/benchmarking-bert-based-models-for-topic-classification

Visit: https://github.com/ilprl/Benchmarking-BERT-based-Models-for-Sentence-level-Topic-Classification-in-Nepali-Language

Citation

Paper Link: https://arxiv.org/abs/2602.23940

If you use this work, please cite:

bibtex

@inproceedings{karki2026benchmarking,
  title={Benchmarking BERT-based Models for Sentence-level Topic Classification in Nepali Language},
  author={Karki, Nischal and Subedi, Bipesh and Poudyal, Prakash and Ghimire, Rupak Raj and Bal, Bal Krishna},
  booktitle={Proceedings of the Regional International Conference on Natural Language Processing (RegICON 2025)},
  year={2026},
  address={Guwahati, India},
  note={Gauhati University, November 27--29, 2025},
  url={https://arxiv.org/abs/2602.23940}
}

ilprl-docse/SL-topic-classification-HindiBERT-25k-110M