You can deploy this model and start earning money today!

irwan19/
albertir-quran-hadith

Name: albertir-quran-hadith
Availability: InStock

Views

No views yet

language: id license: mit base_model: indobenchmark/indobert-base-p1 pipeline_tag: fill-mask tags:

bert
indonesian
islamic
quran
hadith
masked-language-modeling
fill-mask library_name: transformers

ALBERTIR: BERT-Based Pretraining for Indonesian Religious Texts

Model Description

ALBERTIR is a domain-specific Indonesian language model pretrained on Islamic texts (Quran and Hadith translations). It is built upon the indobenchmark/indobert-base-p1 architecture and further pretrained using Masked Language Modeling (MLM) on an Islamic corpus.

This model is designed for:

Semantic alignment of Indonesian Islamic texts
Theological accuracy evaluation
Indonesian Friday sermon (Khutbah) analysis
Islamic content understanding and classification

Developed by

Irwan Darmawan – Universitas Negeri Malang, Indonesia
Hakkun Elmunsyah – Universitas Negeri Malang, Indonesia
Didik Dwi Prasetya – Universitas Negeri Malang, Indonesia
Wahyu Mulyo Utomo – Universiti Tun Hussein Onn Malaysia, Malaysia

Model Details

Parameter	Value
Model Type	BERT (Bidirectional Encoder Representations from Transformers)
Base Model	`indobenchmark/indobert-base-p1`
Task	Masked Language Modeling (MLM)
Language	Indonesian
Domain	Islamic / Religious Texts
Parameters	124,492,880
Tensor Type	F32

Pretraining Details

Parameter	Value
Corpus	Quran (6,236 verses) + Sahih Bukhari (7,008) + Sahih Muslim (5,362)
Total Documents	18,606
Epochs	10
Batch Size	8
Learning Rate	5e-5
Optimizer	AdamW
Total Steps	10,470
Final Training Loss	~1.2473
Final Validation Loss	~1.1415

Training Data

The model was trained on:

Quran: Indonesian translation (Ministry of Religious Affairs)
Sahih Bukhari: Indonesian translation
Sahih Muslim: Indonesian translation

Total: 18,606 verses/hadith documents

Intended Uses

Fill-mask tasks on Indonesian Islamic texts
Semantic similarity in religious domain
Theological content classification
Friday sermon analysis

How to Use

python

from transformers import AutoModelForMaskedLM, AutoTokenizer

model_id = "irwan19/albertir-quran-hadith"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForMaskedLM.from_pretrained(model_id)

# Example: Fill-mask prediction
text = "Umat Islam wajib melaksanakan ibadah [MASK] di bulan Ramadhan."
inputs = tokenizer(text, return_tensors="pt")
outputs = model(**inputs)
# Prediksi: 'puasa'