You can deploy this model and start earning money today!

tanuj437/
SanskritBERT

Name: SanskritBERT
Availability: InStock

Views

No views yet

SanskritBERT (Light)

SanskritBERT is a lightweight Transformer model trained specifically for the Sanskrit language. It is based on the BERT architecture and trained using the Masked Language Modeling (MLM) objective.

Model Description

Shared by: Tanuj Saxena and Soumya Sharma
Model type: Transformers Encoder (BERT-like)
Language: Sanskrit
License: Apache 2.0
Finetuned from model: None (Trained from scratch)

Model Architecture

Layers: 6
Hidden Size: 256
Attention Heads: 4
Feedforward Size: 1024
Max Sequence Length: 512
Vocab Size: 64,000
Parameters: ~15M

Intended Uses & Limitations

Intended Uses

Masked Word Prediction
Fine-tuning for Sanskrit NLP tasks involves (POS Tagging, NER, Text Classification)
Research into low-resource language modeling

Limitations

The model is "Light", so it may not capture as much nuance as a bert-base or bert-large model.
Performance depends heavily on the domain of the downstream task relative to the pre-training corpus.

Training Data

Trained on a corpus of Sanskrit texts including general literature, wikis, and classical texts.

Training Procedure

Optimizer: AdamW
Precision: Mixed Precision (bf16)
Batch Size: 16
Epochs: 6

How to Get Started

You can use the model directly with the Hugging Face transformers library:

python

from transformers import AutoTokenizer, AutoModelForMaskedLM

tokenizer = AutoTokenizer.from_pretrained("tanuj437/SanskritBERT")
model = AutoModelForMaskedLM.from_pretrained("tanuj437/SanskritBERT")

text = "सत्यमेव जयते [MASK]"
inputs = tokenizer(text, return_tensors="pt")
outputs = model(**inputs)

Citation

bibtex

@misc{sanskritbert2024,
  title={SanskritBERT: A Light Transformer Model for Sanskrit},
  author={[Tanuj Saxena,Soumya Sharma,Kusum Lata]},
  year={2026},
  publisher={Hugging Face}
}