Authors: Tanvi Somani , Aryan Babare , Samyak Bora
Model Type: Masked Language Model (MLM)
Base Model: MahaBERT v2
Training Method: DAFT (Domain-Adaptive Fine-Tuning)
Language: Marathi 🇮🇳
Framework: HuggingFace Transformers

📌 Overview

mahabert-v2-daft is a Domain-Adaptive Fine-Tuned (DAFT) version of the original MahaBERT v2 model.
The model has been further trained on a large collection of unlabeled Marathi text to adapt it to the specific domain used in downstream Marathi NLP tasks such as:

Sentiment analysis
Text classification
Emotion detection
Topic analysis
General Marathi language understanding tasks

DAFT helps the model learn domain-specific vocabulary, patterns, and semantics, improving downstream task performance without requiring labeled data.

🧠 What is DAFT?

DAFT (Domain Adaptive Fine-Tuning) is a continued pre-training method where:

You take a pretrained language model
Feed it large amounts of unlabeled domain text
Train it again using the Masked Language Modeling (MLM) objective

This process improves the model’s understanding of domain-specific words, idioms, and sentence structures.

🚀 Training Details

Item	Details
Base Model	MahaBERT v2
Method	DAFT (continued pre-training)
Objective	Masked Language Modeling (MLM)
Dataset	Unlabeled Marathi text (domain-specific)
Batch Size	As per Colab training setup
Epochs	Several thousand steps (as seen in training logs)
Hardware	Google Colab (T4 GPU)
Optimizer	AdamW
Precision	FP32

📁 Files Included

The repository contains:

model.safetensors — model weights
config.json — model architecture
tokenizer.json, tokenizer_config.json — tokenizer settings
vocab.txt — BERT vocabulary
special_tokens_map.json — CLS, SEP, PAD, MASK tokens
training_args.bin — training configuration

🧩 Usage

🔹 Load the Model

python

from transformers import AutoTokenizer, AutoModelForMaskedLM

model_id = "aryanx16/mahabert-v2-daft"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForMaskedLM.from_pretrained(model_id)