Model is trained specifically for indian context, we used sentence-piece tokenizer to train the model, so use the sentences with proper delimeter like(. , ?) and appropiate capitalization of words.

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 32
eval_batch_size: 32
seed: 42
gradient_accumulation_steps: 3
total_train_batch_size: 96
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 4
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	F1
No log	1.0	2509	0.1427	0.7972
No log	2.0	5019	0.1366	0.8101
0.1384	3.0	7529	0.1366	0.8139
0.1384	4.0	10036	0.1404	0.8130

Framework versions

Transformers 4.27.0
Pytorch 2.0.1+cu118
Datasets 2.14.4
Tokenizers 0.13.3

Venkatesh4342/NER-Indian-xlm-roberta

xlm-roberta-base-NER-ind

Model description