Model description
entity-recognition-general-sota-v1-finetuned-ner
This model is a fine-tuned version of
numind/entity-recognition-general-sota-v1 on Babelscape/MultiNerd dataset.
It achieves the following results on the validation set:
- Loss: 0.0396
- Precision: 0.9138
- Recall: 0.9146
- F1: 0.9142
- Accuracy: 0.9857
Training and evaluation data
The dataset if filtered on english language and sampled first 1M on train and 100k on validation.
further filtered with data containing atleast one tag from labels2ids mentioned below.
Training data - 131280 items
Validation data - 16410 items
Trained on all tags from the MultiNERD dataset.
labels2ids = {
"O": 0,
"B-PER": 1,
"I-PER": 2,
"B-ORG": 3,
"I-ORG": 4,
"B-LOC": 5,
"I-LOC": 6,
"B-ANIM": 7,
"I-ANIM": 8,
"B-BIO": 9,
"I-BIO": 10,
"B-CEL": 11,
"I-CEL": 12,
"B-DIS": 13,
"I-DIS": 14,
"B-EVE": 15,
"I-EVE": 16,
"B-FOOD": 17,
"I-FOOD": 18,
"B-INST": 19,
"I-INST": 20,
"B-MEDIA": 21,
"I-MEDIA": 22,
"B-MYTH": 23,
"I-MYTH": 24,
"B-PLANT": 25,
"I-PLANT": 26,
"B-TIME": 27,
"I-TIME": 28,
"B-VEHI": 29,
"I-VEHI": 30,
}
Training procedure
HF Trainer module
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 20
- eval_batch_size: 20
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 1
Training & Test set evaluation results
| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
|---|
| 0.0323 | 1.0 | 6564 | 0.0396 | 0.9138 | 0.9146 | 0.9142 | 0.9857 |
Evaluation on test set:
{'eval_loss': 0.02707073651254177,
'eval_precision': 0.9378337879893957,
'eval_recall': 0.9518034704620784,
'eval_f1': 0.9447669917943954,
'eval_accuracy': 0.9901678553418342,
'eval_runtime': 133.0665,
'eval_samples_per_second': 247.305,
'eval_steps_per_second': 30.917}
Framework versions
- Transformers 4.35.2
- Pytorch 2.1.0+cu118
- Datasets 2.15.0
- Tokenizers 0.15.0
