BC5CDR_ClinicalBERT_NER

• Loss: 0.1107
• Seqeval classification report: precision recall f1-score support
  Chemical 0.71 0.73 0.72 10493 Disease 0.82 0.82 0.82 6944
  micro avg 0.75 0.77 0.76 17437 macro avg 0.76 0.78 0.77 17437 weighted avg 0.75 0.77 0.76 17437

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Chemical       0.67      0.68      0.68     10493
 Disease       0.79      0.78      0.78      6944

Chemical       0.69      0.71      0.70     10493
 Disease       0.77      0.83      0.80      6944

Chemical       0.71      0.73      0.72     10493
 Disease       0.82      0.82      0.82      6944

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.15.0
• Tokenizers 0.15.0