bert-finetuned-ner
This model is a fine-tuned version of
bert-base-cased on
unimelb-nlp/wikiann dataset English Language
.
It achieves the following results on the evaluation set:
- Loss: 0.2904
- Precision: 0.8249
- Recall: 0.8498
- F1: 0.8372
- Accuracy: 0.9311
Model description
This model is a BERT-based Named Entity Recognition (NER) system fine-tuned from bert-base-cased for English token classification.
It identifies and classifies named entities using the BIO tagging scheme across the following entity types:
PER (Person)
ORG (Organization)
LOC (Location)
O (Outside)
The model processes tokenized text and outputs entity spans using contextualized embeddings learned through transformer self-attention mechanisms.
Intended uses & limitations
Intended Uses
Information extraction from English text
Named entity recognition in NLP pipelines
Academic research and educational projects
Preprocessing step for downstream tasks (e.g., relation extraction, QA)
Limitations
Trained only on English data
Performance may degrade on domain-specific text (medical, legal, informal)
Limited to PER, ORG, LOC entity types
Sensitive to tokenization artifacts in noisy or misspelled text
Training and evaluation data
The model was trained and evaluated using the WikiAnn (PAN-X) dataset for English.
Dataset Details
Multilingual, Wikipedia-based NER corpus
Automatically annotated
BIO labeling scheme
Final Data Split
Training: 30,000 sentences
Validation: 5,000 sentences
Test: 5,000 sentences
Entity Labels
O, B-PER, I-PER, B-ORG, I-ORG, B-LOC, I-LOC
Training procedure
Base Model: bert-base-cased
Framework: Hugging Face Transformers
Task: Token Classification (NER)
Epochs: 3
Learning Rate: 2e-5
Optimizer: AdamW
Weight Decay: 0.01
Evaluation Metric: SeqEval (Precision, Recall, F1, Accuracy)
Label Alignment: Subword-aware BIO label propagation
Trainer API: Hugging Face Trainer
The model was evaluated after each epoch and achieved strong overall performance on the held-out test set.
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 3
Training results
| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
|---|
| 0.2821 | 1.0 | 3750 | 0.2421 | 0.7914 | 0.8387 | 0.8143 | 0.9259 |
| 0.1919 | 2.0 | 7500 | 0.2524 | 0.8163 | 0.8433 | 0.8296 | 0.9289 |
| 0.1307 | 3.0 | 11250 | 0.2904 | 0.8249 | 0.8498 | 0.8372 | 0.9311 |
Framework versions
- Transformers 4.57.3
- Pytorch 2.9.0+cu126
- Datasets 4.0.0
- Tokenizers 0.22.1
