This model is a fine-tuned version of bert-base-cased on the WikiANN (WikiNER) English dataset for Named Entity Recognition (NER). The model can identify and classify named entities in text into predefined categories.

Performance Metrics

The model achieves strong performance on the WikiANN English validation set:

F1 Score: 82.47%
Precision: 80.92%
Recall: 84.07%
Accuracy: 92.50%

Supported Entity Types

The model recognizes 3 main entity types with BIO tagging:

PER (Person): Names of people
ORG (Organization): Names of organizations, companies, institutions
LOC (Location): Names of locations, cities, countries

Label Set

O: Outside of any entity
B-PER: Beginning of a person entity
I-PER: Inside a person entity
B-ORG: Beginning of an organization entity
I-ORG: Inside an organization entity
B-LOC: Beginning of a location entity
I-LOC: Inside a location entity

Intended Uses & Limitations

Intended Uses

Extract named entities from English text
Information extraction tasks
Text preprocessing for downstream NLP applications
Content analysis and categorization

Limitations

The model is trained on WikiANN dataset, which may not generalize well to domain-specific texts
Performance may vary on informal text (social media, chat messages)
Limited to English language only
Only recognizes PER, ORG, and LOC entity types

Training Data

The model was trained on the WikiANN (WikiNER) English dataset, which contains:

Training samples: 20,000
Validation samples: 10,000
Test samples: 10,000

WikiANN is a multilingual named entity recognition dataset derived from Wikipedia, providing annotated text with named entity labels.

How to Use

Using with Transformers Pipeline

python

from transformers import pipeline

# Load the NER pipeline
nlp = pipeline("ner", model="yiwenX/bert-finetuned-ner-accelerate")

# Example text
text = "Apple Inc. was founded by Steve Jobs in Cupertino, California."

# Get predictions
results = nlp(text)
print(results)

Using with AutoModel

python

from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("yiwenX/bert-finetuned-ner-accelerate")
model = AutoModelForTokenClassification.from_pretrained("yiwenX/bert-finetuned-ner-accelerate")

# Example text
text = "Apple Inc. was founded by Steve Jobs in Cupertino, California."

# Tokenize and predict
inputs = tokenizer(text, return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)

# Get predictions
predictions = torch.argmax(outputs.logits, dim=-1)
tokens = tokenizer.convert_ids_to_tokens(inputs["input_ids"][0])

# Map predictions to labels
label_list = model.config.id2label
entities = []
for token, pred in zip(tokens, predictions[0]):
    if pred != 0:  # 0 is 'O' (outside)
        entities.append((token, label_list[pred.item()]))

print(entities)

Training Procedure

Training Hyperparameters

The following hyperparameters were used during training:

Learning rate: 2e-05
Batch size: 16 (both training and evaluation)
Number of epochs: 3
Optimizer: AdamW with betas=(0.9, 0.999) and epsilon=1e-08
LR scheduler: Linear
Seed: 42

Training Results

Epoch	Training Loss	Validation Loss	Precision	Recall	F1	Accuracy
1.0	0.3066	0.2636	78.23%	81.86%	80.00%	91.89%
2.0	0.2059	0.2566	79.60%	83.37%	81.44%	92.42%
3.0	0.1455	0.2777	80.92%	84.07%	82.47%	92.50%

Framework Versions

Transformers: 4.56.0
PyTorch: 2.8.0+cu128
Datasets: 4.0.0
Tokenizers: 0.22.0

Citation

If you use this model in your research, please cite:

bibtex

@misc{bert-finetuned-ner-accelerate,
  author = {yiwenX},
  title = {BERT Fine-tuned for Named Entity Recognition},
  year = {2024},
  publisher = {HuggingFace},
  url = {https://huggingface.co/yiwenX/bert-finetuned-ner-accelerate}
}

License

This model is licensed under the Apache 2.0 License.