[roberta-large-ner-english] is an english NER model that was fine-tuned from roberta-large on conll2003 dataset.
Model was validated on emails/chat data and outperformed other models on this type of data specifically.
In particular the model seems to work better on entity that don't start with an upper case.
In order to simplify, the prefix B- or I- from original conll2003 was removed.
I used the train and test dataset from original conll2003 for training and the "validation" dataset for validation. This resulted in a dataset of size:
1from transformers import AutoTokenizer, AutoModelForTokenClassification
2
3tokenizer = AutoTokenizer.from_pretrained("Jean-Baptiste/roberta-large-ner-english")
4model = AutoModelForTokenClassification.from_pretrained("Jean-Baptiste/roberta-large-ner-english")
5
6
7##### Process text sample (from wikipedia)
8
9from transformers import pipeline
10
11nlp = pipeline('ner', model=model, tokenizer=tokenizer, aggregation_strategy="simple")
12nlp("Apple was founded in 1976 by Steve Jobs, Steve Wozniak and Ronald Wayne to develop and sell Wozniak's Apple I personal computer")
13
14
15[{'entity_group': 'ORG',
16 'score': 0.99381506,
17 'word': ' Apple',
18 'start': 0,
19 'end': 5},
20 {'entity_group': 'PER',
21 'score': 0.99970853,
22 'word': ' Steve Jobs',
23 'start': 29,
24 'end': 39},
25 {'entity_group': 'PER',
26 'score': 0.99981767,
27 'word': ' Steve Wozniak',
28 'start': 41,
29 'end': 54},
30 {'entity_group': 'PER',
31 'score': 0.99956465,
32 'word': ' Ronald Wayne',
33 'start': 59,
34 'end': 71},
35 {'entity_group': 'PER',
36 'score': 0.9997918,
37 'word': ' Wozniak',
38 'start': 92,
39 'end': 99},
40 {'entity_group': 'MISC',
41 'score': 0.99956393,
42 'word': ' Apple I',
43 'start': 102,
44 'end': 109}]
Model performances computed on conll2003 validation dataset (computed on the tokens predictions)
On private dataset (email, chat, informal discussion), computed on word predictions:
By comparison on the same private dataset, Spacy (en_core_web_trf-3.2.0) was giving:
