This model is a historical named entity recognition model trained on HIPE-style historical newspaper data.
It is based on
dbmdz/bert-base-historic-multilingual-cased and fine-tuned for token classification using the
NE-COARSE-LIT annotation column.
The model predicts coarse named entity labels for historical French text.
The original data contained labels from slightly different annotation schemes. Labels were normalized before training.
1B-PER -> B-pers
2I-PER -> I-pers
3B-LOC -> B-loc
4I-ORG -> I-org
5B-STREET -> B-loc
6B-BUILDING -> B-loc
7B-HumanProd -> B-prod
8B-object -> B-prod
9B-work -> B-prod
10B-date -> B-time
1from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline
2
3model_id = "emanuelaboros/historical-ner-baseline"
4
5tokenizer = AutoTokenizer.from_pretrained(model_id)
6model = AutoModelForTokenClassification.from_pretrained(model_id)
7
8ner = pipeline(
9 "token-classification",
10 model=model,
11 tokenizer=tokenizer,
12 aggregation_strategy="simple",
13)
14
15text = "Charlotte née Bourgoin, femme de Joseph Digiez, fut admise par le Conseil."
16
17predictions = ner(text)
18
19for entity in predictions:
20 print(entity)
1[
2 {
3 "entity_group": "pers",
4 "score": 0.98,
5 "word": "Charlotte née Bourgoin",
6 "start": 0,
7 "end": 24,
8 },
9 {
10 "entity_group": "pers",
11 "score": 0.97,
12 "word": "Joseph Digiez",
13 "start": 35,
14 "end": 48,
15 },
16 {
17 "entity_group": "org",
18 "score": 0.91,
19 "word": "Conseil",
20 "start": 70,
21 "end": 77,
22 },
23]
