This model is designed to classify place named entities recognized from geographic encyclopedia articles.
It is a fine-tuned version of the bert-base-multilingual-cased model.
It has been trained on
no-name-research/no-name-dataset, a manually annotated subset of the French
Encyclopédie ou dictionnaire raisonné des sciences des arts et des métiers par une société de gens de lettres (1751-1772) edited by Diderot and d'Alembert (provided by the
ARTFL Encyclopédie Project).
The model was trained using the
no-name-research/no-name-dataset dataset.
The dataset is splitted into train, validation and test sets which have the following distribution of entries among classes:
Use the code below to get started with the model.
1import torch
2from transformers import pipeline
3device = torch.device("mps" if torch.backends.mps.is_available() else ("cuda" if torch.cuda.is_available() else "cpu"))
4
5ner = pipeline("token-classification", model="no-name-research/camembert-token-classification", aggregation_strategy="simple", device=device)
6placename_classifier = pipeline("text-classification", model="no-name-research/multilingual-bert-place-ner-classifier", truncation=True, device=device)
7
8def get_context(text, span, ngram_context_size=5):
9 word = span["word"]
10 start = span["start"]
11 end = span["end"]
12 label = span["entity_group"]
13
14 # Extract context
15 previous_text = text[:start].strip()
16 next_text = text[end:].strip()
17 previous_words = previous_text.split()[-ngram_context_size:]
18 next_words = next_text.split()[:ngram_context_size]
19
20 # Build context string
21 context = f"[{word}]: {' '.join(previous_words)} {word} {' '.join(next_words)}"
22 return word, context, label
23
24content = "WINCHESTER, (Géog. mod.) ou plutôt Wintchester, ville d'Angleterre, capitale du Hampshire, sur le bord de l'Itching, à dix-huit milles au sud-est de Salisbury, & à soixante sud-ouest de Londres. Long. 16. 20. latit. 51. 3."
25
26spans = ner(content)
27for span in spans:
28 if span['entity_group'] == 'NP_Spatial':
29 word, context, label = get_context(content, span, ngram_context_size=5)
30 print(f"Place name: {word}")
31
32 label = placename_classifier(context)
33 print(f"Predicted label: {label}")
34
35
36# Output
37Place name: Wintchester
38Predicted label: [{'label': 'City', 'score': 0.9968810081481934}]
39Place name: Angleterre
40Predicted label: [{'label': 'Country', 'score': 0.9953059554100037}]
41Place name: Hampshire
42Predicted label: [{'label': 'Region', 'score': 0.9967537522315979}]
43Place name: Itching
44Predicted label: [{'label': 'River', 'score': 0.9929990768432617}]
45Place name: Salisbury
46Predicted label: [{'label': 'City', 'score': 0.9969013929367065}]
47Place name: Londres
48Predicted label: [{'label': 'City', 'score': 0.9969471096992493}]
49
This model was trained entirely on French encyclopaedic entries classified as Geography and will likely not perform well on text in other languages or other corpora.
