This is a DeBERTa [1] uncased model for the Italian language, fine-tuned for Named Entity Recognition (Person, Location, Organization and Miscellanea classes) on the WikiNER dataset [2], using mdeberta-v3-base as a pre-trained model.

Training and Performances

The model is trained to perform entity recognition over 4 classes: PER (persons), LOC (locations), ORG (organizations), MISC (miscellanea, mainly events, products and services). It has been fine-tuned for Named Entity Recognition, using the WikiNER Italian dataset plus an additional custom dataset of manually annotated Wikipedia paragraphs. The WikiNER dataset has been splitted in 102.352 training instances and 25.588 test instances, and the model has been trained for 1 epoch with a constant learning rate of 1e-5.

The model has been first fine-tuned on WikiNER, then focused on the Italian language and turned to uncased by modifying the embedding layer (as in [3], computing document-level frequencies over the Wikipedia dataset), and lastly fine-tuned on an additional dataset of ~3.500 manually annotated lowercase paragraphs.

Quick usage

python

from transformers import AutoModelForTokenClassification, AutoTokenizer
from transformers import pipeline
import re
import string

tokenizer = AutoTokenizer.from_pretrained("osiria/deberta-base-italian-uncased-ner")
model = AutoModelForTokenClassification.from_pretrained("osiria/deberta-base-italian-uncased-ner", num_labels = 5)

text = "mi chiamo marco rossi, vivo a roma e lavoro per l'agenzia spaziale italiana nella missione prisma"

for p in string.punctuation:
    text = text.replace(p, " " + p + " ")

ner = pipeline("ner", model=model, tokenizer=tokenizer)
ner(text, aggregation_strategy="simple")

[{'entity_group': 'PER',
  'score': 0.9929623,
  'word': 'marco rossi',
  'start': 9,
  'end': 21},
 {'entity_group': 'LOC',
  'score': 0.9898509,
  'word': 'roma',
  'start': 31,
  'end': 36},
 {'entity_group': 'ORG',
  'score': 0.9905911,
  'word': 'agenzia spaziale italiana',
  'start': 53,
  'end': 79},
 {'entity_group': 'MISC',
  'score': 0.92474234,
  'word': 'missione prisma',
  'start': 85,
  'end': 101}]

References

[1] https://arxiv.org/abs/2111.09543

[2] https://www.sciencedirect.com/science/article/pii/S0004370212000276

[3] https://arxiv.org/abs/2010.05609

Limitations

This model is mainly trained on Wikipedia, so it's particularly suitable for natively digital text from the world wide web, written in a correct and fluent form (like wikis, web pages, news, etc.). However, it may show limitations when it comes to chaotic text, containing errors and slang expressions (like social media posts) or when it comes to domain-specific text (like medical, financial or legal content).

License

The model is released under MIT license