This model was contrastively trained for entity coreference on a dataset constructed by mentions of the same entity. The model requires text with entities detected via NER and focuses specifically on Person [PER] tags. The mention of an entity of interest is then decorated with a set of special tokens with the syntax - ([M ] Entity [\M ] .
We start with a base S-BERT MPNet bi-encoder
model. This is contrastively trained on 179 million pairs taken from mentions of entities on Wikipedia, where positives are mentions of the same individual. Hard negatives are mined using individuals that appear on the same disambiguation pages.
This essentially trains a semantic similarity model.
At inference time, to collect mentions of the same entity, we recommend clustering contexts with decorated entities in the embedding space - each cluster is then a set of mentions of the same entity, clustered through the contextual information contained in the enclosing text.
More information about its training (and use) can be found on the associated code
repo and
paper.
