Views
No views yet
1from transformers import pipeline, AutoModelForTokenClassification, AutoTokenizer
2
3# Load model and tokenizer
4model_name = "loyoladatamining/firmNER-v3"
5model = AutoModelForTokenClassification.from_pretrained(
6 model_name,
7 id2label={0: 'O', 1: 'B-ORG', 2: 'I-ORG'},
8 label2id={'O': 0, 'B-ORG': 1, 'I-ORG': 2}
9)
10tokenizer = AutoTokenizer.from_pretrained(model_name, model_max_length=1024)
11
12# Create the NER pipeline
13nlp = pipeline(
14 "ner",
15 model=model,
16 tokenizer=tokenizer,
17 aggregation_strategy="max"
18)
19
20# Inference
21text = "Orange and Macrosoft announced a new partnership yesterday."
22results = nlp(text)
23print(results)aggregation_strategy="max", the model outputs a list of dictionaries. Each dictionary represents a detected entity (labeled as ORG for organizations/firms) and contains the following structure:1[
2 {
3 "entity_group": "ORG",
4 "score": 0.XXX,
5 "word": "Orange",
6 "start": 0,
7 "end": 6
8 },
9 {
10 "entity_group": "ORG",
11 "score": 0.XXX,
12 "word": "Macrosoft",
13 "start": 11,
14 "end": 20
15 }
16]entity_group: The predicted label group (i.e. here, ORG for identified company names).score: The model's confidence probability for the predicted entity.word: The extracted text string representing the firm name.start / end: The character indices indicating the exact position of the text string within the input document.firmNER useful in your work, please consider citing:@article{meisenbacher2025extracting,
title={Extracting O* NET Features from the NLx Corpus to Build Public Use Aggregate Labor Market Data},
author={Meisenbacher, Stephen and Nestorov, Svetlozar and Norlander, Peter},
year={2025}
}