This is a SpanMarker model trained on the YurtsAI/named_entity_recognition_document_context dataset that can be used for Named Entity Recognition. This SpanMarker model uses roberta-large as the underlying encoder.

Model Details

Model Description

Model Type: SpanMarker
Encoder: roberta-large
Maximum Sequence Length: 256 tokens
Maximum Entity Length: 11 words
Training Dataset: YurtsAI/named_entity_recognition_document_context
Language: en

Model Sources

Repository: SpanMarker on GitHub
Thesis: SpanMarker For Named Entity Recognition

Model Labels

Label	Examples
DATETIME__absolute	"14:00 hrs", "15th november 2023 at 10:00 am", "october 15th , 2023"
DATETIME__authored	"25 february 26", "sunday , 21 august , 1938", "1961-05-08"
DATETIME__range	"29th of oct. , 2023", "september 2021 to august 2023", "jan 2022 - dec 2022"
DATETIME__relative	"eod friday", "dec 15 , 11:59 pm", "10/15"
GENERAL__art-broadcastprogram	"stranger things", "live q & a", "product design concept sketchbook for kids"
GENERAL__art-film	"the crown", "kill bill", "stranger things"
GENERAL__art-music
GENERAL__art-other	"statue of liberty", "broadway show", "wicked"
GENERAL__art-painting	"draw your dream house", "design a superhero costume"
GENERAL__art-writtenart	"optimization of quantum algorithms for cryptographic applications", "introduction to algorithms", "intro to cs '' by j. doe"
GENERAL__building-airport	"ory", "charles de gaulle", "cdg"
GENERAL__building-hospital	"green valley clinic", "department of oncology", "st. mary 's hospital"
GENERAL__building-hotel	"le jules verne", "hôtel ritz", "the beverly hills hotel"
GENERAL__building-library	"ancient library", "the grand library", "jefferson library"
GENERAL__building-other	"louvre museum", "engineering building", "eiffel tower"
GENERAL__building-restaurant	"l'ambroisie", "bella 's bistro", "in-n-out burger"
GENERAL__building-sportsfacility	"fenway"
GENERAL__building-theater	"gershwin theatre", "opera house", "broadway"
GENERAL__event-attack/battle/war/militaryconflict	"1863 battle of ridgefield", "battle of gettysburg", "war of 1812"
GENERAL__event-other	"annual science fair", "summer splash '23", "research methodology workshop"
GENERAL__event-sportsevent	"international olympiad in informatics", "ftx", "ioi"
GENERAL__location-GPE	"fr", "paris ,", "italy"
GENERAL__location-bodiesofwater	"river x", "river blue", "seine river"
GENERAL__location-island	"maldives", "similan islands", "ellis island"
GENERAL__location-mountain	"andes mountains", "swiss alps", "pine ridge"
GENERAL__location-other	"times square", "old market", "venice beach"
GENERAL__location-park	"central park", "ueno park", "universal studios"
GENERAL__location-road/railway/highway/transit	"i-95", "underground railroad", "hollywood walk of fame"
GENERAL__organization-company	"green earth organics", "xyz corporation", "north atlantic fisheries"
GENERAL__organization-education	"graduate school", "xyz", "xyz university"
GENERAL__organization-government/governmentagency	"department of economic development", "moe", "ministry of environment"
GENERAL__organization-media/newspaper	"pinterest", "yelp", "insta"
GENERAL__organization-other	"historical society", "grants office", "admissions committee"
GENERAL__organization-religion	"buddhist", "zen buddhist", "shinto"
GENERAL__organization-showorganization	"phare", "the soundbytes"
GENERAL__organization-sportsteam	"varsity soccer team", "red sox"
GENERAL__other-astronomything
GENERAL__other-award	"team excellence award", "innovation award", "employee of the month"
GENERAL__other-biologything	"fodmap", "troponin i", "cmp"
GENERAL__other-chemicalthing	"co2", "pm2.5", "nitrate"
GENERAL__other-currency	"usd", "inr", "$ $ $"
GENERAL__other-disease	"mi", "irritable bowel syndrome", "myocardial infarction"
GENERAL__other-educationaldegree	"executive mba", "phd in quantum computing ,", "phd"
GENERAL__other-god	"inari", "athena", "inari taisha"
GENERAL__other-language	"french", "english", "spanish"
GENERAL__other-law	"cas", "clean air standards", "environmental protection act ( epa ) 2023"
GENERAL__other-livingthing	"eastern box turtle", "monarch butterfly", "western burrowing owl"
GENERAL__other-medical	"asa", "dapt", "clopidogrel"
GENERAL__person-artist/author	"carol", "picasso", "warhol"
GENERAL__person-other	"jamie", "sarah", "mark"
GENERAL__person-politician	"jane doe", "vespasian", "constantine i"
GENERAL__person-scholar	"dr. smith", "dr. lee", "dr. johnson"
GENERAL__person-soldier	"davis", "lt. sarah johnson", "col. r. johnson"
GENERAL__product-airplane	"hmmwvs", "uh-60s", "m1a2s"
GENERAL__product-car	"hmmwvs", "high mobility multipurpose wheeled vehicles", "mine-resistant ambush protected"
GENERAL__product-food	"pumpkin spice", "quinoa salad", "golden jubilee feast"
GENERAL__product-game	"stardew valley", "valorant", "call of duty : warzone"
GENERAL__product-other	"engagement metrics", "xj-200", "smart goal templates"
GENERAL__product-ship	"liberty island ferry", "hms victory", "thames river cruise"
GENERAL__product-software	"instagram", "svm", "r"
GENERAL__product-train	"n'ex", "shinkansen", "tgv"
GENERAL__product-weapon	"m1 abrams", "m4 carbine", "m4 carbines"

Evaluation

Metrics

Label	Precision	Recall	F1
all	0.8309	0.8390	0.8349
DATETIME__absolute	0.8744	0.8577	0.8660
DATETIME__authored	0.9956	0.9935	0.9946
DATETIME__range	0.8451	0.9262	0.8838
DATETIME__relative	0.8266	0.7498	0.7863
GENERAL__art-broadcastprogram	0.6538	0.6296	0.6415
GENERAL__art-film	0.8	1.0	0.8889
GENERAL__art-music	0.0	0.0	0.0
GENERAL__art-other	0.625	0.7143	0.6667
GENERAL__art-painting	0.0	0.0	0.0
GENERAL__art-writtenart	0.7373	0.8047	0.7695
GENERAL__building-airport	0.8668	0.9689	0.9150
GENERAL__building-hospital	0.8378	0.9323	0.8826
GENERAL__building-hotel	0.7577	0.8603	0.8057
GENERAL__building-library	0.0	0.0	0.0
GENERAL__building-other	0.7597	0.8409	0.7982
GENERAL__building-restaurant	0.7953	0.8695	0.8307
GENERAL__building-sportsfacility	0.0	0.0	0.0
GENERAL__building-theater	0.6	0.6667	0.6316
GENERAL__event-attack/battle/war/militaryconflict	0.8438	0.9310	0.8852
GENERAL__event-other	0.6019	0.6382	0.6195
GENERAL__event-sportsevent	0.0	0.0	0.0
GENERAL__location-GPE	0.7232	0.7888	0.7546
GENERAL__location-bodiesofwater	0.6724	0.975	0.7959
GENERAL__location-island	0.7455	0.9111	0.8200
GENERAL__location-mountain	0.7436	0.8529	0.7945
GENERAL__location-other	0.7186	0.7793	0.7477
GENERAL__location-park	0.7899	0.8704	0.8282
GENERAL__location-road/railway/highway/transit	0.6325	0.7095	0.6688
GENERAL__organization-company	0.8665	0.8605	0.8635
GENERAL__organization-education	0.8256	0.8608	0.8428
GENERAL__organization-government/governmentagency	0.8344	0.8318	0.8331
GENERAL__organization-media/newspaper	0.6667	0.4	0.5
GENERAL__organization-other	0.7790	0.8105	0.7944
GENERAL__organization-religion	0.6667	0.8	0.7273
GENERAL__organization-showorganization	0.0	0.0	0.0
GENERAL__organization-sportsteam	0.0	0.0	0.0
GENERAL__other-astronomything	0.0	0.0	0.0
GENERAL__other-award	0.8216	0.8859	0.8525
GENERAL__other-biologything	0.7246	0.8961	0.8013
GENERAL__other-chemicalthing	0.7687	0.8047	0.7863
GENERAL__other-currency	0.6304	0.6744	0.6517
GENERAL__other-disease	0.8594	0.9048	0.8815
GENERAL__other-educationaldegree	0.7119	0.75	0.7304
GENERAL__other-god	0.8	0.5714	0.6667
GENERAL__other-language	0.6818	1.0	0.8108
GENERAL__other-law	0.7978	0.8462	0.8212
GENERAL__other-livingthing	0.7385	0.9320	0.8240
GENERAL__other-medical	0.7778	0.8343	0.8050
GENERAL__person-artist/author	0.625	0.3846	0.4762
GENERAL__person-other	0.8839	0.8979	0.8908
GENERAL__person-politician	0.7534	0.7432	0.7483
GENERAL__person-scholar	0.8640	0.8769	0.8704
GENERAL__person-soldier	0.7674	0.7586	0.7630
GENERAL__product-airplane	0.6774	0.6364	0.6562
GENERAL__product-car	0.9286	0.7879	0.8525
GENERAL__product-food	0.7798	0.7859	0.7828
GENERAL__product-game	0.75	0.75	0.75
GENERAL__product-other	0.7175	0.7537	0.7351
GENERAL__product-ship	0.0	0.0	0.0
GENERAL__product-software	0.8093	0.8403	0.8245
GENERAL__product-train	0.75	0.375	0.5
GENERAL__product-weapon	0.7794	0.8833	0.8281

Uses

Direct Use for Inference

python

from span_marker import SpanMarkerModel

# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("YurtsAI/named_entity_recognition_document_context")
# Run inference
entities = model.predict("monday is a chill day – beach time at barceloneta and maybe some shopping at la rambla.")

Downstream Use

You can finetune this model on your own dataset.

Click to expand

python

from span_marker import SpanMarkerModel, Trainer

# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("YurtsAI/ner-document-context")

# Specify a Dataset with "tokens" and "ner_tag" columns
dataset = load_dataset("conll2003") # For example CoNLL2003

# Initialize a Trainer using the pretrained model & dataset
trainer = Trainer(
    model=model,
    train_dataset=dataset["train"],
    eval_dataset=dataset["validation"],
)
trainer.train()
trainer.save_model("YurtsAI/named_entity_recognition_document_context-finetuned")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Sentence length	1	14.6796	691
Entities per sentence	0	0.4235	35

Training Hyperparameters

learning_rate: 1e-05
train_batch_size: 32
eval_batch_size: 32
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 64
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.1
num_epochs: 3

Training Results

Epoch	Step	Validation Loss	Validation Precision	Validation Recall	Validation F1	Validation Accuracy
0.0299	500	0.0254	0.5244	0.0116	0.0228	0.9292
0.0597	1000	0.0144	0.5380	0.3492	0.4235	0.9444
0.0896	1500	0.0099	0.7134	0.4410	0.5450	0.9534
0.1194	2000	0.0088	0.6461	0.6571	0.6516	0.9596
0.1493	2500	0.0074	0.7177	0.6363	0.6745	0.9628
0.1791	3000	0.0075	0.6612	0.7342	0.6958	0.9637
0.2090	3500	0.0073	0.6686	0.7286	0.6973	0.9634
0.2388	4000	0.0061	0.7552	0.7044	0.7289	0.9693
0.2687	4500	0.0062	0.7385	0.7150	0.7266	0.9682
0.2986	5000	0.0070	0.6667	0.7792	0.7186	0.9654
0.3284	5500	0.0063	0.6984	0.7774	0.7358	0.9689
0.3583	6000	0.0055	0.7941	0.7023	0.7454	0.9706
0.3881	6500	0.0055	0.7540	0.7640	0.7589	0.9722
0.4180	7000	0.0053	0.7700	0.7614	0.7657	0.9732
0.4478	7500	0.0053	0.7791	0.7698	0.7744	0.9742
0.4777	8000	0.0054	0.7396	0.8062	0.7715	0.9729
0.5075	8500	0.0051	0.7653	0.7944	0.7796	0.9741
0.5374	9000	0.0050	0.7773	0.7844	0.7808	0.9747
0.5672	9500	0.0049	0.7954	0.7711	0.7830	0.9757
0.5971	10000	0.0049	0.7844	0.7876	0.7860	0.9754
0.6270	10500	0.0047	0.7898	0.7940	0.7919	0.9761
0.6568	11000	0.0047	0.7852	0.7929	0.7890	0.9761
0.6867	11500	0.0047	0.8001	0.7908	0.7954	0.9770
0.7165	12000	0.0050	0.7643	0.8145	0.7886	0.9755
0.7464	12500	0.0047	0.7991	0.7892	0.7941	0.9764
0.7762	13000	0.0046	0.7948	0.8084	0.8015	0.9774
0.8061	13500	0.0046	0.7841	0.8154	0.7994	0.9771
0.8359	14000	0.0043	0.8283	0.7776	0.8021	0.9783
0.8658	14500	0.0044	0.8054	0.7993	0.8023	0.9773
0.8957	15000	0.0047	0.7704	0.8152	0.7922	0.9758
0.9255	15500	0.0043	0.8018	0.8149	0.8083	0.9782
0.9554	16000	0.0043	0.8255	0.7938	0.8093	0.9789
0.9852	16500	0.0042	0.8201	0.8008	0.8104	0.9787
1.0151	17000	0.0044	0.7947	0.8175	0.8059	0.9784
1.0449	17500	0.0044	0.7942	0.8195	0.8066	0.9777
1.0748	18000	0.0043	0.8124	0.8110	0.8117	0.9789
1.1046	18500	0.0043	0.7987	0.8157	0.8071	0.9788
1.1345	19000	0.0043	0.8037	0.8171	0.8103	0.9789
1.1644	19500	0.0042	0.8178	0.8076	0.8127	0.9796
1.1942	20000	0.0044	0.7803	0.8389	0.8085	0.9780
1.2241	20500	0.0043	0.8040	0.8210	0.8124	0.9790
1.2539	21000	0.0043	0.8038	0.8245	0.8141	0.9788
1.2838	21500	0.0041	0.8318	0.7973	0.8142	0.9794
1.3136	22000	0.0041	0.8106	0.8211	0.8158	0.9796
1.3435	22500	0.0041	0.8288	0.8046	0.8165	0.9796
1.3733	23000	0.0041	0.8218	0.8170	0.8194	0.9799
1.4032	23500	0.0042	0.8164	0.8171	0.8168	0.9799
1.4330	24000	0.0041	0.8105	0.8248	0.8176	0.9793
1.4629	24500	0.0042	0.8073	0.8196	0.8134	0.9791
1.4928	25000	0.0040	0.8211	0.8162	0.8187	0.9797
1.5226	25500	0.0040	0.8195	0.8225	0.8210	0.9800
1.5525	26000	0.0040	0.8372	0.8018	0.8191	0.9799
1.5823	26500	0.0040	0.8263	0.8161	0.8212	0.9802
1.6122	27000	0.0039	0.8275	0.8141	0.8208	0.9802
1.6420	27500	0.0040	0.8264	0.8198	0.8231	0.9804
1.6719	28000	0.0040	0.8218	0.8195	0.8206	0.9799
1.7017	28500	0.0039	0.8286	0.8195	0.8240	0.9803
1.7316	29000	0.0041	0.8004	0.8357	0.8177	0.9788
1.7615	29500	0.0040	0.8138	0.8304	0.8220	0.9801
1.7913	30000	0.0040	0.8160	0.8309	0.8234	0.9804
1.8212	30500	0.0039	0.8204	0.8262	0.8233	0.9802
1.8510	31000	0.0038	0.8292	0.8228	0.8260	0.9810
1.8809	31500	0.0039	0.8247	0.8246	0.8246	0.9806
1.9107	32000	0.0038	0.8267	0.8258	0.8262	0.9810
1.9406	32500	0.0039	0.8102	0.8398	0.8248	0.9805
1.9704	33000	0.0039	0.8321	0.8185	0.8253	0.9809
2.0003	33500	0.0038	0.8325	0.8261	0.8293	0.9814
2.0302	34000	0.0038	0.8352	0.8228	0.8289	0.9813
2.0600	34500	0.0041	0.8144	0.8369	0.8255	0.9809
2.0899	35000	0.0039	0.8274	0.8281	0.8277	0.9813
2.1197	35500	0.0039	0.8198	0.8353	0.8275	0.9812
2.1496	36000	0.0039	0.8211	0.8358	0.8284	0.9811
2.1794	36500	0.0039	0.8242	0.8300	0.8271	0.9809
2.2093	37000	0.0039	0.8194	0.8317	0.8255	0.9808
2.2391	37500	0.0039	0.8258	0.8344	0.8301	0.9814
2.2690	38000	0.0039	0.8292	0.8302	0.8297	0.9816
2.2989	38500	0.0039	0.8281	0.8315	0.8298	0.9813
2.3287	39000	0.0039	0.8174	0.8386	0.8279	0.9808
2.3586	39500	0.0039	0.8208	0.8364	0.8285	0.9810
2.3884	40000	0.0039	0.8230	0.8379	0.8304	0.9815
2.4183	40500	0.0038	0.8355	0.8273	0.8314	0.9816
2.4481	41000	0.0038	0.8290	0.8347	0.8319	0.9816
2.4780	41500	0.0038	0.8233	0.8403	0.8317	0.9815
2.5078	42000	0.0039	0.8186	0.8417	0.8300	0.9814
2.5377	42500	0.0038	0.8321	0.8343	0.8332	0.9818
2.5675	43000	0.0038	0.8239	0.8396	0.8317	0.9816
2.5974	43500	0.0038	0.8267	0.8378	0.8322	0.9816
2.6273	44000	0.0038	0.8325	0.8343	0.8334	0.9818
2.6571	44500	0.0038	0.8254	0.8399	0.8326	0.9817
2.6870	45000	0.0038	0.8339	0.8338	0.8339	0.9820
2.7168	45500	0.0038	0.8301	0.8381	0.8341	0.9819
2.7467	46000	0.0038	0.8309	0.8371	0.8340	0.9818
2.7765	46500	0.0038	0.8296	0.8377	0.8337	0.9817
2.8064	47000	0.0037	0.8337	0.8349	0.8343	0.9820
2.8362	47500	0.0037	0.8303	0.8387	0.8345	0.9820
2.8661	48000	0.0037	0.8289	0.8401	0.8344	0.9819
2.8960	48500	0.0037	0.8299	0.8400	0.8349	0.9820
2.9258	49000	0.0037	0.8289	0.8401	0.8344	0.9819
2.9557	49500	0.0037	0.8322	0.8380	0.8351	0.9821
2.9855	50000	0.0037	0.8312	0.8384	0.8348	0.9820

Framework Versions

Python: 3.11.7
SpanMarker: 1.5.0
Transformers: 4.42.1
PyTorch: 2.1.1+cu121
Datasets: 2.14.5
Tokenizers: 0.19.1

Citation

BibTeX

@software{Aarsen_SpanMarker,
    author = {Aarsen, Tom},
    license = {Apache-2.0},
    title = {{SpanMarker for Named Entity Recognition}},
    url = {https://github.com/tomaarsen/SpanMarkerNER}
}

LegionIntel/ner-document-context

SpanMarker with roberta-large on YurtsAI/named_entity_recognition_document_context