It is possible to use this model to generate text, which is useful for experimentation and understanding its capabilities. It should not be directly used for production or work that may directly impact people.

How to Use

You can very easily load the models with Transformers, instead of downloading them manually. The T5-Large model is the backbone of our model. Here is how to use the model in PyTorch:

python

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("him1411/EDGAR-T5-Large")
model = AutoModelForSeq2SeqLM.from_pretrained("him1411/EDGAR-T5-Large")

Or just clone the model repo

git lfs install
git clone https://huggingface.co/him1411/EDGAR-T5-Large

Inference Example

Here, we provide an example for the "ContextNER" task. Below is an example of one instance.

python

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("him1411/EDGAR-T5-Large")
model = AutoModelForSeq2SeqLM.from_pretrained("him1411/EDGAR-T5-Large")
# Input shows how we have appended instruction from our file for HoC dataset with instance.
input = "14.5 years . The definite lived intangible assets related to the contracts and trade names had estimated weighted average useful lives of 5.9 years and 14.5 years, respectively, at acquisition."
tokenized_input= tokenizer(input)
# Ideal output for this input is 'Definite lived intangible assets weighted average remaining useful life'
output = model(tokenized_input)

Results on Dowstream datasets

EDGAR-T5-Large was finetuned on some downstream datasets to get better results than T5 large. BloombergGPT 50B was used as baseline.

Dataset	Bloomberg GPT 50B	T5 Large	Edgar T5 Large
FiQA SA	75.07	74.89	80.42
FPB	51.07	55.77	79.69
Headline	82.20	90.55	93.55

BibTeX Entry and Citation Info

If you are using our model, please cite our paper:

bibtex

@article{gupta2021context,
  title={Context-NER: Contextual Phrase Generation at Scale},
  author={Gupta, Himanshu and Verma, Shreyas and Kumar, Tarun and Mishra, Swaroop and Agrawal, Tamanna and Badugu, Amogh and Bhatt, Himanshu Sharad},
  journal={arXiv preprint arXiv:2109.08079},
  year={2021}
}