Overview
The model is a LoRa Adaptor based on Llama-2-7b-chat-hf. The model has been trained on a
re-annotated version of the
CaRB dataset.
The model produces multi-valent Open IE tuples, i.e. relations with various numbers of arguments (1, 2, or more). We provide an example below:
Consider the following sentence (taken from the CaRB dev set):
Earlier this year , President Bush made a final `` take - it - or - leave it '' offer on the minimum wage
Our model would extract the following relation from the sentence:
<President Bush, made, a final "take-it-or-leave-it" offer, on the minimum wage, earlier this year>
where we include President Bush as the subject, made as the object, a final "take-it-or-leave-it" offer as thedirect object, and on the minimum wage and earlier this year> as salient complements.
We briefly describe how to use our model in the below, and provide further details in our
MulVOIEL repository on GithubGetting Started
Model Output Format
Given a sentence, the model produces textual predictions in the following format:
<subj> ,, (<auxi> ###) <predicate> ,, (<prep1> ###) <obj1>, (<prep2> ###) <obj2>, ...
How to Use
-
Install the relevant libraries as well as the
MulVOIEL package:
1pip install transformers datasets peft torch
2git clone https://github.com/Teddy-Li/MulVOIEL
3cd MulVOIEL
-
Load the model and perform inference (example):
1from transformers import AutoModelForCausalLM, AutoTokenizer
2from peft import PeftModel
3import torch
4from llamaOIE import parse_outstr_to_triples
5from llamaOIE_dataset import prepare_input
6
7base_model_name = "meta-llama/Llama-2-7b-chat-hf"
8peft_adapter_name = "Teddy487/LLaMA2-7b-for-OpenIE"
9
10model = AutoModelForCausalLM.from_pretrained(base_model_name)
11model = PeftModel.from_pretrained(model, peft_adapter_name)
12tokenizer = AutoTokenizer.from_pretrained(base_model_name)
13
14input_text = "Earlier this year , President Bush made a final `` take - it - or - leave it '' offer on the minimum wage"
15input_text, _ = prepare_input({'s': input_text}, tokenizer, has_labels=False)
16
17input_ids = tokenizer(input_text, return_tensors="pt").input_ids
18
19outputs = model.generate(input_ids)
20outstr = tokenizer.decode(outputs[0][len(input_ids):], skip_special_tokens=True)
21triples = parse_outstr_to_triples(outstr)
22
23for tpl in triples:
24 print(tpl)
🍺
Model Performance
The primary benefit of our model is the ability to extract finer-grained information for predicates. On the other hand, we also report performance on a roughly comparable basis with prior SOTA open IE models, where our method is comparable and even superior to prior models, while producing finer-grained and more complex outputs. We report evaluation results in (macro) F-1 metric, as well as in the average
Levenshtein Distance between gold and predicted relations:
Note that the precision and recall values are not directly comparable, because we evaluate the model prediction at a finer granularity, and we use different train/dev/test arrangements as the original CaRB dataset, hence the asterisk.
