Views
No views yet
Language model: Pegasus-xsum
Language: English
Downstream-task: Question-Answering Generation
Training data: SQuAD 2.0, NewsQA
Eval data: SQuAD 2.0, NewsQA
Infrastructure: Nvidia Tesla K80 12Gb RAMper_device_train_batch_size = 2
per_device_eval_batch_size = 2
num_train_epochs = 3
base_LM_model = "pegasus-xsum"
source_max_token_len = 256
target_max_token_len = 64
learning_rate = 5e-5
lr_schedule = LinearWarmup
warmup_steps = 1501import transformers
2from transformers import PegasusForConditionalGeneration, PegasusTokenizerFast
3
4model_name = 'nloc2578/QAG_Pegasus_3ep_eval'
5tokenizer = PegasusTokenizerFast.from_pretrained(model_name)
6model = PegasusForConditionalGeneration.from_pretrained(model_name, pad_token_id=tokenizer.eos_token_id)
7
8text = '''The primary goal of distractor generation is generating answer
9options that are plausibly answers to the question, and might appear
10correct to a user who does know the correct answer. Distractors
11should also be clearly distinct from the key and each other and
12they should not be correct answers to the question (for questions
13that might have multiple correct answers).'''
14
15input_id = tokenizer(text, return_tensors='pt')
16output = model.generate(input_id['input_ids'])
17result = tokenizer.decode(output[0])
18
19print(result)