Views
No views yet
from transformers import (
AutoModelForSeq2SeqLM,
AutoTokenizer
)
model_checkpoint = "consciousAI/question-generation-auto-t5-v1-base-s-q-c"
model = AutoModelForSeq2SeqLM.from_pretrained(model_checkpoint)
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
## Input with prompt
context="question_context: <context>"
encodings = tokenizer.encode(context, return_tensors='pt', truncation=True, padding='max_length').to(device)
## You can play with many hyperparams to condition the output, look at demo
output = model.generate(encodings,
#max_length=300,
#min_length=20,
#length_penalty=2.0,
num_beams=4,
#early_stopping=True,
#do_sample=True,
#temperature=1.1
)
## Multiple questions are expected to be delimited by '?' You can write a small wrapper to elegantly format. Look at the demo.
questions = [tokenizer.decode(id, clean_up_tokenization_spaces=False, skip_special_tokens=False) for id in output]| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum |
|---|---|---|---|---|---|---|---|
| 1.8732 | 1.0 | 942 | 1.4330 | 0.5558 | 0.3857 | 0.5219 | 0.5231 |
| 1.2361 | 2.0 | 1884 | 1.3720 | 0.5649 | 0.4042 | 0.5332 | 0.5338 |
| 0.9515 | 3.0 | 2826 | 1.3703 | 0.5699 | 0.4044 | 0.5373 | 0.5385 |
| 0.7383 | 4.0 | 3768 | 1.4039 | 0.5753 | 0.4159 | 0.5414 | 0.5426 |
| 0.6291 | 5.0 | 4710 | 1.4661 | 0.5809 | 0.4227 | 0.5488 | 0.5498 |