Views
No views yet
1>>> from transformers import AutoTokenizer, AutoModelForCausalLM
2>>> model_name = "Sharathhebbar24/t5_reviews_tuned"
3>>> model = T5ForConditionalGeneration.from_pretrained(model_name)
4>>> tokenizer = T5Tokenizer.from_pretrained("t5-small")
5>>> def generate_review(text):
6>>> inputs = tokenizer("review: " + text, return_tensors="pt", max_lenght=512, padding='max_length', truncation=True,)
7>>> outputs = mod1.generate(inputs['input_ids'], max_length=128, no_repeat_ngram_size=3, num_beams=6, early_stopping=True)
8>>> summary = tokenizer.decode(outputs[0], skip_special_tokens=True)
9>>> return summary
10
11>>> generate_text("Android 3 Stars!!")
12
13It's goodWith T5, we propose reframing all NLP tasks into a unified text-to-text format where the input and output are always text strings, in contrast to BERT-style models that can only output either a class label or a span of the input. Our text-to-text framework allows us to use the same model, loss function, and hyperparameters on any NLP task.
Our text-to-text framework allows us to use the same model, loss function, and hyperparameters on any NLP task, including machine translation, document summarization, question answering, and classification tasks (e.g., sentiment analysis). We can even apply T5 to regression tasks by training it to predict the string representation of a number instead of the number itself.
In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks.
1@article{2020t5,
2 author = {Colin Raffel and Noam Shazeer and Adam Roberts and Katherine Lee and Sharan Narang and Michael Matena and Yanqi Zhou and Wei Li and Peter J. Liu},
3 title = {Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer},
4 journal = {Journal of Machine Learning Research},
5 year = {2020},
6 volume = {21},
7 number = {140},
8 pages = {1-67},
9 url = {http://jmlr.org/papers/v21/20-074.html}
10}1from transformers import T5Tokenizer, T5Model
2
3tokenizer = T5Tokenizer.from_pretrained("t5-small")
4model = T5Model.from_pretrained("t5-small")
5
6input_ids = tokenizer(
7 "Studies have been shown that owning a dog is good for you", return_tensors="pt"
8).input_ids # Batch size 1
9decoder_input_ids = tokenizer("Studies show that", return_tensors="pt").input_ids # Batch size 1
10
11# forward pass
12outputs = model(input_ids=input_ids, decoder_input_ids=decoder_input_ids)
13last_hidden_states = outputs.last_hidden_state