tiny_shakespeare
model_description: |
This is a small autoregressive language model based on the Transformer architecture trained on the Tiny Shakespeare dataset.
The model is a custom implementation of a TransformerDecoderModel, which uses a decoder-only architecture similar to GPT-2.
It was trained on the Tiny Shakespeare dataset to generate text in the style of William Shakespeare.
The model was trained and tracked using
Weights & Biases.
1from transformers import AutoTokenizer
2from transformers import GPT2LMHeadModel
3
4# Load the model and tokenizer
5model = GPT2LMHeadModel.from_pretrained('NataliaH/TransformerDecoderModel')
6tokenizer = AutoTokenizer.from_pretrained('NataliaH/TransformerDecoderModel')
7
8# Provide input text and generate output
9input_text = 'To be or not to be'
10inputs = tokenizer(input_text, return_tensors='pt')
11outputs = model.generate(**inputs)
12print(tokenizer.decode(outputs[0], skip_special_tokens=True))
