Shakespeare GPT-2
A GPT-2 model fine-tuned on Shakespeare's complete works to generate Shakespeare-style text.
Model Description
This model is a fine-tuned version of GPT-2 (124M parameters) trained on Shakespeare's complete works. It can generate text in Shakespeare's distinctive style, including dialogue, soliloquies, and dramatic prose.
Model Architecture
- Base Model: GPT-2 (124M parameters)
- Layers: 12
- Heads: 12
- Embedding Dimension: 768
- Context Length: 1024 tokens
- Total Parameters: ~124M
Training Details
- Dataset: Complete works of Shakespeare
- Training Steps: 100,000
- Batch Size: 4
- Sequence Length: 32
- Learning Rate: 3e-4
- Optimizer: AdamW
- Device: MPS/CUDA/CPU
Intended Use
This model is intended for:
- Generating Shakespeare-style text
- Creative writing assistance
- Educational purposes in literature
- Entertainment and artistic projects
Limitations
- May generate text that mimics but doesn't perfectly replicate Shakespeare's style
- Limited by training data to Shakespeare's vocabulary and themes
- Can produce anachronistic or inconsistent content
- Maximum context length of 1024 tokens
Training Data
The model was trained on Shakespeare's complete works, including:
- All plays (comedies, tragedies, histories)
- Sonnets and poems
- Total training tokens: [Insert number of tokens]
Performance
The model achieves:
- Training Loss: [Insert final training loss]
- Best Loss: [Insert best loss achieved]
Example Usage
python
from transformers import GPT2LMHeadModel, GPT2Tokenizer
Load model and tokenizer
model_name = "your-username/shakespeare-gpt"
tokenizer = GPT2Tokenizer.from_pretrained(model_name)
model = GPT2LMHeadModel.from_pretrained(model_name)
Generate text
prompt = "To be, or not to be,"
input_ids = tokenizer.encode(prompt, return_tensors='pt')
output = model.generate(
input_ids,
max_length=500,
temperature=0.8,
top_k=40,
do_sample=True
)
generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
print(generated_text)
Sample Outputs
Prompt: "To be, or not to be,"
Output: [Insert sample generation]
Prompt: "Friends, Romans, countrymen,"
Output: [Insert sample generation]
