You can deploy this model and start earning money today!

prokaggler7/
slm-gpt2

Name: slm-gpt2
Availability: InStock

Views

No views yet

🧠 SLM-GPT2: Tiny Shakespeare GPT-2 Model

SLM-GPT2 is a small GPT-2-like language model trained from scratch on the Tiny Shakespeare dataset. It’s a toy model meant for educational purposes, experimentation, and understanding how transformer-based language models work.

✨ Model Details

Architecture: GPT-2 (custom config)
Layers: 4
Hidden size: 256
Heads: 4
Max sequence length: 128
Vocabulary size: Same as tokenizer (based on distilgpt2 or custom)
Training epochs: 3
Dataset: tiny_shakespeare

🧪 Intended Use

Educational demos
Debugging/training pipeline validation
Low-resource inference tests
Not suitable for production or accurate text generation

🚫 Limitations

Trained on a tiny dataset (~100 KB)
Limited vocabulary and generalization
Can generate incoherent or biased outputs
Not safe for deployment in real-world applications

💻 How to Use

python

from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline

model = AutoModelForCausalLM.from_pretrained("your-username/slm-gpt2")
tokenizer = AutoTokenizer.from_pretrained("your-username/slm-gpt2")

generator = pipeline("text-generation", model=model, tokenizer=tokenizer)
output = generator("To be or not to be", max_length=50)
print(output[0]['generated_text'])

prokaggler7/slm-gpt2