Description: “A small GPT model (~1-2M parameters) trained on 10,000 TinyStories samples for text generation.”
Training Details: Dataset (TinyStories), model (2 layers, 128 hidden size, 4 heads), 3 epochs, trained on GTX 1650.
Example Usage:
python
from transformers import AutoTokenizer, AutoModel
from mini_gpt import MiniGPT, MiniGPTConfig
AutoModel.register(MiniGPTConfig, MiniGPT)
model = MiniGPT.from_pretrained("abhishekpoddar/mini-gpt")
tokenizer = AutoTokenizer.from_pretrained("abhishekpoddar/mini-gpt")
inputs = tokenizer("Once upon a time", return_tensors="pt")
outputs = model.generate(inputs["input_ids"], max_length=50)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
license: apache-2.0
