This model is intended for lightweight English story generation experiments, educational use, and local text-generation testing. It is a very small model and should not be expected to perform broad instruction following, factual answering, or robust reasoning.

Usage

python

from transformers import AutoTokenizer, GPT2LMHeadModel
import torch

model_id = "StoryBananaMind-7.5M"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = GPT2LMHeadModel.from_pretrained(model_id).eval()

prompt = "Once upon a time, there was a little cat named Lily."
inputs = tokenizer(prompt, return_tensors="pt")

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=120,
        temperature=0.8,
        top_p=0.95,
        do_sample=True,
        repetition_penalty=1.1,
        pad_token_id=tokenizer.eos_token_id,
    )

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training

The model was trained from scratch with a custom GPT-2 tokenizer trained to an 8,192-token vocabulary. Training used fixed 256-token blocks from TinyStories.

Key configuration:

n_layer: 5
n_embd: 288
n_head: 6
n_inner: 1152
n_positions: 256
vocab_size: 8192

Files

model.safetensors: model weights
config.json: Transformers model configuration
generation_config.json: default generation configuration
tokenizer.json: tokenizer
tokenizer_config.json: tokenizer metadata
storybananamind-preview.png: model card preview image

BananaMind/StoryBananaMind-7.5M

StoryBananaMind-7.5M

Model Details

Intended Use

Usage

Training

Files

BananaMind/
StoryBananaMind-7.5M