AetherStory is a tiny decoder-only transformer (GPT-style) trained entirely on CPU on the procedurally generated AetherStory dataset. It writes short fantasy fables given an opening prompt.

This is not a fine-tune of a larger model and not a wrapper around transformers — every layer is implemented by hand in plain PyTorch.

 token + position embeddings
            |
   +----------------+
   | Transformer x4 |
   |  causal MHA     |
   |  GELU FFN       |
   +----------------+
            |
        LayerNorm
            |
   tied output head

Trained with AdamW (lr 3e-4, cosine schedule, warmup 200) for 4 epochs on a 4-core CPU. Best validation loss: 0.3103, trained in unknown (recovered) on CPU.

A ~2M-parameter model trained on synthetic fables will not produce literature. It will produce charming, sometimes incoherent, fairy-tale-flavoured text — which is the point. It is a demonstration that a small, fully custom model can be trained, evaluated, and shipped end-to-end on commodity hardware.

MIT.