RinKana/

Amadeus-RNN-4.7M-B

Amadeus-RNN-4.7M-B A lightweight linear RNN with data-dependent gating and parallel associative scan (SSM-style training). Trained on TinyStories. 4.7M parameters, runs on consumer GPU (<2GB VRAM).

Architecture: Linear RNN · O(T log T) associative scan · Mamba-inspired λ gate · Custom BPE tokenizer (vocab=4096)

Training: TinyStories · 2-3hrs · RTX 3060 Mobile · PyTorch + AMP