LSTM Time Capsule is a modern small-compute retest of long-lag credit assignment.
Every sequence contains random values and two distant markers; the model must add
only the marked values after processing the entire sequence. A parameter-matched
vanilla tanh RNN, an LSTM, and a GRU train on identical data and are evaluated at
the training length and at longer unseen lengths.
The historical anchor is Hochreiter and Schmidhuber's 1997
Long Short-Term Memory, which… See the full description on the dataset page:
https://huggingface.co/datasets/ARotting/long-lag-adding.
