Tokenized Manhattan random-walk dataset for world modeling evaluation of autoregressive models.
Each example stores a single pretokenized sequence in input_ids (list of int32), where the sequence format is:
<start_node> <end_node> <direction_1> ... <direction_n> end
train/
.parquet: training split
heldout/.parquet: heldout/eval split
manifest.json: split sizes and export metadata
tokenizer_meta.json: tokenizer vocabulary/id… See the full description on the dataset page:
https://huggingface.co/datasets/JaydenTeoh/manhattan.
