We discretized Copenhagen into a 2048x2048 grid (~7m per cell) and trained GPT-2 to autocomplete walking routes as token sequences. Each coordinate becomes two tokens (row, column). The model learns to generate plausible walking paths without any knowledge of the road network.

Training

Data: 100,000 random walking routes from Valhalla pedestrian routing
Tokenization: 2048x2048 grid over Copenhagen (55.60-55.75N, 12.45-12.70E), ~7m resolution
Architecture: GPT-2 (vocab=4100, layers=6, dim=256, heads=8, positions=512)
Training: 50 epochs on rented A100 via Google Colab, ~1 hour
Final validation loss: 0.19

Files

grid/ - Grid tokenization model (PyTorch, HuggingFace format)
- config.json, model.safetensors, generation_config.json
grid/onnx/model.onnx - ONNX export for browser inference (27MB)

Token format

[BOS, start_lat, start_lng, end_lat, end_lng, SEP, wp1_lat, wp1_lng, wp2_lat, wp2_lng, ..., EOS]

Tokens 0-2047: latitude bins
Tokens 2048-4095: longitude bins
Token 4096: BOS
Token 4097: EOS
Token 4098: SEP
Token 4099: PAD

Usage

python

from transformers import GPT2LMHeadModel, GPT2Config
import torch

config = GPT2Config.from_pretrained("flipbitsnotburgers/routegpt-copenhagen", subfolder="grid")
model = GPT2LMHeadModel.from_pretrained("flipbitsnotburgers/routegpt-copenhagen", subfolder="grid")

# Encode start/end coordinates
GRID_SIZE = 2048
LAT_MIN, LAT_MAX = 55.60, 55.75
LNG_MIN, LNG_MAX = 12.45, 12.70
BOS, EOS, SEP = 4096, 4097, 4098

def encode(lat, lng):
    lat_bin = int((lat - LAT_MIN) / (LAT_MAX - LAT_MIN) * (GRID_SIZE - 1))
    lng_bin = int((lng - LNG_MIN) / (LNG_MAX - LNG_MIN) * (GRID_SIZE - 1)) + GRID_SIZE
    return lat_bin, lng_bin

s_lat, s_lng = encode(55.67, 12.57)
e_lat, e_lng = encode(55.68, 12.58)

prompt = torch.tensor([[BOS, s_lat, s_lng, e_lat, e_lng, SEP]])
output = model.generate(prompt, max_new_tokens=300, temperature=0.6, top_k=20, eos_token_id=EOS)

Limitations

Only works in Copenhagen (trained exclusively on Copenhagen routes)
Does not follow roads (routes cut through buildings, parks, water)
8M parameters is not enough to memorize a city's street network
The hint mechanism (feeding real waypoints as prompt) dramatically improves quality

Part of Shady Baby

Shady Baby is a stroller-friendly walking app for Copenhagen. RouteGPT was an experiment in replacing traditional routing with neural sequence generation.