motion-tf-fsq64k-causal
A causal Transformer VQ-VAE motion tokenizer trained on the MotionMillion dataset.
Encodes 272-dimensional human motion sequences to discrete token indices using
Finite Scalar Quantization (FSQ) with a codebook of 64,000 entries.
Architecture
- Encoder: 6-layer causal Transformer with RoPE + 4× temporal mean-pool downsampling
- Quantizer: FSQ with levels [8,8,8,5,5,5] → 64,000 codes (deterministic, no EMA)
- Decoder: 6-layer causal Transformer with 4× learned upsampling
- Input: 272D motion (HumanML3D absolute-root representation, 20 fps)
- Parameters: 68.6M
Usage
Training
- Dataset: MotionMillion (consolidated_v1, absolute-root variant)
- Training steps: 300,000
- Codebook utilization: 99.3% (63,574 / 64,000 codes active)
- Token autocorrelation lag-1: 0.0011 (near-zero repetition)
- Normalized entropy: 0.960
