MotionGPT is the motion-language baseline from MotionGPT: Human Motion as a
Foreign Language (Jiang et al., NeurIPS 2023). This Motius release packages
the HumanML3D motion tokenizer, FLAN-T5-base-style language model with motion
tokens, HumanML3D statistics, and task-facing text-to-motion / motion-to-text
pipeline methods without requiring the original checkout.
Preview
HumanML3D Sample
Input Text
SMPL Preview
001840
someone executes a roundhouse kick with their left foot.
MotionGPT HumanML3D 001840 SMPL demo
004545
a person jumping while raising both hands and moving apart legs.
MotionGPT HumanML3D 004545 SMPL demo
006944
a person moves their right hand left, right, up, and down.
MotionGPT HumanML3D 006944 SMPL demo
512px / 30fps GIF previews rendered from released HumanML3D test outputs.
Release Snapshot
Item
Value
Method
MotionGPT, language modeling over text and motion tokens
Tasks
T2M, M2T
Venue
NeurIPS 2023
Motion representation
HumanML3D-263, 20 fps
Language backbone
FLAN-T5-base-style encoder-decoder with motion tokens
The checkpoint artifact contains motiongpt_s3_h3d.tar,
assets/meta/mean.npy, assets/meta/std.npy, deps/flan-t5-base/, and
model_index.json.
Usage
python
1from motius.pipelines.motiongpt import MotionGPTPipeline
23pipe = MotionGPTPipeline.from_pretrained(4"ZeyuLing/Motius-MotionGPT-HumanML3D",5 bundle_kwargs={"local_files_only":False},6 device="cuda",7)89motions = pipe.infer_t2m(10["a person walks forward then sits down"],11[120],12)
motions is a list of NumPy arrays. Each array has shape (T, 263) and is
denormalized to HumanML3D physical scale. The same pipeline also exposes
infer_m2t for captioning denormalized HumanML3D-263 motions.
Protocol: HumanML3D Official uses the selected-caption HumanML3D test protocol. MotionStreamer Evaluator and Motius Joint-Position Evaluator are computed after converting outputs through the shared SMPL-22 evaluation bridge. For FID and MM-Dist, lower is better.
MotionGPT generates HumanML3D-263 features at 20 fps. Per frame:
Slice
Dim
Meaning
root_rot_vel
1
root angular velocity
root_lin_vel
2
root linear velocity in the horizontal plane
root_y
1
root height
ric_data
63
local joint positions
rot_data
126
local joint rotations in continuous 6D format
local_vel
66
local joint velocities
foot_contact
4
binary foot-contact labels
The VQ-VAE converts normalized HumanML3D features into discrete motion tokens.
MotionGPT then treats those tokens as a language vocabulary item alongside text
tokens.
Motius Components
Component
Path
Pipeline
motius.pipelines.motiongpt.MotionGPTPipeline
Bundle
motius.models.motiongpt.MotionGPTBundle
Runtime
motius.models.motiongpt.network.mGPT.archs
Only the inference-time MotionGPT modules required by the bundle are included
in this public package.
Citation
bibtex
1@inproceedings{jiang2023motiongpt,
2 title={MotionGPT: Human Motion as a Foreign Language},
3 author={Jiang, Biao and Chen, Xin and Liu, Wen and Yu, Jingyi and Yu, Gang and Chen, Tao},
4 booktitle={Advances in Neural Information Processing Systems},
5 year={2023}
6}
