You can deploy this model and start earning money today!

Amazon-FAR/
deltatok-kinetics

Name: deltatok-kinetics
Availability: InStock

pytorch

deltatok

cvpr2026-highlight

image-feature-extraction

kinetics700

2604.04913

apache-2.0

Views

No views yet

DeltaTok (Tokenizer) — Kinetics-700

DeltaTok is a video tokenizer that compresses the frame-to-frame change in vision foundation model features into a single continuous "delta" token, as introduced in A Frame is Worth One Token: Efficient Generative World Modeling with Delta Tokens (CVPR 2026 Highlight). This approach significantly reduces the token count in video sequences (e.g., 1,024x reduction) and enables efficient generative world modeling.

This repository contains the ViT-B encoder and decoder trained on Kinetics-700 at 512x512 resolution.

Metrics

Reconstruction quality, measured by applying downstream task heads to the reconstructed features.

Method	Horizon	VSPW mIoU (↑)	Cityscapes mIoU (↑)	KITTI RMSE (↓)
Present (upper bound)	—	58.4	70.5	2.79
DeltaTok	Short (1 frame)	58.5	69.7	2.80
DeltaTok	Mid (3 frames)*	58.5	68.1	2.88

*Parallel encoding from ground-truth frames with autoregressive decoding from previous reconstructions.

Usage

Requires a frozen DINOv3 ViT-B backbone. Full training and evaluation code is available in the DeltaTok GitHub repository. To evaluate:

bash

python main.py validate -c configs/deltatok_vitb_dinov3_vitb_kinetics.yaml \
  --model.ckpt_path=path/to/deltatok-kinetics/pytorch_model.bin

Acknowledgements

Citation

bibtex

@inproceedings{kerssies2026deltatok,
  title     = {A Frame is Worth One Token: Efficient Generative World Modeling with Delta Tokens},
  author    = {Kerssies, Tommie and Berton, Gabriele and He, Ju and Yu, Qihang and Ma, Wufei and de Geus, Daan and Dubbelman, Gijs and Chen, Liang-Chieh},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2026}
}

Amazon-FAR/deltatok-kinetics