TwiFF is a unified model fine-tuned on a high-quality dynamic visual Chain-of-Thought (VCoT) dataset comprising 2.7 million samples. In dynamic multimodal question-answering tasks involving instructional, predictive, and camera motion, TwiFF iteratively generates future event frames alongside textual reasoning, thereby producing temporally coherent visual reasoning trajectories.
Experimental results demonstrate that, on dynamic scenario reasoning benchmarks, our dynamic VCoT approach outperforms both static VCoT methods based on tool-calling paradigms and purely textual chain-of-thought baselines.
To use TwiFF, follow the instructions below derived from the
official repository.
1git clone https://github.com/LiuJunhua02/TwiFF.git
2cd TwiFF
3conda create -n TwiFF python=3.10 -y
4conda activate TwiFF
5pip install -r requirements.txt
6pip install flash_attn==2.5.8 --no-build-isolation
1from huggingface_hub import snapshot_download
2
3save_dir = "models/TwiFF-7B"
4repo_id = "Liu-Junhua/TwiFF-7B"
5cache_dir = save_dir + "/cache"
6
7snapshot_download(cache_dir=cache_dir,
8 local_dir=save_dir,
9 repo_id=repo_id,
10 local_dir_use_symlinks=False,
11 resume_download=True,
12 allow_patterns=["*.json", "*.safetensors", "*.bin", "*.py", "*.md", "*.txt"],
13)
1python \
2 scripts/inference.py \
3 --max_round 8 \
4 --model_dir models/TwiFF-7B \
5 --checkpoint_file model.safetensors \
6 --checkpoint_dir models/TwiFF-7B \
7 --QA_file output/demo.jsonl \
8 --seed 42
1@article{liu2026twiff,
2 title={TwiFF (Think With Future Frames): A Large-Scale Dataset for Dynamic Visual Reasoning},
3 author={Liu, Junhua and Wang, Zhangcheng and Han, Zhike and Wang, Ningli and Liang, Guotao and Kuang, Kun},
4 journal={arXiv preprint arXiv:2602.10675},
5 year={2026},
6}
TwiFF is licensed under the Apache 2.0.
