This repository contains the model checkpoints used in the paper
"Chapter-Llama: Efficient Chaptering in Hour-Long Videos with LLMs" (CVPR 2025).
Chapter-Llama is based on fine-tuned Llama-3.1-8B-Instruct with LoRA adapters. We provide three main model variants:
-
asr-10k: Model trained with ASR from 10k videos of the VidChapter-7M dataset
- Used for our Speech-based frame selector
- Input: Only speech transcripts with timestamps
-
captions_asr-10k: Model trained with Captions+ASR from 10k videos
- Our primary model used for most experiments
- Input: Both speech transcripts and visual captions with timestamps
-
captions_asr-1k: Model trained with Captions+ASR from 1k videos
- Smaller training set variant
- Input: Both speech transcripts and visual captions with timestamps
Our best model achieves 45.3 F1 score on the VidChapters-7M benchmark, substantially outperforming previous state-of-the-art methods.
The models can be downloaded and used with the
Chapter-Llama codebase:
1# Download model LoRA adapters
2python tools/download/models.py "asr-10k" --local_dir "."
3python tools/download/models.py "captions_asr-10k" --local_dir "."
4python tools/download/models.py "captions_asr-1k" --local_dir "."
5
6# Inference on a single video
7python inference.py /path/to/your/video.mp4
1@article{ventura25chapter,
2 title = {{Chapter-Llama}: Efficient Chaptering in Hour-Long Videos with {LLM}s},
3 author = {Lucas Ventura and Antoine Yang and Cordelia Schmid and G{\"u}l Varol},
4 journal = {CVPR},
5 year = {2025}
6}
These models are distributed under an MIT License. Please check the
repository for more details.
