VideoPrism is a family of foundational video-encoder models from Google Research, designed to be a universal "prism" for understanding the diverse facets of video content. Built on a massive scale of 36 million high-quality video-caption pairs and 582 million video clips, VideoPrism is engineered to excel across a wide range of video understanding tasks, including classification, localization, retrieval, and captioning. VideoPrism models utilize a Vision Transformer (ViT) architecture and are pre-trained using a combination of video-text contrastive learning and masked video modeling. This dual approach allows the model to capture both global semantic meaning and fine-grained spatio-temporal details, making it a powerful backbone for state-of-the-art video AI applications.

Keras and KerasHub can be installed with:

pip install -U -q keras-hub
pip install -U -q keras>=3

JAX, TensorFlow, and Torch come pre-installed in Kaggle Notebooks. For instructions on installing them in another environment, see the Keras Getting Started page.

The following model checkpoints are provided by the Keras team. For the Video-Text (LvT) variants, both the video encoder and the text encoder are provided to enable multimodal tasks like zero-shot retrieval.