Pixel-based VDMs can generate motion accurately aligned with the textual prompt but typically demand expensive computational costs in terms of time and GPU memory, especially when generating high-resolution videos. Latent-based VDMs are more resource-efficient because they work in a reduced-dimension latent space. But it is challenging for such small latent space (e.g., 64×40 for 256×160 videos) to cover rich yet necessary visual semantic details as described by the textual prompt.

To marry the strength and alleviate the weakness of pixel-based and latent-based VDMs, we introduce Show-1, an efficient text-to-video model that generates videos of not only decent video-text alignment but also high visual quality.

This is the base model of Show-1 that generates videos with 8 keyframes at a resolution of 64x40. The model is finetuned from DeepFloyd/IF-I-L-v1.0 on the WebVid-10M and InternVid dataset.

Clone the GitHub repository and install the requirements:

Run the following command to generate a video from a text prompt. By default, this will automatically download all the model weights from huggingface.

You can also download the weights manually and change the pretrained_model_path in run_inference.py to run the inference.

If you make use of our work, please cite our paper.

This model card is maintained by David Junhao Zhang and Jay Zhangjie Wu. For any questions, please feel free to contact us or open an issue in the repository.