You can deploy this model and start earning money today!

MyNiuuu/
MOFA-Video-Traj

Name: MOFA-Video-Traj
Availability: InStock

Views

No views yet

Updates 🔥🔥🔥

We have released the Gradio demo for Hybrid (Trajectory + Landmark) Controls HERE!

Introduction

This repo provides the inference Gradio demo for Trajectory Control of MOFA-Video.

Environment Setup

pip install -r requirements.txt

Download checkpoints

Download the pretrained checkpoints of SVD_xt from huggingface to ./ckpts.
Download the checkpint of MOFA-Adapter from huggingface to ./ckpts.

The final structure of checkpoints should be:

text

./ckpts/
|-- controlnet
|   |-- config.json
|   `-- diffusion_pytorch_model.safetensors
|-- stable-video-diffusion-img2vid-xt-1-1
|   |-- feature_extractor
|       |-- ...
|   |-- image_encoder
|       |-- ...
|   |-- scheduler
|       |-- ...
|   |-- unet
|       |-- ...
|   |-- vae
|       |-- ...
|   |-- svd_xt_1_1.safetensors
|   `-- model_index.json

Run Gradio Demo

python run_gradio.py

Please refer to the instructions on the gradio interface during the inference process.

Paper

arxiv.org/abs/2405.20222

MyNiuuu/MOFA-Video-Traj