This modelcard provides details about a Video-to-Video (V2V) transformer model trained on human workout videos for video synthesis and transformation tasks. It includes model usage, training details, evaluation results, and guidance for deployment.
This Video-to-Video (V2V) model leverages a transformer-based autoencoder to perform video synthesis by learning latent representations from video sequences. It can take one or multiple input videos and generate a corresponding transformed output video. Designed for tasks like action transfer, video interpolation, and visual enhancement.
This model can be used directly to perform video-to-video transformations such as:
1from video_to_video_model import VideoFusionAutoencoder
2import torch
3
4device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
5model = VideoFusionAutoencoder(embed_dim=512, frame_size=128).to(device)
6model.load_state_dict(torch.load("checkpoint.pth", map_location=device))
7model.eval()
8
9# Pass in input video tensor of shape [N, T, C, H, W]
10output_video, latent = model(input_video_tensor)
