DATAGRID-Open-Sora-Plan-v1.3.0-0.16M

Model Details

• Developed by: DATAGRID Inc.
• Model type: Text-to-Video, Inpainting
• Languages: English
• License: Apache 2.0
• Finetuned from model: Open-Sora-Plan-v1.3.0

Model Description

Training Details

• Training Data: Fine-tuned on a custom dataset of 0.16 million royalty-free video-text pairs. This dataset was independently collected and curated by DATAGRID Inc., focusing on diverse scenes, motions, and objects. For V2V inpainting training data preparation, we built an automated mask generation pipeline utilizing state-of-the-art models like Meta AI's SAM2 (Segment Anything Model 2) and Microsoft's Florence2 to automatically generate masks for target objects in videos. This significantly improved efficiency and reduced costs compared to traditional manual annotation methods.

Inference Details

Results

T2V

category	prompt	Open-Sora-Plan-v1.3.0	DATAGRID-Open-Sora-Plan-v1.3.0-0.16M
dynamic degree	an airplane accelerating to gain speed
object class	a bicycle
human action	A person is ice skating
color	A pink bird
imaging quality	this is how I do makeup in the morning
spatial relationship	a kite on the top of a skateboard, front view

V2V(Inpainting)

original video	prompt (short)	mask	DATAGRID-Open-Sora-Plan-v1.3.0-0.16M(Inpaint)
	A juicer pours orange juice into a container. ・・・Morning juice-making scene.
	A reddish-brown cat resting in a grassy field, ・・・relaxed and content.

License

Citation