Views
No views yet
libx264 encoder for H.264 pose-video output. The Python dependencies also install an imageio-ffmpeg fallback.NPROC_PER_NODE=1, subject to available VRAM.1git clone https://github.com/yyang181/LatentDance.git
2cd LatentDance
3
4conda create -n latentdance python=3.10 -y
5conda activate latentdance
6
7pip install torch==2.8.0 torchvision==0.23.0 \
8 --index-url https://download.pytorch.org/whl/cu128
9pip install -e ".[multi-gpu]"
10
11# Recommended for faster attention on NVIDIA GPUs.
12FLASH_ATTENTION_FORCE_BUILD=TRUE pip install flash_attn --no-build-isolation
13
14# Optional when a system ffmpeg is not already available. This provides libx264.
15conda install -c conda-forge ffmpegffmpeg -hide_banner -encoders | grep libx264libx264 encoder, pose extraction will write H.264 (yuv420p) MP4 files. When no system ffmpeg is found, the imageio[ffmpeg] dependency installed above supplies a bundled encoder automatically.1mkdir -p models/Wan-AI models/LatentDance
2
3HF_HUB_ENABLE_HF_TRANSFER=1 hf download Wan-AI/Wan2.2-I2V-A14B \
4 --local-dir models/Wan-AI/Wan2.2-I2V-A14B
5
6HF_HUB_ENABLE_HF_TRANSFER=1 hf download yyang181/LatentDance \
7 --local-dir models/LatentDance1models/
2├── LatentDance/
3│ ├── latentdance_high_noise.safetensors
4│ └── latentdance_low_noise.safetensors
5└── Wan-AI/
6 └── Wan2.2-I2V-A14B/
7 └── ...pip install -e ".[pose]"pose extra installs ONNX Runtime with CUDA support. For CPU-only preprocessing, replace it with the CPU build:1pip uninstall -y onnxruntime-gpu
2pip install onnxruntimevideo/, then run preprocessing:1data/evaldata/
2├── input_image/
3│ └── 000.png
4└── video/
5 └── 000.mp41CUDA_VISIBLE_DEVICES=0 python scripts/extract_pose.py \
2 --input_dir data/evaldata \
3 --body_score_threshold 0.3yolox_l.onnx and dw-ll_ucoco_384.onnx from yzd-v/DWPose. In an offline environment, download them in advance and pass --detector_model /path/to/yolox_l.onnx --pose_model /path/to/dw-ll_ucoco_384.onnx. Add --cpu to explicitly disable CUDA or --overwrite to regenerate existing results.video/<name>.mp4, preprocessing writes:1data/evaldata/
2├── pose/<name>.mp4 # H.264 DWPose rendering (yuv420p), with input FPS and resolution
3└── pose_keypoints/<name>.npz # bodies_candidate: float32 [frames, 128, 3](x, y, confidence). The 128 points are ordered as 18 body joints, 68 face landmarks, 21 right-hand joints, and 21 left-hand joints, exactly matching the released inference loader. Input videos may be nested under video/; the same relative structure is preserved in both output directories.1data/evaldata/
2├── input_image/
3│ └── 000.png
4├── video/
5│ └── 000.mp4
6├── pose/
7│ └── 000.mp4
8├── pose_keypoints/
9│ └── 000.npz
10└── pllava_caption/
11 └── caption.csvpath and text columns. For example:1path,text
2video/000.mp4,"A person is dancing."CUDA_VISIBLE_DEVICES=0,1,2,3 NPROC_PER_NODE=4 bash scripts/batch_test.shEVAL_LIMIT=1 CUDA_VISIBLE_DEVICES=0,1,2,3 NPROC_PER_NODE=4 bash scripts/batch_test.shoutput/ by default. Set OUTPUT_DIR=/path/to/output to override the output directory, EVALUATION_PATH=/path/to/data to use another evaluation set, or CHUNK_SIZE=161 to change the temporal chunk size.1pip install -e ".[demo,pose,multi-gpu]"
2python demo.pyhttp://127.0.0.1:7860 in your browser. The demo uses two stages:.npz keypoints file. Final generation uses the reference image, the rendered pose control video, and the selected .npz keypoints..npz file in the keypoints input before generating..npz still takes priority for keypoints.models/LatentDance/latentdance_high_noise.safetensors and models/LatentDance/latentdance_low_noise.safetensors; they can be changed in the UI. A CUDA-capable GPU, the pose extraction dependencies, xFuser for multi-GPU inference, and the Wan2.2-I2V-A14B base model are required.