from latentsync.pipelines.lipsync_pipeline import LipsyncPipeline

pipeline = LipsyncPipeline.from_pretrained(
    "ByteDance/LatentSync-1.5",       # base components
    unet_path="FrankLiuDundun/latentsync-finetune-lora-v1/latentsync_unet.pt",   # this merged file
)

or via the Gradio Fine-tune Studio (gradio_finetune.py) by selecting this repo's latentsync_unet.pt as the inference checkpoint.

Files in this repo

latentsync_unet.pt — drop-in replacement for the base UNet
adapter/ — original peft-format LoRA adapter (if you want to keep training)
unet_config.yaml — UNet architectural config
scheduler/ — DDIM scheduler

Provenance

LoRA rank: 48
Base checkpoint: checkpoints/latentsync_unet.pt
LoRA adapter: /root/autodl-tmp/latentsync_finetune/unet/train_lora-2026_07_20-14:03:57/checkpoints/checkpoint-4500
UNet config: configs/unet/stage2.yaml

Caveats

This model is a community finetune of the base LatentSync checkpoint. Use the same audio-video preprocessing as the base model. No guarantees are made about identity preservation, side-face robustness, or lip-sync on out-of-distribution sources.