You can deploy this model and start earning money today!

takesuke/
SIFT-VTON

Name: SIFT-VTON
Rating: 2.5 (48 reviews)

Views

No views yet

SIFT-VTON: Geometric Correspondence Supervision on Cross-Attention for Virtual Try-On

ICPR 2026

SIFT-VTON is a diffusion-based virtual try-on model that uses SIFT feature correspondences between a garment image and a person image to supervise cross-attention maps during training, improving geometric alignment in the generated results.

Paper: arXiv:2605.01296

This model is derived from StableVITON and built on a Stable Diffusion backbone.

The code repository is available at takesukeDS/SIFT-VTON.

Model Files

File	Description
`model.ckpt`	Model checkpoint
`config.yaml`	Model architecture config

Requirements

Clone the code repository and set up the environment:

bash

git clone https://github.com/takesukeDS/SIFT-VTON
cd SIFT-VTON

conda create -n siftvton python==3.12.8 -y
conda activate siftvton

pip install torch==2.7.1 torchvision==0.22.1 --index-url https://download.pytorch.org/whl/cu128
pip install matplotlib einops omegaconf yacs
pip install pytorch-lightning==2.5.2
pip install open-clip-torch==3.1.0
pip install diffusers==0.34.0
pip install scipy==1.16.1
pip install transformers==4.55.0
conda install -c anaconda ipython -y
pip install scikit-image clean-fid albumentations==2.0.8
pip3 install -U xformers==0.0.31.post1
pip install tensorboard
pip install accelerate==1.10.0
pip install numpy==2.2.6

Data

Download the VITON-HD dataset and prepare the following directory structure:

[data_root_dir]
└── test
    |-- image
    |-- image-densepose
    |-- agnostic-v3.2
    |-- agnostic-mask
    |-- cloth
    |-- cloth-mask

A pairs file yahavton_test_pairs.txt is also required under [data_root_dir], listing image and cloth filenames one pair per line:

image_00001.jpg cloth_00001.jpg
image_00002.jpg cloth_00002.jpg
...

Inference

bash

python inference_hf.py \
    --repo_id takesukeDS/SIFT-VTON \
    --data_root_dir [data_root_dir] \
    --save_dir [output_dir] \
    --phase test \
    --batch_size 4 \
    --start_from_noised_agn \
    --cfg_scale 1.5 \
    --repaint

The model and config are downloaded automatically from this Hub repository on the first run and cached locally under ~/.cache/huggingface/hub/.

Key inference arguments

Argument	Default	Description
`--repo_id`	—	This Hub repo (`takesukeDS/SIFT-VTON`)
`--phase`	`test`	`test` for the test split, `train` for the training split
`--cfg_scale`	`1.0`	Classifier-free guidance scale
`--denoise_steps`	`50`	Number of PLMS denoising steps
`--start_from_noised_agn`	off	Start denoising from noised agnostic image instead of pure noise (recommended)
`--repaint`	off	Paste back the unmasked region from the original image after generation (recommended)
`--unpair`	off	Run unpaired inference (person and garment from different samples)
`--batch_size`	`16`	Batch size
`--seed`	`1235`	Random seed

Citation

bibtex

@misc{takemoto2026siftvton,
  title         = {{SIFT-VTON}: Geometric Correspondence Supervision on Cross-Attention for Virtual Try-On},
  author        = {Takemoto, Kosuke and Koshinaka, Takafumi},
  year          = {2026},
  eprint        = {2605.01296},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2605.01296}
}

License

Licensed under the CC BY-NC-SA 4.0 license.

takesuke/SIFT-VTON