Current methods rely on existing objects/segments, or user effort (masks/detailed text),
to localize object additions. Our approach enables free-form editing,
where the manipulated area is not well-defined, using just a Click for localization.
Try it instantly in your browser - no setup required.
Includes both Gradio interface and command line for advanced usage.
A brief glimpse into the qualitative comparison of SoTA methods —
Emu Edit,
MagicBrush
and
InstructPix2Pix
— against our model,
Click2Mask.
Upper prompts were given to baselines, and lower (shorter) ones to
Click2Mask.
Inputs contain the
Click given to
Click2Mask.
We introduce
Edited Alpha-CLIP to evaluate mask-free methods by extracting a mask of the edited region
and using
Alpha-CLIP to assess its alignment with the prompt.
Examples of mask extractions: outputs are on the left, extracted masks (green overlay) on the right.
1@inproceedings{regev2025click2mask,
2 title={Click2Mask: Local Editing with Dynamic Mask Generation},
3 author={Regev, Omer and Avrahami, Omri and Lischinski, Dani},
4 booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
5 volume={39},
6 number={7},
7 pages={6713-6721},
8 year={2025},
9 url={https://arxiv.org/abs/2409.08272},
10 note={Full version with appendices available on arXiv}
11}
Our code is based on
Blended Latent Diffusion
and
Stable Diffusion, and utilizes
AlphaCLIP (this model card contains a mirror of AlphaCLIP's weights).
