GaRNet is a generator that create non-text image with given image and coresponding text box mask. It consists of convolution encoder and decoder. The encoder consists of residual block with attention module called Gated Attention.
Gated Attention module has two Spatial attention branch. Each attention branch finds text stroke or its surrounding regions. The module adjusts the weight of these two domains by trainable parameters.
The model was trained in PatchGAN manner with Region-of-Interest Generation.
The discriminator is consists of convolution encoder. Given an image, it determines whether each patch, which indicates text-box regions, is real or fake.
All loss functions treat non-textbox regions as 'don't care'.
Intended uses & limitations
This model can be used for areas that require the process of erasing text from an image, such as concealment private information, text editing.
You can use the raw model or pre-trained model.
Note that pre-trained model was trained in both Synthetic and SCUT_EnsText dataset. And the SCUT-EnsText dataset can only be used for non-commercial research purposes.
@inproceedings{lee2022surprisingly,
title={The Surprisingly Straightforward Scene Text Removal Method with Gated Attention and Region of Interest Generation: A Comprehensive Prominent Model Analysis},
author={Lee, Hyeonsu and Choi, Chankyu},
booktitle={European Conference on Computer Vision},
pages={457--472},
year={2022},
organization={Springer}
}
