Self-prior guided pixel adversarial networks for blind image inpainting
Wang, J. and Yuan, C. and Li, B. and Deng, Y. and Hu, W. and Maybank, Stephen (2023) Self-prior guided pixel adversarial networks for blind image inpainting. IEEE Transactions on Pattern Analysis and Machine Intelligence , ISSN 0162-8828.
|
Text
SelfPriorGuidedPixel.pdf Available under License Creative Commons Attribution. Download (11MB) | Preview |
Abstract
Blind image inpainting involves two critical aspects, i.e. "where to inpaint" and "how to inpaint". Knowing "where to inpaint" can eliminate the interference arising from corrupted pixel values; a good "how to inpaint" strategy yields high-quality inpainted results robust to various corruptions. In existing methods, these two aspects usually lack explicit and separate consideration. This paper fully explores these two aspects and proposes a self-prior guided inpainting network (SIN). The self-priors are obtained by detecting semantic-discontinuous regions and by predicting global semantic structures of the input image. On the one hand, the self-priors are incorporated into the SIN, which enables the SIN to perceive valid context information from uncorrupted regions and to synthesize semantic-aware textures for corrupted regions. On the other hand, the self-priors are reformulated to provide a pixel-wise adversarial feedback and a high-level semantic structure feedback, which can promote the semantic continuity of inpainted images. Experimental results demonstrate that our method achieves state-of-the-art perfomance in metric scores and in visual quality. It has an advantage over many existing methods that assume "where to inpaint" is known in advance. Extensive experiments on a series of related image restoration tasks validate the effectiveness of our method in obtaining high-quality inpainting.
Metadata
Item Type: | Article |
---|---|
Keyword(s) / Subject(s): | blind image inpainting, semantic-discontinuity detection, layout map prediction, pixel generative adversarial network |
School: | Birkbeck Faculties and Schools > Faculty of Science > School of Computing and Mathematical Sciences |
Depositing User: | Steve Maybank |
Date Deposited: | 17 Jul 2023 13:11 |
Last Modified: | 09 Aug 2023 12:54 |
URI: | https://eprints.bbk.ac.uk/id/eprint/51566 |
Statistics
Additional statistics are available via IRStats2.