Self-prior guided pixel adversarial networks for blind image inpainting

Wang, J. and Yuan, C. and Li, B. and Deng, Y. and Hu, W. and Maybank, Stephen (2023) Self-prior guided pixel adversarial networks for blind image inpainting. IEEE Transactions on Pattern Analysis and Machine Intelligence , ISSN 0162-8828.

Abstract

Blind image inpainting involves two critical aspects, i.e. "where to inpaint" and "how to inpaint". Knowing "where to inpaint" can eliminate the interference arising from corrupted pixel values; a good "how to inpaint" strategy yields high-quality inpainted results robust to various corruptions. In existing methods, these two aspects usually lack explicit and separate consideration. This paper fully explores these two aspects and proposes a self-prior guided inpainting network (SIN). The self-priors are obtained by detecting semantic-discontinuous regions and by predicting global semantic structures of the input image. On the one hand, the self-priors are incorporated into the SIN, which enables the SIN to perceive valid context information from uncorrupted regions and to synthesize semantic-aware textures for corrupted regions. On the other hand, the self-priors are reformulated to provide a pixel-wise adversarial feedback and a high-level semantic structure feedback, which can promote the semantic continuity of inpainted images. Experimental results demonstrate that our method achieves state-of-the-art perfomance in metric scores and in visual quality. It has an advantage over many existing methods that assume "where to inpaint" is known in advance. Extensive experiments on a series of related image restoration tasks validate the effectiveness of our method in obtaining high-quality inpainting.

Metadata

Statistics

DownloadsShow export options

Export as XMLJSONCSV

Activity Overview

6 month trend

248Downloads

6 month trend

84Hits

Additional statistics are available via IRStats2.

Archive Staff Only (login required)

Edit/View Item