BIROn - Birkbeck Institutional Research Online

    Self-prior guided pixel adversarial networks for blind image inpainting

    Wang, J. and Yuan, C. and Li, B. and Deng, Y. and Hu, W. and Maybank, Stephen (2023) Self-prior guided pixel adversarial networks for blind image inpainting. IEEE Transactions on Pattern Analysis and Machine Intelligence , ISSN 0162-8828.

    [img]
    Preview
    Text
    SelfPriorGuidedPixel.pdf
    Available under License Creative Commons Attribution.

    Download (11MB) | Preview

    Abstract

    Blind image inpainting involves two critical aspects, i.e. "where to inpaint" and "how to inpaint". Knowing "where to inpaint" can eliminate the interference arising from corrupted pixel values; a good "how to inpaint" strategy yields high-quality inpainted results robust to various corruptions. In existing methods, these two aspects usually lack explicit and separate consideration. This paper fully explores these two aspects and proposes a self-prior guided inpainting network (SIN). The self-priors are obtained by detecting semantic-discontinuous regions and by predicting global semantic structures of the input image. On the one hand, the self-priors are incorporated into the SIN, which enables the SIN to perceive valid context information from uncorrupted regions and to synthesize semantic-aware textures for corrupted regions. On the other hand, the self-priors are reformulated to provide a pixel-wise adversarial feedback and a high-level semantic structure feedback, which can promote the semantic continuity of inpainted images. Experimental results demonstrate that our method achieves state-of-the-art perfomance in metric scores and in visual quality. It has an advantage over many existing methods that assume "where to inpaint" is known in advance. Extensive experiments on a series of related image restoration tasks validate the effectiveness of our method in obtaining high-quality inpainting.

    Metadata

    Item Type: Article
    Keyword(s) / Subject(s): blind image inpainting, semantic-discontinuity detection, layout map prediction, pixel generative adversarial network
    School: Birkbeck Faculties and Schools > Faculty of Science > School of Computing and Mathematical Sciences
    Depositing User: Steve Maybank
    Date Deposited: 17 Jul 2023 13:11
    Last Modified: 09 Aug 2023 12:54
    URI: https://eprints.bbk.ac.uk/id/eprint/51566

    Statistics

    Activity Overview
    6 month trend
    210Downloads
    6 month trend
    70Hits

    Additional statistics are available via IRStats2.

    Archive Staff Only (login required)

    Edit/View Item Edit/View Item