Multi-level Discriminator and Wavelet Loss for Image Inpainting with Large Missing Area

Abstract

Recent image inpainting works have shown promising results thanks to great advances of generative adversarial networks (GANs). However, these methods would still generate distorted structures or blurry textures for the situation of large missing area, which is mainly due to the inherent difficulty to train GANs. In this paper, we propose a novel multi-level discriminator (MLD) and wavelet loss (WT) to improve the learning of image inpainting generators. Our method does not change the structure of generator and only works in the training phase, which thus can be easily embedded into sophisticated inpainting networks and would not increase the inference time. Specifically, MLD divides the mask into multiple subregions and then imposes an independent discriminator to each subregion. It essentially increases the distribution overlap between the real images and generated images. Consequently, MLD improves the optimization of GANs by providing more effective gradients to generators. In addition, WT builds a reconstruction loss in the frequency domain, which can facilitate the training of image inpainting networks as a regularization term. Consequently, WT can enforce the generated contents to be more consistent and sharper than the traditional pixel-wise reconstruction loss. We integrate WLD and WT into off-the-shelf image inpainting networks, and conduct extensive experiments on CelebA-HQ, Paris StreetView, and Places2. The results well demonstrate the effectiveness of the proposed method, which achieves state-of-the-art performance and generates higher-quality images than the baselines.

J. Li—Student.