Image Generation from Text Using StackGAN with Consistency Regularization

Abstract

Image generation from natural language is one of the most exciting areas of image and language multimodal research in recent years. The Stacked Generative Adversarial Networks (StackGAN) model generates images from text; however, although this model has been successful in generating high-resolution images, it has some problems. Generated images can be unintelligible, and there are cases of mode collapse. Therefore, this study attempts to solve these two StackGAN problems and aims to generate more accurate images. We propose incorporating balanced consistency regularization (bCR) into StackGAN. The bCR method uses data augmentation to learn the meaning of data by making the identification results consistent. Additionally, bCR can stabilize learning in adversarial networks. Our experiments show that the Inception Score of StackGAN with bCR was 7% better than StackGAN alone. In addition, mode collapse was eliminated.