Abstract
In this paper, we proposed a method to synthesizing a super-resolution image with the given image and text descriptions. Our work contains two parts. Wasserstein GAN is used to generate low-level resolution image under the guidance of a novel loss function. Then, a convolution net is followed to refine the resolution. This is an end-to-end network architecture. We have validated our model on Caltech-200 bird dataset, Oxford-102 flower dataset, and BSD300 dataset. The experiments show that the generated images not only match the given descriptions well but also maintain detailed features of original images with a higher resolution.
Han and Zhang—Equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Reed S, et al. Generative adversarial text to image synthesis. In: International conference on machine learning, 2016. p. 1060–9 (JMLR.org)
Dong H, Yu S, Wu C, Guo, Y. Semantic image synthesis via adversarial learning. In: IEEE international conference on computer vision (ICCV). New York: IEEE; 2017. p. 5707–15.
Kingma D, Ba J. Adam: A method for stochastic optimization. In: ICLR, 2014.
Goodfellow IJ, et al. Generative adversarial nets. In: Neural information processing systems, 2014. p. 2672–80.
Zhang, H, Xu, T, Li, H. StackGAN: Text to photo-realistic image synthesis with stacked generative adversarial networks. In: International conference on computer vision. New York: IEEE; 2017. p. 5908–16.
Donahue J, Krahenbuhl P, Darrell T. Adversarial feature learning. In: International conference on learning representations, 2017.
Larsen, ABL, Larochelle, H, Winther O. Autoencoding beyond pixels using a learned similarity metric. In: International conference on machine learning, 2016. p. 1558–66 (JMLR.org).
Shi W, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Computer vision and pattern recognition. New York: IEEE; 2016. p. 1874–83.
Arjovsky M, Chintala S, Bottou L. Wasserstein GAN. arXiv, 2017.
Wah C, et al. The Caltech-UCSD Birds-200-2011 Dataset. In: Advances in water resources, 2011.
Nilsback M, Zisserman A. Automated flower classification over a large number of classes. In: Indian conference on computer vision, graphics and image processing, 2008. p. 722–9.
Salimans T, et al. Improved techniques for training GANs. In: Neural information processing systems, 2016. p. 2234–42.
Russakovsky O, et al. ImageNet large scale visual recognition challenge. Int J Comput Vis. 2015;115:211–52.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Han, J., Zhang, Z., Mao, A., Zhou, Y. (2020). Semantics Images Synthesis and Resolution Refinement Using Generative Adversarial Networks. In: Liang, Q., Liu, X., Na, Z., Wang, W., Mu, J., Zhang, B. (eds) Communications, Signal Processing, and Systems. CSPS 2018. Lecture Notes in Electrical Engineering, vol 516. Springer, Singapore. https://doi.org/10.1007/978-981-13-6504-1_74
Download citation
DOI: https://doi.org/10.1007/978-981-13-6504-1_74
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-6503-4
Online ISBN: 978-981-13-6504-1
eBook Packages: EngineeringEngineering (R0)