Niladri Shekhar Dutt
ABSTRACT
Generative Adversarial Networks (GANs) are algorithmic architectures that use two neural networks, pitting one against the opposite (thus the “adversarial”) so as to come up with new, synthetic instances of data that can pass for real data. GANs have been highly successful on datasets like MNIST, SVHN, CelebA, etc but training a GAN on large scale datasets like ImageNet is a challenging problem because they are deemed as not very regular. In this paper, we perform empirical experiments using parameterized synthetic datasets to probe how regularity of a dataset affects learning in GANs. We emperically show that regular datasets are easier to model for GANs because of their stable training process.
- [1] Arjovsky, M., Chintala, S., and Bottou, L. (2017a). “Wasserstein gan.”arXiv preprintarXiv:1701.07875. London, vol. A247, pp. 529–551, April 1955.Google Scholar
- [2] Arjovsky, M., Chintala, S., and Bottou, L. (2017b). “Wasserstein generative adversarialnetworks.”Proceedings of the 34th International Conference on Machine Learning, D. Pre-cup and Y. W. Teh, eds., Vol. 70 ofProceedings of Machine Learning Research, InternationalConvention Centre, Sydney, Australia. PMLR, 214–223.Google Scholar
- [3] Chalmeta, R., Hurtado, F., Sacristán, V., and Saumell, M. (2013). “Measuring regularity ofconvex polygons.”Computer-Aided Design, 45(2), 93–104.Google ScholarDigital Library
- [4] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., andCourville, A. (2014). “ bengio, y.(2014). generative adversarial nets.”Advances in neuralinformation processing systems, 27.Google Scholar
- [5] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S.,Courville, A., and Bengio, Y. (2014). “Generative adversarial nets.”Advances in neuralinformation processing systems. 2672–2680.Google Scholar
- [6] Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. (2017). “Ganstrained by a two time-scale update rule converge to a local nash equilibrium.”Advances inneural information processing systems. 6626–6637.Google Scholar
- [7] Huszár, F. (2015). “How (not) to train your generative model: Scheduled sampling, likeli-hood, adversary?.”arXiv preprint arXiv:1511.05101.Google Scholar
- [8] Lucic, M., Kurach, K., Michalski, M., Gelly, S., and Bousquet, O. (2018). “Are ganscreated equal? a large-scale study.”Advances in neural information processing systems.700–709.Google Scholar
- [9] Mao, X., Li, Q., Xie, H., Lau, R. Y., Wang, Z., and Paul Smolley, S. (2017). “Leastsquares generative adversarial networks.”Proceedings of the IEEE International Conferenceon Computer Vision. 2794–2802.Google Scholar
- [10] Odena, A., Olah, C., and Shlens, J. (2017). “Conditional image synthesis with auxiliaryclassifier gans.”Proceedings of the 34th International Conference on Machine Learning-Volume 70, JMLR. org. 2642–2651.Google Scholar
- [11]Radford, A., Metz, L., and Chintala, S. (2015). “Unsupervised representation learningwith deep convolutional generative adversarial networks.”arXiv preprint arXiv:1511.06434.Google Scholar
- [12]Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., and Chen, X. (2016).“Improved techniques for training gans.”Advances in neural information processing sys-tems. 2234–2242.40Google Scholar
- [13]Theis, L., Oord, A. v. d., and Bethge, M. (2015). “A note on the evaluation of generativemodels.”arXiv preprint arXiv:1511.01844.Google Scholar
- [14] Wu, Y., Burda, Y., Salakhutdinov, R., and Grosse, R. (2016). “On the quantitative analysisof decoder-based generative models.”arXiv preprint arXiv:1611.04273.Google Scholar
- [15] Zhang, H., Xu, T., Li, H., Zhang, S., Wang, X., Huang, X., and Metaxas, D. N. (2017).“Stackgan: Text to photo-realistic image synthesis with stacked generative adversarial net-works.”Proceedings of the IEEE international conference on computer vision. 5907–5915.Google Scholar
- [16] Such, Felipe Petroski and Rawal, Aditya and Lehman, Joel and Stanley, Kenneth O and Clune, Jeff (2019).”Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Data.”arXiv preprint arXiv:1912.07768Google Scholar
- [17] Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. ”Imagenet classification with deep convolutional neural networks.” Advances in neural information processing systems 25 (2012): 1097-1105.Google Scholar
- [18] LeCun, Y. & Cortes, C. (2010). MNIST handwritten digit database.,.Google Scholar
- [19] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, Andrew Y. Ng Reading Digits in Natural Images with Unsupervised Feature Learning NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011.Google Scholar
- [20] Liu, Ziwei, et al. ”Deep learning face attributes in the wild.” Proceedings of the IEEE international conference on computer vision. 2015.Google ScholarDigital Library
- [21] Heusel, Martin, et al. ”Gans trained by a two time-scale update rule converge to a local nash equilibrium.” Advances in neural information processing systems 30 (2017).Google Scholar
- [22] Yu, Fisher, et al. ”Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop.” arXiv preprint arXiv:1506.03365 (2015).Google Scholar
Index Terms
- Effect of regularity on learning in GANs
Recommendations
Generative Adversarial Networks (GANs): Challenges, Solutions, and Future Directions
Generative Adversarial Networks (GANs) is a novel class of deep generative models that has recently gained significant attention. GANs learn complex and high-dimensional distributions implicitly over images, audio, and data. However, there exist major ...
Data augmentation using fast converging CIELAB-GAN for efficient deep learning dataset generation
The commercial deep learning applications require large training datasets with many samples from different classes. The generative adversarial networks (GAN) are able to create new data samples for training these machine learning models. However, the low ...
Exploring the Effect of High-frequency Components in GANs Training
Generative Adversarial Networks (GANs) have the ability to generate images that are visually indistinguishable from real images. However, recent studies have revealed that generated and real images share significant differences in the frequency domain. In ...
Comments