Abstract
For modeling the data distribution or the latent representation distribution in the image domain, deep learning methods such as the variational autoencoder (VAE) and the generative adversarial network (GAN) have been proposed. However, despite its capability of modeling these two distributions, VAE tends to learn less meaningful latent representations; GAN can only model the data distribution using the challenging and unstable adversarial training. To address these issues, we propose an unsupervised learning framework to perform coupled learning of these two distributions based on kernel maximum mean discrepancy (MMD). Specifically, the proposed framework consists of (1) an inference network and a generation network for mapping between the data space and the latent space, and (2) a latent tester and a data tester for performing two-sample tests in these two spaces, respectively. On one hand, we perform a two-sample test between stochastic representations from the prior distribution and inferred representations from the inference network. On the other hand, we perform a two-sample test between the real data and generated data. In addition, we impose structural regularization that the two networks are inverses of each other, so that the learning of these two distributions can be coupled. Experimental results on benchmark image datasets demonstrate that the proposed framework is competitive on image generation and latent representation inference of images compared with representative approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein GAN. CoRR arXiv:1701.07875 (2017)
Bojanowski, P., Joulin, A., Lopez-Paz, D., Szlam, A.: Optimizing the latent space of generative networks. CoRR arXiv:1707.05776
Chen, X., et al.: Variational lossy autoencoder. CoRR arXiv:1611.02731 (2016)
Denton, E.L., Chintala, S., Szlam, A., Fergus, R.: Deep generative image models using a laplacian pyramid of adversarial networks. In: NIPS, pp. 1486–1494 (2015)
Dosovitskiy, A., Brox, T.: Generating images with perceptual similarity metrics based on deep networks. In: NIPS, pp. 658–666 (2016)
Dumoulin, V., et al.: Adversarially learned inference. CoRR arXiv:1606.00704 (2016)
Dziugaite, G.K., Roy, D.M., Ghahramani, Z.: Training generative neural networks via maximum mean discrepancy optimization. In: UAI, pp. 258–267 (2015)
Goodfellow, I.J., et al.: Generative adversarial nets. In: NIPS, pp. 2672–2680 (2014)
Gretton, A., Borgwardt, K.M., Rasch, M.J., Schölkopf, B., Smola, A.J.: A kernel method for the two-sample-problem. In: NIPS, pp. 513–520 (2006)
Grewal, K., Hjelm, R.D., Bengio, Y.: Variance regularizing adversarial learning. CoRR arXiv:1707.00309
Isola, P., Zhu, J., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: CVPR, pp. 5967–5976 (2017)
Kim, T., Cha, M., Kim, H., Lee, J.K., Kim, J.: Learning to discover cross-domain relations with generative adversarial networks. In: ICML, pp. 1857–1865 (2017)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR arXiv:1412.6980
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. CoRR arXiv:1312.6114
Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images (2009)
Larsen, A.B.L., Sønderby, S.K., Larochelle, H., Winther, O.: Autoencoding beyond pixels using a learned similarity metric. In: ICML, pp. 1558–1566 (2016)
Li, C., Chang, W., Cheng, Y., Yang, Y., Póczos, B.: MMD GAN: towards deeper understanding of moment matching network. CoRR arXiv:1705.08584
Li, Y., Swersky, K., Zemel, R.S.: Generative moment matching networks. In: ICML, pp. 1718–1727 (2015)
Liu, M., Tuzel, O.: Coupled generative adversarial networks. In: NIPS, pp. 469–477 (2016)
Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: ICCV, pp. 3730–3738 (2015)
Mao, X., Li, Q., Xie, H., Lau, R.Y.K., Wang, Z.: Multi-class generative adversarial networks with the L2 loss function. CoRR arXiv:1611.04076 (2016)
Mathieu, M., Couprie, C., LeCun, Y.: Deep multi-scale video prediction beyond mean square error. CoRR arXiv:1511.05440 (2015)
Mirza, M., Osindero, S.: Conditional generative adversarial nets. CoRR arXiv:1411.1784
Nowozin, S., Cseke, B., Tomioka, R.: f-GAN: training generative neural samplers using variational divergence minimization. In: NIPS, pp. 271–279 (2016)
Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. CoRR arXiv:1511.06434
Sutherland, D.J., et al.: Generative models and model criticism via optimized maximum mean discrepancy. CoRR arXiv:1611.04488
Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.: Extracting and composing robust features with denoising autoencoders. In: ICML, pp. 1096–1103 (2008)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Wu, J., Zhang, C., Xue, T., Freeman, B., Tenenbaum, J.: Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling. In: NIPS, pp. 82–90 (2016)
Zhang, H., et al.: StackGAN: text to photo-realistic image synthesis with stacked generative adversarial networks. CoRR abs/1612.03242 (2016)
Zhao, J.J., Mathieu, M., LeCun, Y.: Energy-based generative adversarial network. CoRR arXiv:1609.03126
Zhao, S., Song, J., Ermon, S.: InfoVAE: information maximizing variational autoencoders. CoRR arXiv:1706.02262
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Qian, S., Cao, Wm., Li, R., Wu, S., Wong, Hs. (2018). Coupled Learning for Image Generation and Latent Representation Inference Using MMD. In: Hong, R., Cheng, WH., Yamasaki, T., Wang, M., Ngo, CW. (eds) Advances in Multimedia Information Processing – PCM 2018. PCM 2018. Lecture Notes in Computer Science(), vol 11165. Springer, Cham. https://doi.org/10.1007/978-3-030-00767-6_40
Download citation
DOI: https://doi.org/10.1007/978-3-030-00767-6_40
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00766-9
Online ISBN: 978-3-030-00767-6
eBook Packages: Computer ScienceComputer Science (R0)