Abstract
Energy-based generation models have attracted plenty of attention in last few years, but there is a lack of guidance on how to generate a condition-specific samples. In this work, we propose an energy based framework with an autoencoder and a standard discriminative classifier. Within this framework, we demonstrate that classifier can be reinterpreted as an EBM and we can accelerate sampling with fast MCMC in latent space of autoencoder. Both latent EBM and autoencoder can be learned jointly by maximum likelihood. Ultimately, our experimental results show that the trained model exhibits decent performance in both unconditional and conditional generation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Goodfellow, I., Pouget-Abadie, J., Mirza, M., et al.: Generative adversarial networks. Commun. ACM 63(11), 139–144 (2020)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)
Du, Y., Mordatch, I.: Implicit generation and modeling with energy based models. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Gao, R., Song, Y., Poole, B., et al.: Learning energy-based models by diffusion recovery likelihood. arXiv preprint arXiv:2012.08125 (2020)
Grathwohl, W., Wang, K.C., Jacobsen, J.H., et al.: Your classifier is secretly an energy based model and you should treat it like one. arXiv preprint arXiv:1912.03263 (2019)
Pang, B., Wu, Y.N.: Latent space energy-based model of symbol-vector coupling for text generation and classification. In: International Conference on Machine Learning, pp. 8359–8370. PMLR (2021)
LeCun, Y., Chopra, S., Hadsell, R., et al.: A tutorial on energy-based learning. Predicting Struct. Data 1(0) (2006)
Zhao, Y., Xie, J., Li, P.: Learning energy-based generative models via coarse-to-fine expanding and sampling. In: International Conference on Learning Representations (2020)
Geng, C., Wang, J., Gao, Z., et al.: Bounds all around: training energy-based models with bidirectional bounds. In: Advances in Neural Information Processing Systems, vol. 34, pp. 19808–19821 (2021)
Kumar, R., Ozair, S., Goyal, A., et al.: Maximum entropy generators for energy-based models. arXiv preprint arXiv:1901.08508 (2019)
Arbel, M., Zhou, L., Gretton, A.: Generalized energy based models. arXiv preprint arXiv:2003.05033 (2020)
Pang, B., Han, T., Nijkamp, E., et al.: Learning latent space energy-based prior model. In: Advances in Neural Information Processing Systems, vol. 33, pp. 21994–22008 (2020)
Xiao, Z., Kreis, K., Kautz, J., et al.: VAEBM: a symbiosis between variational autoencoders and energy-based models. arXiv preprint arXiv:2010.00654 (2020)
Xie, J., Lu, Y., Gao, R., et al.: Cooperative learning of energy-based model and latent variable model via MCMC teaching. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1 (2018)
Yoon, S., Noh, Y.K., Park, F.: Autoencoding under normalization constraints. In: International Conference on Machine Learning, pp. 12087–12097. PMLR (2021)
Zhao, J., Mathieu, M., LeCun, Y.: Energy-based generative adversarial network. arXiv preprint arXiv:1609.03126 (2016)
Zhai, S., Cheng, Y., Lu, W., et al.: Deep structured energy based models for anomaly detection. In: International Conference on Machine Learning, pp. 1100–1109. PMLR (2016)
Pidhorskyi, S., Almohsen, R., Doretto, G.: Generative probabilistic novelty detection with adversarial autoencoders. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Brehmer, J., Cranmer, K.: Flows for simultaneous manifold learning and density estimation. In: Advances in Neural Information Processing Systems, vol. 33, pp. 442–453 (2020)
Alain, G., Bengio, Y.: What regularized auto-encoders learn from the data-generating distribution. J. Mach. Learn. Res. 15(1), 3563–3593 (2014)
Brock, A., Donahue, J., Simonyan, K.: Large scale GAN training for high fidelity natural image synthesis. arXiv preprint arXiv:1809.11096 (2018)
Kingma, D.P., Dhariwal, P.: Glow: generative flow with invertible 1x1 convolutions. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Dhariwal, P., Nichol, A.: Diffusion models beat GANs on image synthesis. In: Advances in Neural Information Processing Systems, vol. 34, pp. 8780–8794 (2021)
Wang, Z., Bovik, A.C., Sheikh, H.R., et al.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Bergmann, P., Löwe, S., Fauser, M., et al.: Improving unsupervised defect segmentation by applying structural similarity to autoencoders. arXiv preprint arXiv:1807.02011 (2018)
Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images (2009)
Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning (2011)
He, K., Zhang, X.: Shaoqing Ren, and Jian Sun. deep residual learning for image recognition. In: Proceedings of the IEEE CVPR, pp. 770–778 (2016)
He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 630–645. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_38
Heusel, M., Ramsauer, H., Unterthiner, T., et al.: GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Nijkamp, E., Hill, M., Zhu, S.C., et al.: Learning non-convergent non-persistent short-run MCMC toward energy-based model. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Ghosh, P., Sajjadi, M.S., Vergari, A., Black, M., Scholkopf, B.: From variational to deterministic autoencoders. In International Conference on Learning Representations (2020)
Acknowledgments
This work is supported by the National Science Foundation of China (NSFC) under grant 61927809. Here, the authors thank all anonymous reviewers as well as the processing area chair for their valuable comments on an earlier version of this work.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zeng, W., Wang, J. (2023). Latent Energy Based Model with Classifier Guidance. In: Zhai, G., Zhou, J., Yang, H., Yang, X., An, P., Wang, J. (eds) Digital Multimedia Communications. IFTC 2022. Communications in Computer and Information Science, vol 1766. Springer, Singapore. https://doi.org/10.1007/978-981-99-0856-1_39
Download citation
DOI: https://doi.org/10.1007/978-981-99-0856-1_39
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-0855-4
Online ISBN: 978-981-99-0856-1
eBook Packages: Computer ScienceComputer Science (R0)