Abstract
Sum-Product Networks (SPNs) are a new class of deep probabilistic model allowing tractable and exact inference. Recently SPNs have been successfully employed as autoencoder framework in Representation Learning. However, SPNs autoencoding mechanism ignores the model structural duality and train the models separately and independently. In this paper, we propose the Dual-SPNs autoencoding mechanism which design model structure as a dual close loop. This approach training the models simultaneously, and explicitly exploiting their structural duality correlation to guide the training process. As shown in extensive multilabel classification experiments, Dual-SPNs autoencoding mechanism prove highly competitive against the ones employing SPNs autoencoding mechanism and other stacked autoencoder architectures.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Choi, A., Darwiche, A.: On relaxing determinism in arithmetic circuits. In: Thirty-Fourth International Conference on Machine Learning, pp. 825–833. ACM, Sydney (2017)
Poon, H., Domingos, P.: Sum-product networks: a new deep architecture. In: The 27th Conference on Uncertainty in Artificial Intelligence, pp. 337–346. AUAI, Barcelona (2011)
Cheng, W., Kok, S., Pham, H.: Language modeling with sum-product networks. In: Fifteenth Annual Conference of the International Speech Communication Association, pp. 2098–2102. ISSN, Singapore (2014)
Amer, M.R., Todorovic, S.: Sum product networks for activity recognition. IEEE Trans. Pattern Anal. Mach. Intell. 38(4), 800–813 (2016)
Vergari, A., Peharz, R.: Sum-product autoencoding: encoding and decoding representations using sum-product networks. In: The Thirty-Second Conference on Artificial Intelligence. AAAI, New Orleans (2018)
He, D., Xia, Y., Qin, T., et al.: Dual learning for machine translation. In: The Thirtieth Annual Conference on Neural Information Processing Systems, pp. 820–828. MIT Press, Barcelona (2016)
Yi, Z., Zhang, H., Tan, P., et al.: DualGAN: unsupervised dual learning for image-to-image translation. In: International Conference on Computer Vision, pp. 2868–2876. IEEE, Venice (2017)
Gens, R., Pedro, D.: Learning the structure of sum-product networks. In: 30th International Conference on Machine Learning, pp. 873–880. ACM, Atlanta (2013)
Gens, R., Domingos, P.: Discriminative learning of sum-product networks, In: 26th Advances in Neural Information Processing Systems, pp. 3239–3247. MIT Press, Lake Tahoe (2012)
Vergari, A., Di Mauro, N., Esposito, F.: Simplifying, regularizing and strengthening sum-product network structure learning. In: Appice, A., Rodrigues, P.P., Santos Costa, V., Gama, J., Jorge, A., Soares, C. (eds.) ECML PKDD 2015. LNCS, vol. 9285, pp. 343–358. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23525-7_21
Kingma, D., Adam, J.B.: A method for stochastic optimization. Comput. Sci. (2014)
Finley, T., Joachims, T.: Training structural SVMs when exact inference is intractable. In: International Conference on Machine Learning, pp. 304–311. ACM, Helsinki (2008)
Germain, M., Gregor, K., Murray, I., et al.: MADE: masked autoencoder for distribution estimation. In: The 32nd International Conference on Machine Learning, pp. 881–889. ACM, Lille (2015)
Wicker, J., Tyukin, A., Kramer, S.: A nonlinear label compression and transformation method for multi-label classification using autoencoders. In: Bailey, J., Khan, L., Washio, T., Dobbie, G., Huang, J.Z., Wang, R. (eds.) PAKDD 2016. LNCS, vol. 9651, pp. 328–340. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-31753-3_27
Ueda, N., Saito, K.: Single-shot detection of multiple categories of text using parametric mixture models. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 626–631. ACM, Alberta (2001)
Cano, A., Luna, J.M., Gibaja, E.L.: LAIM discretization for multi-label data. Inf. Sci. 330, 370–384 (2016)
Acknowledgments
This work is supported by the National Natural Science Foundation of China (61472161), Science & Technology Development Project of Jilin Province (20180101334JC).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, S., Zhang, H., Liu, J., Yu, Qy. (2018). Dual Sum-Product Networks Autoencoding. In: Liu, W., Giunchiglia, F., Yang, B. (eds) Knowledge Science, Engineering and Management. KSEM 2018. Lecture Notes in Computer Science(), vol 11061. Springer, Cham. https://doi.org/10.1007/978-3-319-99365-2_33
Download citation
DOI: https://doi.org/10.1007/978-3-319-99365-2_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-99364-5
Online ISBN: 978-3-319-99365-2
eBook Packages: Computer ScienceComputer Science (R0)