Abstract
Attention mechanisms have been successfully applied in many fields, including sequential recommendation. Existing recommendation methods often use the deterministic attention network to consider latent user preferences as fixed points in low-dimensional spaces. However, the fixed-point representation is not sufficient to characterize the uncertainty of user preferences that prevails in recommender systems. In this paper, we propose a new Hierarchical Variational Attention Model (HVAM), which employs variational inference to model the uncertainty in sequential recommendation. Specifically, the attention vector is represented as density by imposing a Gaussian distribution rather than a fixed point in the latent feature space. The variance of the attention vector measures the uncertainty associated with the user’s preference representation. Furthermore, the user’s long-term and short-term preferences are captured through a hierarchical variational attention network. Finally, we evaluate the proposed model HVAM using two public real-world datasets. The experimental results demonstrate the superior performance of our model comparing to the state-of-the-art methods for sequential recommendation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bahuleyan, H., Mou, L., Vechtomova, O., Poupart, P.: Variational attention for sequence-to-sequence models. arXiv preprint arXiv:1712.08207 (2017)
Chatzis, S.P., Christodoulou, P., Andreou, A.S.: Recurrent latent variable networks for session-based recommendation. In: DLRS@RecSys, pp. 38–45. ACM (2017)
Chen, J., Zhang, H., He, X., Nie, L., Liu, W., Chua, T.S.: Attentive collaborative filtering: Multimedia recommendation with item-and component-level attention. In: SIGIR, pp. 335–344 (2017)
Chen, S., Moore, J.L., Turnbull, D., Joachims, T.: Playlist prediction via metric embedding. In: KDD, pp. 714–722 (2012)
Feng, S., Li, X., Zeng, Y., Cong, G., Chee, Y.M., Yuan, Q.: Personalized ranking metric embedding for next new POI recommendation. In: IJCAI (2015)
He, R., McAuley, J.: Fusing similarity models with Markov chains for sparse sequential recommendation. In: ICDM, pp. 191–200. IEEE (2016)
Hidasi, B., Karatzoglou, A., Baltrunas, L., Tikk, D.: Session-based recommendations with recurrent neural networks. ICLR (2016)
Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. arXiv preprint arXiv:1312.6114 (2013)
Li, J., Ren, P., Chen, Z., Ren, Z., Lian, T., Ma, J.: Neural attentive session-based recommendation. In: CIKM, pp. 1419–1428 (2017)
Liu, J., et al.: Attention and convolution enhanced memory network for sequential recommendation. In: Li, G., Yang, J., Gama, J., Natwichai, J., Tong, Y. (eds.) DASFAA 2019. LNCS, vol. 11447, pp. 333–349. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-18579-4_20
Luong, M.T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025 (2015)
Rendle, S., Freudenthaler, C., Gantner, Z., Schmidt-Thieme, L.: BPR: Bayesian personalized ranking from implicit feedback. In: UAI, pp. 452–461. AUAI Press (2009)
Rendle, S., Freudenthaler, C., Schmidt-Thieme, L.: Factorizing personalized Markov chains for next-basket recommendation. In: WWW, pp. 811–820. ACM (2010)
Sachdeva, N., Manco, G., Ritacco, E., Pudi, V.: Sequential variational autoencoders for collaborative filtering. In: WSDM, pp. 600–608. ACM (2019)
Sedhain, S., Menon, A.K., Sanner, S., Xie, L.: Autorec: autoencoders meet collaborative filtering. In: WWW, pp. 111–112 (2015)
Tang, J., Wang, K.: Personalized top-n sequential recommendation via convolutional sequence embedding. In: WSDM, pp. 565–573. ACM (2018)
Wang, P., Guo, J., Lan, Y., Xu, J., Wan, S., Cheng, X.: Learning hierarchical representation model for next basket recommendation. In: SIGIR, pp. 403–412 (2015)
Xiao, J., Ye, H., He, X., Zhang, H., Wu, F., Chua, T.S.: Attentional factorization machines: Learning the weight of feature interactions via attention networks. arXiv preprint arXiv:1708.04617 (2017)
Xu, C., et al.: Recurrent convolutional neural network for sequential recommendation. In: WWW, pp. 3398–3404 (2019)
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E.: Hierarchical attention networks for document classification. In: HLT-NAACL, pp. 1480–1489 (2016)
Ying, H., et al.: Sequential recommender system based on hierarchical attention network. In: IJCAI (2018)
Zhao, P., et al.: Where to go next: a spatio-temporal gated network for next poi recommendation. In: AAAI, vol. 33, pp. 5877–5884 (2019)
Zhu, D., Cui, P., Wang, D., Zhu, W.: Deep variational network embedding in wasserstein space. In: KDD, pp. 2827–2836. ACM (2018)
Acknowledgments
This research was partially supported by NSFC (No. 61876117, 61876217, 61872258, 61728205), Suzhou Science and Technology Development Program (SYG201803), Open Program of Key Lab of IIP of CAS (No. IIP2019-1) and PAPD.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, J., Zhao, P., Liu, Y., S. Sheng, V., Li, Z., Zhao, L. (2020). Hierarchical Variational Attention for Sequential Recommendation. In: Nah, Y., Cui, B., Lee, SW., Yu, J.X., Moon, YS., Whang, S.E. (eds) Database Systems for Advanced Applications. DASFAA 2020. Lecture Notes in Computer Science(), vol 12114. Springer, Cham. https://doi.org/10.1007/978-3-030-59419-0_32
Download citation
DOI: https://doi.org/10.1007/978-3-030-59419-0_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59418-3
Online ISBN: 978-3-030-59419-0
eBook Packages: Computer ScienceComputer Science (R0)