Skip to main content
SpringerLink
Log in

Multiple interleaving interests modeling of sequential user behaviors in e-commerce platform

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

An anonymous user-behavior session in an e-commerce platform is a time-stamped series of sequential implicit feedback (e.g., clicks and orders of items) in a short period, without user profiles available (e.g., the non-logged-in users). The accurate modeling of such sessions is crucial for distributed representation learning in user profiling and item embedding. It broadly spans the recommendation scenarios with capabilities, such as next-click item prediction. The statistics of sessions provided by one of the largest e-commerce platforms indicate that a user generally has multiple interests in a session simultaneously, and the interests may be interleaved. Recent advances in recurrent neural networks and attention mechanisms have led to promising approaches for modeling such sessions. However, few of the existing models explicitly consider the effects of multiple interleaving interests. Through specific data analysis, we find there are two characteristics present in those sessions: 1) contiguous items usually share the same user interest(e.g., category), which can measure the intensity of interests in the current window scope (i.e., local features of interests); 2) each interest repeatedly occurs in a session, which shows its importance in the current session (i.e., global features of interests). Based on the observations, we present a novel framework that provides M ultiple I nterleaving I nterests M odeling with the following contributions: 1) a local layer is adopted to extract the local features of interests by the convolution operations; 2) a global layer is utilized to capture the global features of interests in the current sequence by considering the frequency of items; 3) an Interest-GRU layer is adopted to track each interest’s sequential evolution by fusing local and global features. Experimental results of the next-click prediction task on two real-world datasets demonstrate that our proposed method significantly outperforms state-of-the-art models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. http://taobao.com

  2. http://2015.recsyschallenge.com

  3. https://github.com/hidasib/GRU4Rec

  4. https://github.com/lijingsdu/sessionRec_NARM

  5. https://github.com/uestcnlp/STAMP

  6. https://github.com/graytowne/caser_pytorch

  7. https://drive.google.com/drive/folders/1No6SyVois0q-15M7JpQSKotjvFMiEEYA?usp=sharing

References

  1. Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al: Tensorflow: A system for large-scale machine learning. In: 12th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2016, pp. 265–283. USENIX Association, Savannah (2016)

  2. Aggarwal, C.C.: Content-based recommender systems. In: Recommender systems, pp. 139–166. Springer (2016)

  3. Burke, R.: Hybrid recommender systems: Survey and experiments. User Model. User-Adapted Interact. 12(4), 331–370 (2002)

    Article  Google Scholar 

  4. Chen, H., Li, Y., Sun, X., dong Xu, G., Yin, H.: Temporal meta-path guided explainable recommendation. arXiv:2101.01433 (2021)

  5. Chen, T., Yin, H., Chen, H., Yan, R., Nguyen, Q.V.H., Li, X.: Air: Attentional Intention-Aware Recommender Systems. In: 2019 IEEE 35Th International Conference on Data Engineering (ICDE), pp. 304–315. IEEE (2019)

  6. Chen, X., Xu, H., Zhang, Y., Tang, J., Cao, Y., Qin, Z., Zha, H.: Sequential recommendation with user memory networks. In: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, WSDM 2018, pp. 108–116. ACM, Marina Del Rey (2018)

  7. Cui, Q., Wu, S., Liu, Q., Zhong, W., Wang, L.: Mv-rnn: a multi-view recurrent neural network for sequential recommendation. IEEE Trans. Knowl. Data Eng. 32(2), 317–331 (2018)

    Article  Google Scholar 

  8. Ekstrand, M.D., Riedl, J.T., Konstan, J.A.: Collaborative filtering recommender systems. Now Publishers Inc (2011)

  9. Gehring, J., Auli, M., Grangier, D., Yarats, D., Dauphin, Y.N.: Convolutional sequence to sequence learning. In: Proceedings of the 34th International Conference on Machine Learning-Volume 70, JMLR. org, pp. 1243–1252 (2017)

  10. Gu, W., Dong, S., Zeng, Z.: Increasing recommended effectiveness with markov chains and purchase intervals. Neural Comput. Appl. 25(5), 1153–1162 (2014)

    Article  Google Scholar 

  11. Guo, H., Tang, R., Ye, Y., Li, Z., He, X.: DeepFM: a factorization-machine based neural network for ctr prediction. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, pp. 1725–1731. ijcai.org, Melbourne (2017)

  12. Hansen, C., Hansen, C., Maystre, L., Mehrotra, R., Brost, B., Tomasi, F., Lalmas, M.: Contextual and sequential user embeddings for large-scale music recommendation. In: RecSys 2020: Fourteenth ACM Conference on Recommender Systems, pp. 53–62. ACM, Virtual Event (2020)

  13. He, X., Liao, L., Zhang, H., Nie, L., Hu, X., Chua, T.S.: Neural collaborative filtering. In: Proceedings of the 26th International Conference on World Wide Web, WWW 2017, pp. 173–182. ACM, Perth (2017)

  14. Hidasi, B., Karatzoglou, A.: Recurrent neural networks with top-k gains for session-based recommendations. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, CIKM 2018, pp. 843–852. ACM, Torino (2018)

  15. Hidasi, B., Karatzoglou, A., Baltrunas, L., Tikk, D.: Session-based recommendations with recurrent neural networks. In: 4th International Conference on Learning Representations, ICLR 2016. Conference Track Proceedings, San Juan (2016)

  16. Hidasi, B., Quadrana, M., Karatzoglou, A., Tikk, D.: Parallel recurrent neural network architectures for feature-rich session-based recommendations. In: Proceedings of the 10th ACM Conference on Recommender Systems, pp. 241–248. ACM, Boston (2016)

  17. Hosseini, S., Yin, H., Zhou, X., Sadiq, S., Kangavari, M.R., Cheung, N.M.: Leveraging multi-aspect time-related influence in location recommendation. World Wide Web 22(3), 1001–1028 (2019)

    Article  Google Scholar 

  18. Hou, Y., Yang, N., Wu, Y., Philip, S.Y.: Explainable recommendation with fusion of aspect information. World Wide Web 22(1), 221–240 (2019)

    Article  Google Scholar 

  19. Huang, J., Zhao, W.X., Dou, H., Wen, J.R., Chang, E.Y.: Improving sequential recommendation with knowledge-enhanced memory networks. In: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018, pp. 505–514. ACM, Ann Arbor (2018)

  20. Jannach, D., Zanker, M., Felfernig, A., Friedrich, G.: Recommender systems: an introduction. Cambridge University Press (2010)

  21. Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009)

    Article  Google Scholar 

  22. Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Stat. 22(1), 79–86 (1951)

    Article  MathSciNet  Google Scholar 

  23. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436 (2015)

    Article  Google Scholar 

  24. Lei, M., Chu, L., Wang, Z., Pei, J., He, C., Zhang, X., Fang, B.: Mining top-k sequential patterns in transaction database graphs. World Wide Web 23(1), 103–130 (2020)

    Article  Google Scholar 

  25. Li, C., Liu, Z., Wu, M., Xu, Y., Zhao, H., Huang, P., Kang, G., Chen, Q., Li, W., Lee, D.L.: Multi-interest network with dynamic routing for recommendation at tmall. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, pp. 2615–2623. ACM, Beijing (2019)

  26. Li, J., Ren, P., Chen, Z., Ren, Z., Lian, T., Ma, J., 2017: Neural attentive session-based recommendation. In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, CIKM 2017, pp. 1419–1428. ACM, Singapore

  27. Li, Z., Xie, H., Xu, G., Li, Q., Leng, M., Zhou, C.: Towards purchase prediction: a transaction-based setting and a graph-based method leveraging price information. Pattern Recogn., 107824 (2021)

  28. Linden, G., Smith, B., York, J.: Amazon.com recommendations: Item-to-item collaborative filtering. IEEE Internet Comput. 7(1):76–80. https://doi.org/10.1109/MIC.2003.1167344 (2003)

  29. Liu, Q., Zeng, Y., Mokhosi, R., Zhang, H.: STAMP: short-term attention/memory priority model for session-based recommendation. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, pp. 1831–1839. ACM, London (2018)

  30. Lv, F., Jin, T., Yu, C., Sun, F., Lin, Q., Yang, K., Ng, W.: SDM: sequential deep matching model for online large-scale recommender system. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, pp. 2635–2643. ACM, Beijing (2019)

  31. Marcus, M.P., Marcinkiewicz, M.A., Santorini, B.: Building a large annotated corpus of english: The penn treebank. Comput. Linguist. 19(2), 313–330 (1993)

    Google Scholar 

  32. Pazzani, M.J., Billsus, D.: Content-based recommendation systems. In: The adaptive Web. Springer, pp 325–341 (2007)

  33. Rendle, S., Freudenthaler, C., Schmidt-Thieme, L.: Factorizing personalized markov chains for next-basket recommendation. In: Proceedings of the 19th International Conference on World Wide Web, WWW 2010, pp. 811–820. ACM, Raleigh (2010)

  34. Sarwar, B., Karypis, G., Konstan, J., Riedl, J.: Item-based collaborative filtering recommendation algorithms. In: Proceedings of the Tenth International World Wide Web Conference, WWW 10, pp. 285–295. ACM, Hong Kong (2001)

  35. Schafer, J.B., Frankowski, D., Herlocker, J., Sen, S.: Collaborative filtering recommender systems. In: The adaptive Web. Springer, pp. 291–324 (2007)

  36. Shani, G., Heckerman, D., Brafman, R.I.: An mdp-based recommender system. J. Mach. Learn. Res. 6(Sep), 1265–1295 (2005)

    MathSciNet  MATH  Google Scholar 

  37. Tan, Y.K., Xu, X., Liu, Y.: Improved recurrent neural networks for session-based recommendations. In: Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, DLRS@RecSys 2016, pp. 17–22. ACM, Boston (2016)

  38. Tang, J., Wang, K.: Personalized top-n sequential recommendation via convolutional sequence embedding. In: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, WSDM 2018, pp. 565–573. ACM, Marina Del Rey (2018)

  39. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, pp. 5998–6008, Long Beach (2017)

  40. Wang, D., Deng, S., Xu, G.: Sequence-based context-aware music recommendation. Inf. Retrieval J. 21(2), 230–252 (2018)

    Article  Google Scholar 

  41. Wang, H., Liu, G., Liu, A., Li, Z., Zheng, K.: DMRAN: A hierarchical fine-grained attention-based network for recommendation. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, pp. 3698–3704. AAAI Press, Macao (2019)

  42. Wang, S., Hu, L., Wang, Y., Sheng, Q.Z., Orgun, M., Cao, L.: Modeling multi-purpose sessions for next-item recommendations via mixture-channel purpose routing networks. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, pp. 3771–3777. AAAI Press, Macao (2019)

  43. Wang, Y., Zhang, C., Wang, S., Philip, S.Y., Bai, L., Cui, L., Xu, G.: Generative temporal link prediction via self-tokenized sequence modeling. World Wide Web 23(4), 2471–2488 (2020)

    Article  Google Scholar 

  44. Weston, J., Chopra, S., Bordes, A.: Memory networks. In: 3rd International Conference on Learning Representations, ICLR 2015. Conference Track Proceedings, San Diego (2015)

  45. Wikipedia: Gini impurity. https://en.wikipedia.org/wiki/Decision_tree_learning#Gini_impurity (2020)

  46. Wu, C.Y., Ahmed, A., Beutel, A., Smola, A.J., Jing, H.: Recurrent recommender networks. In: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, WSDM 2017, pp. 495–503. ACM, Cambridge (2017)

  47. Wu, S., Tang, Y., Zhu, Y., Wang, L., Xie, X., Tan, T.: Session-based recommendation with graph neural networks. In: The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, pp. 346–353. AAAI Press, Honolulu (2019)

  48. Yang, N., Ma, Y., Chen, L., Philip, S.Y.: A meta-feature based unified framework for both cold-start and warm-start explainable recommendations. World Wide Web 23(1), 241–265 (2020)

    Article  Google Scholar 

  49. Ying, R., He, R., Chen, K., Eksombatchai, P., Hamilton, W.L., Leskovec, J.: Graph convolutional neural networks for web-scale recommender systems. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, pp. 974–983. ACM, London (2018)

  50. Yu, F., Liu, Q., Wu, S., Wang, L., Tan, T.: A dynamic recurrent model for next basket recommendation. In: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, SIGIR 2016, pp. 729–732. ACM, Pisa (2016)

  51. Yuan, F., Karatzoglou, A., Arapakis, I., Jose, J.M., He, X.: A simple convolutional generative network for next item recommendation. In: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, WSDM 2019, pp. 582–590. ACM, Melbourne (2019)

  52. Zhang, S., Yao, L., Sun, A., Tay, Y.: Deep learning based recommender system: A survey and new perspectives. ACM Comput. Surv. (CSUR) 52(1):1–38 (2019)

  53. Zhang, T., Zhao, P., Liu, Y., Sheng, V., Xu, J., Wang, D., Liu, G., Zhou, X.: Feature-level deeper self-attention network for sequential recommendation. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, pp. 4320–4326. AAAI Press, Macao (2019)

  54. Zhao, X., Zhang, L., Ding, Z., Xia, L., Tang, J., Yin, D.: Recommendations with negative feedback via pairwise deep reinforcement learning. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, pp. 1040–1048. ACM, London (2018)

  55. Zhou, C., Sun, C., Liu, Z., Lau, F.: A C-LSTM neural network for text classification. arXiv:1511.08630 (2015)

  56. Zhou, G., Zhu, X., Song, C., Fan, Y., Zhu, H., Ma, X., Yan, Y., Jin, J., Li, H., Gai, K.: Deep interest network for click-through rate prediction. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, pp. 1059–1068. ACM, London (2018)

  57. Zhou, G., Mou, N., Fan, Y., Pi, Q., Bian, W., Zhou, C., Zhu, X., Gai, K.: Deep Interest Evolution Network for Click-Through Rate Prediction. In: The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, vol. 33, pp. 5941–5948. AAAI Press, Honolulu (2019)

Download references

Acknowledgements

This work was done while Yuqiang Han was an intern at Alibaba Group. This research was also partially supported by National Research and Development Program of China under grant No.2019YFB1404802, No.2019YFC0118802, and No.2018AAA0102102.

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Zhejiang University, Hangzhou, China

    Yuqiang Han & Jian Wu

  2. Alibaba Group, Hangzhou, China

    Qian Li, Yang Xiao & Hucheng Zhou

  3. CCCE, Nankai University, Tianjin, China

    Zhenglu Yang

Authors
  1. Yuqiang Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qian Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yang Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hucheng Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhenglu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jian Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jian Wu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Han, Y., Li, Q., Xiao, Y. et al. Multiple interleaving interests modeling of sequential user behaviors in e-commerce platform. World Wide Web 24, 1121–1146 (2021). https://doi.org/10.1007/s11280-021-00889-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-021-00889-0

Keywords

Search

Navigation