Skip to main content
SpringerLink
Log in

Two-stage sequential recommendation for side information fusion and long-term and short-term preferences modeling

  • Published:
Journal of Intelligent Information Systems Aims and scope Submit manuscript

Abstract

Sequential recommender systems aim to model users’ changing interests based on their historical behavior and predict what they will be interested in at the next moment. In recent years, approaches to modeling users’ long-term/short-term preferences have achieved promising results. Previous works typically model historical interactions through an end-to-end neural network incorporating rich side information, which relies on a final loss function to optimize all parameters. However, they tend to concatenate side information and item ID into a vector representation, leading to irreversible fusion. We propose a two-stage sequence recommendation framework to address this problem. The first stage aims to enhance the representation ability of sequence through a non-invasive bidirectional self-attentive item embedding. In the second stage, we use a time-interval aware Gated Recurrent Units with attention to capture the user’s latest intents, while predicting long-term preferences based on the first stage. To integrate the long-term/short-term preferences, we generate the final preference representation using an attention-based adaptive fusion module. We conduct extensive experiments on four benchmark datasets and the results demonstrate the effectiveness of our proposed model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. https://nijianmo.github.io/amazon/index.html

  2. https://grouplens.org/datasets/movielens/

References

  • An, M., Wu, F., Wu, C., & et al. (2019). Neural news recommendation with long-and short-term user representations. In Proceedings of the 57th annual meeting of the association for computational linguistics (pp. 336–345). https://doi.org/10.18653/v1/P19-1033

  • Cho, K., Van Merriënboer, B., Gulcehre, C., & et al. (2014). Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv:1406.1078.

  • Devlin, J., Chang, M. W., Lee, K., & et al. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805.

  • Devooght, R., & Bersini, H. (2017). Long and short-term recommendations with recurrent neural networks. In Proceedings of the 25th conference on user modeling, adaptation and personalization (pp. 13–21). https://doi.org/10.1145/3079628.3079670

  • Donkers, T., Loepp, B., & Ziegler, J. (2017). Sequential user-based recurrent neural network recommendations. In Proceedings of the eleventh ACM conference on recommender systems (pp. 152–160). https://doi.org/10.1145/3109859.3109877

  • Gehring, J., Auli, M., Grangier, D., & et al. (2017). Convolutional sequence to sequence learning. In International conference on machine learning, PMLR (pp. 1243–1252). https://doi.org/10.48550/arXiv.1705.03122

  • He, Y., Zhang, Y., Liu, W., & et al. (2020). Consistency-aware recommendation for user-generated item list continuation. In Proceedings of the 13th international conference on web search and data mining (pp. 250–258). https://doi.org/10.1145/3336191.3371776

  • Hidasi, B., & Karatzoglou, A. (2018). Recurrent neural networks with top-k gains for session-based recommendations. In Proceedings of the 27th ACM international conference on information and knowledge management (pp. 843–852). https://doi.org/10.1145/3269206.3271761

  • Hidasi, B., Karatzoglou, A., Baltrunas, L., & et al. (2015). Session-based recommendations with recurrent neural networks. arXiv:1511.06939.

  • Hidasi, B., Quadrana, M., Karatzoglou, A., & et al. (2016). Parallel recurrent neural network architectures for feature-rich session-based recommendations. In Proceedings of the 10th ACM conference on recommender systems (pp. 241–248). https://doi.org/10.1145/2959100.2959167

  • Hjelm, R. D., Fedorov, A., Lavoie-Marchildon, S., & et al. (2018). Learning deep representations by mutual information estimation and maximization. arXiv:1808.06670.

  • Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  • Huang, J., Ren, Z., Zhao, W.X., & et al. (2019). Taxonomy-aware multi-hop reasoning networks for sequential recommendation. In Proceedings of the twelfth ACM international conference on web search and data mining (pp. 573–581). https://doi.org/10.1145/3289600.3290972

  • Kang, W. C., & McAuley, J. (2018). Self-attentive sequential recommendation. In 2018 IEEE international conference on data mining (ICDM) (pp. 197–206). https://doi.org/10.1109/ICDM.2018.00035

  • Kong, L., d’Autume, C.D.M., Ling, W., & et al. (2019). A mutual information maximization perspective of language representation learning. arXiv:1910.08350.

  • Li, J., Ren, P., Chen, Z., & et al. (2017). Neural attentive session-based recommendation. In Proceedings of the 2017 ACM on conference on information and knowledge management (pp. 1419–1428). https://doi.org/10.1145/3132847.3132926

  • Li, W., Saigo, H., Tong, B., & et al. (2021). Topic modeling for sequential documents based on hybrid inter-document topic dependency. Journal of Intelligent Information Systems, 56 (3), 435–458. https://doi.org/10.1007/s10844-020-00635-4.

    Article  Google Scholar 

  • Li, Z., Lei, C., Zou, P., & et al. (2020). Attention with long-term interval-based gated recurrent units for modeling sequential user behaviors. In International conference on database systems for advanced applications (pp. 663–670). https://doi.org/10.1007/978-3-030-59410-7_44

  • Li, Z., Zhao, H., Liu, Q., & et al. (2018). Learning from history and present: Next-item recommendation via discriminatively exploiting user behaviors. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining (pp. 1734–1743). https://doi.org/10.1145/3219819.3220014

  • Liu, C., Li, X., & Cai, G. (2021). Non-invasive self-attention for side information fusion in sequential recommendation. arXiv:2103.03578.

  • Neil, D., Pfeiffer, M., & Liu, S.C. (2016). Phased lstm: Accelerating recurrent network training for long or event-based sequences. Advances in Neural Information Processing Systems, 29. https://doi.org/10.48550/arXiv.1610.09513.

  • Quadrana, M., Karatzoglou, A., Hidasi, B., & et al. (2017). Personalizing session-based recommendations with hierarchical recurrent neural networks. In Proceedings of the eleventh ACM conference on recommender systems (pp. 130–137). https://doi.org/10.1145/3109859.3109896

  • Rendle, S., Freudenthaler, C., Schmidt-Thieme, L., & et al. (2010). Factorizing personalized markov chains for next-basket recommendation. In Proceedings of the 19th international conference on World Wide Web (pp. 811–820). https://doi.org/10.1145/1772690.1772773

  • Stratigi, M., Pitoura, E., Nummenmaa, J., & et al. (2022). Sequential group recommendations based on satisfaction and disagreement scores. Journal of Intelligent Information Systems, 58 (2), 227–254. https://doi.org/10.1007/s10844-021-00652-x.

    Article  Google Scholar 

  • Sun, F., Liu, J., Wu, J., & et al. (2019). Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM international conference on information and knowledge management (pp. 1441–1450). https://doi.org/10.1145/3357384.3357895

  • Vassøy, B., Ruocco, M., de Souza da Silva, E., & et al. (2019). Time is of the essence: a joint hierarchical rnn and point process model for time and item predictions. In Proceedings of the twelfth ACM international conference on Web search and data mining (pp. 591–599). https://doi.org/10.1145/3289600.3290987

  • Wu, C.Y., Ahmed, A., Beutel, A., & et al. (2017). Recurrent recommender networks. In Proceedings of the tenth ACM international conference on web search and data mining (pp. 495–503). https://doi.org/10.1145/3018661.3018689

  • Xie, X., Sun, F., Liu, Z., & et al. (2020). Contrastive learning for sequential recommendation. arXiv:2010.14395.

  • Xu, C., Zhao, P., Liu, Y., & et al. (2019). Graph contextualized self-attention network for session-based recommendation. In IJCAI (Vol. 19 pp. 3940–3946). https://doi.org/10.24963/ijcai.2019/547 

  • Yu, F., Liu, Q., Wu, S., & et al. (2016). A dynamic recurrent model for next basket recommendation. In Proceedings of the 39th International ACM SIGIR conference on research and development in information retrieval. (pp. 729–732) https://doi.org/10.1145/2911451.2914683

  • Zhang, T., Zhao, P., Liu, Y., & et al. (2019). Feature-level deeper self-attention network for sequential recommendation. In IJCAI (pp. 4320–4326). https://doi.org/10.24963/ijcai.2019/600

  • Zhou, C., Bai, J., Song, J., & et al. (2018). Atrank: An attention-based user behavior modeling framework for recommendation. In Thirty-Second AAAI conference on artificial intelligence. https://doi.org/10.48550/arXiv.1711.06632

  • Zhou, K., Wang, H., Zhao, W.X., & et al. (2020). S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization. In Proceedings of the 29th ACM international conference on information & knowledge management (pp. 1893–1902). https://doi.org/10.1145/3340531.3411954

  • Zhu, Y., Li, H., Liao, Y., & et al. (2017). What to do next: Modeling user behaviors by time-lstm. In IJCAI (Vol. 17 pp. 3602–3608). https://doi.org/10.24963/ijcai.2017/504

Download references

Acknowledgements

This work is supported by Natural Science Foundation of China (No.61672337, 61972357) and Medical Science and Technology Project of Zhejiang Province (No.2022KY104).

Author information

Authors and Affiliations

  1. Zhejiang University of Science and Technology, Hangzhou, 310023, China

    Jingsheng Lei, Yuexin Li, Shengying Yang & Wenbin Shi

  2. Zhejiang Cancer Hospital, Hangzhou, 310022, China

    Yi Wu

Authors
  1. Jingsheng Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuexin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shengying Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wenbin Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yi Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shengying Yang.

Ethics declarations

Conflict of Interests

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Additional information

Data availability

Amazon Beauty, Books and MovieLens are openly available dataset and can be downloaded from their official website.

1. https://nijianmo.github.io/amazon/index.html

2. https://grouplens.org/datasets/movielens/

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lei, J., Li, Y., Yang, S. et al. Two-stage sequential recommendation for side information fusion and long-term and short-term preferences modeling. J Intell Inf Syst 59, 657–677 (2022). https://doi.org/10.1007/s10844-022-00723-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10844-022-00723-7

Keywords

Search

Navigation