Kun Zhou
Skip Abstract Section
Abstract
Sequential recommendation models aim to predict the interested items to a user based on his historical behaviors. To train sequential recommenders, implicit feedback data is widely adopted since it is easier to obtain than explicit feedback data. In the setting of implicit feedback, a user’s historical behaviors can be characterized as a chronologically ordered sequence of interacted items. From a perspective of machine learning, the historical interaction sequence and the recommended items can be considered as context and label, respectively, which are usually in one-hot representations in the recommendation models.
However, due to the discrete nature, one-hot representations are hard to sufficiently reflect the underlying user preference, and might also contain noise from implicit feedback that will mislead the model training. To solve these issues, we propose a general optimization framework, Multi-View Smoothness (MVS), to enhance the smoothness of sequential recommendation models in both data representations and model learning. Specifically, with the help of a complementary model, we smooth and enrich the one-hot representations of contexts and labels to better depict the underlying user preference (i.e., context smoothness and label smoothness), and devise a model regularization strategy to enforce the neighborhood smoothness of the model itself (i.e., model smoothness). Based on these strategies, we design three regularizers to constrain and improve the training of sequential recommendation models. Extensive experiments on five datasets show that our approach is able to improve the performance of various base models consistently and outperform other regularization training methods.
- [1] . 2022. Multi-interest diversification for end-to-end sequential recommendation. ACM Transactions on Information Systems 40, 1 (2022), Article 20, 30 pages. Google Scholar
Digital Library
- [2] . 2018. Sequential recommendation with user memory networks. In Proceedings of the 11th ACM International Conference on Web Search and Data Mining (WSDM’18). ACM New York, NY, 108–116.Google ScholarDigital Library
- [3] . 2022. ELECRec: Training sequential recommenders as discriminators. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’22). ACM, New York, NY.Google ScholarDigital Library
- [4] . 2022. Intent contrastive learning for sequential recommendation. In Proceedings of the ACM Web Conference 2022 (WWW’22). ACM, New York, NY, 2172–2182.Google ScholarDigital Library
- [5] . 2021. Learning recommender systems with implicit feedback via soft target enhancement. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’21). ACM, New York, NY, 575–584.Google ScholarDigital Library
- [6] . 2022. Sequential recommendation via stochastic self-attention. In Proceedings of the ACM Web Conference 2022 (WWW’22). ACM, New York, NY, 2036–2047.Google ScholarDigital Library
- [7] . 2021. Continuous-time sequential recommendation with temporal graph collaborative transformer. In Proceedings of the 30th ACM International Conference on Information and Knowledge Management (CIKM’21). ACM, New York, NY, 433–442.Google ScholarDigital Library
- [8] . 2020. Deep learning for sequential recommendation: Algorithms, influential factors, and evaluations. ACM Transactions on Information Systems 39, 1 (2020), 1–42.Google ScholarDigital Library
- [9] . 2018. Improved robustness to adversarial examples using Lipschitz regularization of the loss. arXiv:1810.000953 (2018).Google Scholar
- [10] . 2020. Towards a better understanding of label smoothing in neural machine translation. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing (AACL/IJCNLP’20). 212–223.Google Scholar
- [11] . 2017. SSE: Semantically smooth embedding for knowledge graphs. IEEE Transactions on Knowledge and Data Engineering 29, 4 (2017), 884–897.Google ScholarDigital Library
- [12] . 2017. Neural collaborative filtering. In Proceedings of the 26th International Conference on World Wide Web (WWW’17). 173–182.Google ScholarDigital Library
- [13] . 2016. Session-based recommendations with recurrent neural networks. In Proceedings of the 4th International Conference on Learning Representations (ICLR’16).Google Scholar
- [14] . 2016. Parallel recurrent neural network architectures for feature-rich session-based recommendations. In Proceedings of the 10th ACM Conference on Recommender Systems. ACM, New York, NY, 241–248.Google ScholarDigital Library
- [15] . 2015. Distilling the knowledge in a neural network. CoRR abs/1503.02531 (2015).Google Scholar
- [16] . 2022. Learning vector-quantized item representation for transferable sequential recommenders. arXiv preprint arXiv:2210.12316 (2022).Google Scholar
- [17] . 2022. Towards universal sequence representation learning for recommender systems. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD’22). 585–593.Google ScholarDigital Library
- [18] . 2022. Learning to learn a cold-start sequential recommender. ACM Transactions on Information Systems 40, 2 (2022), Article 30, 25 pages.Google ScholarDigital Library
- [19] . 2020. SMART: Robust and efficient fine-tuning for pre-trained natural language models through principled regularized optimization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL’20). 2177–2190.Google ScholarCross Ref
- [20] . 2007. Instance weighting for domain adaptation in NLP. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (ACL’07).Google Scholar
- [21] . 2022. Learn over past, evolve for future: Search-based time-aware recommendation with sequential behavior data. In Proceedings of the ACM Web Conference 2022 (WWW’22). ACM, New York, NY, 2451–2461.Google ScholarDigital Library
- [22] . 2018. Self-attentive sequential recommendation. In Proceedings of the IEEE International Conference on Data Mining (ICDM’18). IEEE, Los Alamitos, CA, 197–206.Google ScholarCross Ref
- [23] . 2015. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations (ICLR’15): Conference Track Proceedings.Google Scholar
- [24] . 2020. On sampled metrics for item recommendation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’20). ACM, New York, NY, 1748–1757.Google ScholarDigital Library
- [25] . 2017. Adversarial examples in the physical world. In Proceedings of the 5th International Conference on Learning Representations (ICLR’17): Workshop Track.Google Scholar
- [26] . 2015. Deep learning. Nature 521, 7553 (2015), 436–444.Google ScholarCross Ref
- [27] . 2017. Neural attentive session-based recommendation. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management (CIKM’17). ACM, New York, NY, 1419–1428.Google ScholarDigital Library
- [28] . 2021. TAVAT: Token-aware virtual adversarial training for language understanding. In Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI’21).Google Scholar
- [29] . 2018. STAMP: Short-term attention/memory priority model for session-based recommendation. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’18). ACM, New York, NY, 1831–1839.Google ScholarDigital Library
- [30] . 2020. Memory augmented graph neural networks for sequential recommendation. In Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI’20). 5045–5052.Google ScholarCross Ref
- [31] . 2019. Virtual adversarial training: A regularization method for supervised and semi-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 41, 8 (2019), 1979–1993.Google ScholarCross Ref
- [32] . 2019. When does label smoothing help? In Advances in Neural Information Processing Systems 32. 4696–4705.Google Scholar
- [33] . 2018. Lipschitz regularized deep neural networks converge and generalize. CoRR abs/1808.09540 (2018).Google Scholar
- [34] . 2022. Multimodal meta-learning for cold-start sequential recommendation. In Proceedings of the 31st ACM International Conference on Information and Knowledge Management (CIKM’22). 3421–3430.Google ScholarDigital Library
- [35] . 2017. Regularizing neural networks by penalizing confident output distributions. In Proceedings of the 5th International Conference on Learning Representations (ICLR’17): Workshop Track Proceedings. https://openreview.net/forum?id=HyhbYrGYe.Google Scholar
- [36] . 2019. Adversarial robustness through local linearization. In Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS’19). 13824–13833.Google Scholar
- [37] . 2022. Exploiting positional information for session-based recommendation. ACM Transactions on Information Systems 40, 2 (2022), Article 35, 24 pages. Google ScholarDigital Library
- [38] . 2021. Memory augmented multi-instance contrastive predictive coding for sequential recommendation. In Proceedings of the IEEE International Conference on Data Mining (ICDM’21). IEEE, Los Alamitos, CA, 519–528.Google ScholarCross Ref
- [39] . 2022. Contrastive learning for representation degeneration problem in sequential recommendation. In Proceedings of the 15th ACM International Conference on Web Search and Data Mining (WSDM’22). ACM, New York, NY, 813–823.Google ScholarDigital Library
- [40] . 2017. Personalizing session-based recommendations with hierarchical recurrent neural networks. In Proceedings of the 11th ACM Conference on Recommender Systems (RecSys’17). ACM, New York, NY, 130–137.Google ScholarDigital Library
- [41] . 2010. Factorization machines. In Proceedings of the 10th IEEE International Conference on Data Mining (ICDM’10). 995–1000.Google ScholarDigital Library
- [42] . 2010. Factorizing personalized Markov chains for next-basket recommendation. In Proceedings of the 19th International Conference on World Wide Web (WWW’10)ACM, New York, NY, 811–820.Google ScholarDigital Library
- [43] . 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management (CIKM’19). ACM, New York, NY, 1441–1450.Google ScholarDigital Library
- [44] . 2009. Learning to recommend questions based on user ratings. In Proceedings of the 18th ACM Conference on Information and Knowledge Management (CIKM’09). ACM, New York, NY, 751–758.Google ScholarDigital Library
- [45] . 2020. Multi-modal knowledge graphs for recommender systems. In Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM’20). 1405–1414.Google ScholarDigital Library
- [46] . 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2818–2826.Google ScholarCross Ref
- [47] . 2018. Personalized top-N sequential recommendation via convolutional sequence embedding. In Proceedings of the 11th ACM International Conference on Web Search and Data Mining (WSDM’18). ACM, New York, NY, 565–573.Google ScholarDigital Library
- [48] . 2019. Knowledge-aware graph neural networks with label smoothness regularization for recommender systems. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’19). ACM, New York, NY, 968–977.Google ScholarDigital Library
- [49] . 2021. StackRec: Efficient training of very deep sequential recommender models by iterative stacking. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’21). 357–366.Google ScholarDigital Library
- [50] . 2019. Session-based recommendation with graph neural networks. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI’19). 346–353.Google Scholar
- [51] . 2020. Distilling knowledge from pre-trained language models via text smoothing. arXiv preprint arXiv:2005.03848 (2020).Google Scholar
- [52] . 2020. Contrastive pre-training for sequential recommendation. CoRR abs/2010.14395 (2020).Google Scholar
- [53] . 2019. A simple convolutional generative network for next item recommendation. In Proceedings of the 12th ACM International Conference on Web Search and Data Mining (WSDM’19). ACM, New York, NY, 582–590.Google ScholarDigital Library
- [54] . 2019. Deep learning based recommender system: A survey and new perspectives. ACM Computing Surveys 52, 1 (2019), 1–38.Google ScholarDigital Library
- [55] . 2019. Feature-level deeper self-attention network for sequential recommendation. In Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI’19). 4320–4326.Google ScholarDigital Library
- [56] . 2021. Causal intervention for leveraging popularity bias in recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’21). ACM, New York, NY, 11–20.Google ScholarDigital Library
- [57] . 2022. RecBole 2.0: Towards a more up-to-date recommendation library. In Proceedings of the 31st ACM International Conference on Information and Knowledge Management. 4722–4726.Google ScholarDigital Library
- [58] . 2021. RecBole: Towards a unified, comprehensive and efficient framework for recommendation algorithms. In Proceedings of the 30th ACM International Conference on Information and Knowledge Management. 4653–4664.Google ScholarDigital Library
- [59] . 2020. S3-Rec: Self-supervised learning for sequential recommendation with mutual information maximization. In Proceedings of the 29th ACM International Conference on Information and Knowledge Management. 1893–1902.Google ScholarDigital Library
- [60] . 2021. CRSLab: An open-source toolkit for building conversational recommender system. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations. 185–193.Google ScholarCross Ref
- [61] . 2022. Filter-enhanced MLP is all you need for sequential recommendation. In Proceedings of the ACM Web Conference 2022 (WWW’22). 2388–2399.Google ScholarDigital Library
- [62] . 2020. Improving conversational recommender systems via knowledge graph based semantic fusion. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’20). 1006–1014.Google ScholarDigital Library
- [63] . 2020. Towards topic-guided conversational recommender system. In Proceedings of the 28th International Conference on Computational Linguistics. 4128–4139.Google ScholarCross Ref
- [64] . 2020. FreeLB: Enhanced adversarial training for natural language understanding. In Proceedings of the Conference on Learning Representations (ICLR’20).Google Scholar
Index Terms
- Enhancing Multi-View Smoothness for Sequential Recommendation Models
Recommendations
User Popularity Preference Aware Sequential Recommendation
Computational Science – ICCS 2023AbstractIn recommender systems, users’ preferences for item popularity are diverse and dynamic, which reveals the different items that users prefer. Therefore, identifying user popularity preferences are significant for personalized recommendations. ...
Sequential Recommendation with Dual Side Neighbor-based Collaborative Relation Modeling
WSDM '20: Proceedings of the 13th International Conference on Web Search and Data MiningSequential recommendation task aims to predict user preference over items in the future given user historical behaviors. The order of user behaviors implies that there are resourceful sequential patterns embedded in the behavior history which reveal the ...
Multi-interest Diversification for End-to-end Sequential Recommendation
Sequential recommenders capture dynamic aspects of users’ interests by modeling sequential behavior. Previous studies on sequential recommendations mostly aim to identify users’ main recent interests to optimize the recommendation accuracy; they often ...
Comments