Matrix factorization with heterogeneous multiclass preference context☆
Introduction
Recommendation systems [2], [3], [4], which are designed to help users pick out preferable items from a massive collection, benefit both customers and supplier with convenience and efficiency, and therefore attract a great deal of attention from both the academic and industrial communities.
Nowadays, apart from traditional user-item rating matrix data, an infinite variety of auxiliary data can be captured and utilized by recommendation systems to enhance recommendation performance, for they can reveal different external context information such as physical context (e.g., time [5], [6], location [7]) and social context (e.g., trust [8]). However, with the growing awareness of personal privacy, using rating matrix only to discover more internal context (latent collaborative pattern) [9], [10] is a more reliable and perpetually efficient strategy, and still occupies an important place in the research community. Different from context-aware recommendation systems [5], [6], [7], [8], [11] that make recommendations with external context, traditional recommendation systems, typically collaborative filtering recommendation systems [9], [10], [11], [12], [13] only consider internal context that is extracted form ratings assigned by users to items.
A recently proposed model called matrix factorization with multiclass preference context (MF-MPC) [10] is a unified method which combines the two major categories of collaborative filtering – neighborhood-based [11] and model-based [9], [12]. Briefly, MF-MPC is an improved method of SVD [12], which is also a matrix factorization (MF) method. Specifically, MF-MPC strengthens the MF method by modeling the multiclass preference context (MPC) of users as a newly added matrix, which also performs the same functions as the user similarity in a neighborhood-based method.
In this paper, we further introduce two MF models that contain not only user similarity but also item similarity, and collectively referred to as matrix factorization with heterogeneous multiclass preference context (MF-HMPC). More specifically, MF-HMPC consists of matrix factorization with dual multiclass preference context (MF-DMPC) and matrix factorization with pipelined multiclass preference context (MF-PMPC). MF-DMPC combines user-oriented MPC and item-oriented MPC in a concurrent way by joining them into one prediction formula. While in MF-PMPC, user-oriented MPC and item-oriented MPC are presented in different prediction rules for two sequential steps.
For empirical studies, we compare our two models against not only the two referenced models – SVD and MF-MPC, but also two representative deep learning models – RBM [14] and NCF [15]. We also analyze the advantage factors of these two proposed models by studying the independent and joint effect of the two types of MPC. Experimental results show that: (1) it is helpful to introduce MPC into MF models but the influences of user-oriented and item-oriented MPC differ according to the data characteristics; (2) MF-DMPC outperforms two single-oriented MF-MPC; (3) MF-PMPC can obtain better performance than MF-DMPC and it relies more on the first step training; (4) both MF-DMPC and MF-PMPC defeat the two deep learning methods, i.e., RBM and NCF. In general, our MF-HMPC that unifies MF-DMPC and MF-PMPC inherits both high accuracy of model-based recommendation algorithms and good explainability of neighborhood-based algorithms, and further strikes a good balance between user-oriented neighborhood information and item-oriented neighborhood information.
We organize the remainder of this paper as follows. We introduce some matrix factorization based and deep learning based recommendation algorithms that use rating matrix only in Section 2. We describe the rating prediction problem that we focus on, and make an interpretation of MPC in Section 3. We formulate our proposed framework MF-HMPC, which consists of MF with dual MPC in Section 4, and MF with pipelined MPC in Section 5. We conduct experiments of our proposed models and other related models, and analyze the results in Section 6. Finally, we give some concluding remarks in Section 7.
Section snippets
Related work
As mentioned above, discovering recommendation algorithms that exploit rating matrix only is still a research hotspot especially when novel technologies become popular. In this section, we introduce the traditional CF methods that our proposed models based on, as well as some deep learning methods that are in vogue recently.
Preliminaries
Before bringing out our proposed models, we need to formalize the rating prediction problem that we focus on, and make an interpretation of multiclass preference context (MPC), which is also our pointcut to upgrade the MF models.
Matrix factorization with dual multiclass preference context
Inspired by the differences between user-oriented and item-oriented collaborative filtering [11], we can infer that item similarity (item-oriented MPC) can also be introduced to improve the performance of matrix factorization models. Furthermore, thanks to the extendibility of MF models, we can hopefully join both user-oriented MPC and item-oriented MPC into the prediction rule so as to obtain a hybrid model, i.e., matrix factorization with dual multiclass preference context (MF-DMPC).
Matrix factorization with pipelined multiclass preference context
In the above section, user-oriented MPC and item-oriented MPC join together through the addition of < user-oriented MPC, item > interaction and < item-oriented MPC, user > interaction in an MF prediction rule. Intuitively, it is a concurrent way of combination. However, some studies have found that it can be remarkable if we combine two kinds of methods using the residual training strategy [24], [25]. For instance, we can obtain the predicted ratings through a
Experiments
In this section, we conduct extensive empirical studies in order to evaluate the recommendation performance of our proposed models and as well as to investigate the comparison and complementarity between the two single-oriented MPC.
Conclusions and future works
In this paper, we study a classical recommendation problem, i.e., rating prediction in a user-item matrix. In particular, we focus on exploiting the important but rarely studied internal context beneath users’ ratings, and develop a generic factorization-based framework, i.e., matrix factorization with heterogeneous multiclass preference context (MF-HMPC). We then design two specific variants with different structures, including MF with dual MPC (MF-DMPC) for concurrent structure and MF with
CRediT authorship contribution statement
Jing Lin: Conceptualization, Methodology, Software, Validation, Investigation, Writing - original draft. Weike Pan: Conceptualization, Methodology, Writing - review & editing, Supervision, Funding acquisition. Lin Li: Software, Validation, Investigation. Zixiang Chen: Software, Validation, Investigation. Zhong Ming: Resources, Supervision, Funding acquisition.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgement
We thank the handling Associate Editor and Reviewers for their efforts and constructive expert comments, and Mr. Yunfeng Huang for assistance in code review and helpful discussions. We thank the support of National Natural Science Foundation of China Nos. 61872249, 61836005 and 61672358. Weike Pan and Zhong Ming are the corresponding authors for this work.
Jing Lin received the B.S. degree in Electronic and Information Engineering from the Shenzhen University, Shenzhen, China, in 2018. She is currently a master student in the National Engineering Laboratory for Big Data System Computing Technology and the College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China. Her research interests include collaborative recommendation, deep learning and transfer learning. Contact her at [email protected].
References (28)
- et al.
Recommender systems survey
Knowl. Based Syst.
(2013) - et al.
Recommender system application developments
Decis. Support Syst.
(2015) - et al.
MF-DMPC: Matrix factorization with dual multiclass preference context for rating prediction
Proceedings of the International Conference on Web Services
(2018) - et al.
Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions
IEEE Trans. Knowl. Data Eng.
(2005) - et al.
Time-aware recommender systems: a comprehensive survey and analysis of existing evaluation protocols
User Model. User Adapt. Interact.
(2014) Collaborative filtering with temporal dynamics
Proceedings of the Fifteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
(2009)- et al.
Exploring the context of locations for personalized location recommendations
Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence
(2016) - et al.
Social collaborative filtering by trust
IEEE Trans. Pattern Anal. Mach. Intell.
(2017) Factorization meets the neighborhood: a multifaceted collaborative filtering model
Proceedings of the Fourteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
(2008)- et al.
Collaborative recommendation with multiclass preference context
IEEE Intell. Syst.
(2017)
Recommender Systems Handbook
Factorization machines with libfm
ACM Trans. Intell. Syst. Technol.
Item-based collaborative filtering recommendation algorithms
Proceedings of the Tenth International Conference on World Wide Web
Restricted boltzmann machines for collaborative filtering
Proceedings of the Twenty-Fourth International Conference on Machine Learning
Cited by (0)
Jing Lin received the B.S. degree in Electronic and Information Engineering from the Shenzhen University, Shenzhen, China, in 2018. She is currently a master student in the National Engineering Laboratory for Big Data System Computing Technology and the College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China. Her research interests include collaborative recommendation, deep learning and transfer learning. Contact her at [email protected].
Weike Pan received the Ph.D. degree in Computer Science and Engineering from the Hong Kong University of Science and Technology, Kowloon, Hong Kong, China, in 2012. He is currently an associate professor with the National Engineering Laboratory for Big Data System Computing Technology and the College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China. His research interests include transfer learning, recommender systems and machine learning. He has been active in professional services. He has served as an editorial board member of Neurocomputing, a co-guest editor of a special issue on big data of IEEE Intelligent Systems (2015–2016), an information officer of ACM Transactions on Intelligent Systems and Technology (2009–2015), and journal reviewer and conference/workshop PC member for dozens of journals, conferences and workshops. Contact him at [email protected].
Lin Li received the B.S. degree in Computer Science and Technology from the Shenzhen University, Shenzhen, China, in 2017. She is currently a master student in the National Engineering Laboratory for Big Data System Computing Technology and the College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China. Her research interests include collaborative recommendation, deep learning and transfer learning. Contact her at [email protected].
Zixiang Chen received the B.S. degree in Software Engineering from the Ludong University, Yantai, China, in 2016. He is currently a master student in the National Engineering Laboratory for Big Data System Computing Technology and the College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China. His research interests include collaborative recommendation and deep learning. Contact him at [email protected].
Zhong Ming received the Ph.D. degree in Computer Science and Technology from the Sun Yat-Sen University, Guangzhou, China, in 2003. He is currently a professor with the National Engineering Laboratory for Big Data System Computing Technology and the College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China. His research interests include software engineering and web intelligence. Contact him at [email protected].
- ☆
This work is an extension of our previous work [1]. In this paper, we have added the following new contents: (i) we have developed a new method, i.e., MF-PMPC, in Section 5; (ii) we have included new experimental results and associated analyses in Section 6; (iii) we have added more related works and more detailed descriptions of baseline methods in Sections 2–4; and (iv) we have made many improvements throughout the whole paper including figure illustrations and linguistic quality.