Elsevier

Neurocomputing

Volume 390, 21 May 2020, Pages 148-157
Neurocomputing

Matrix factorization with heterogeneous multiclass preference context

https://doi.org/10.1016/j.neucom.2020.01.060Get rights and content

Highlights

  • We extend a previous work on multiclass preference context (MPC) to both user-oriented MPC and itemoriented MPC.

  • We propose a unified framework called matrix factorization with heterogeneous multiclass preference context (MFHMPC).

  • We have MF with dual MPC (MF-DMPC) for concurrent structure and MF with pipelined MPC (MF-PMPC) for sequential structure.

  • We conduct extensive comparative empirical studies and show the effectiveness of our MF-DMPC and MF-PMPC.

Abstract

The spreading use of the Internet and big data technology has spawned the need for recommendation systems. However, to alleviate public anxiety about privacy, this paper advocates making recommendation with internal context, which refers to the implicit context hidden beneath the rating matrix only. Inspired by a recent work that embeds neighborhood information among users (as represented by multiclass preference context, MPC) into a factorization-based method, we extend it to both user-oriented and item-oriented, and put forward both user-oriented MPC and item-oriented MPC into a generic factorization framework called matrix factorization with heterogeneous MPC (MF-HMPC). In particular, we derive two specific recommendation methods from MF-HMPC with different structures, including MF with dual MPC (MF-DMPC) and MF with pipelined MPC (MF-PMPC), which are corresponding to concurrent structure and sequential structure, respectively. Extensive empirical studies on four public datasets show that our two forms of MF-HMPC outperform the referenced state-of-the-art MF methods as well as two representative deep learning methods. Moreover, we also discover some interesting facts about how these two kinds of internal contextual information complement each other. The main advantage of our methods is that they manage to strike a good balance between user-oriented neighborhood information and item-oriented neighborhood information.

Introduction

Recommendation systems [2], [3], [4], which are designed to help users pick out preferable items from a massive collection, benefit both customers and supplier with convenience and efficiency, and therefore attract a great deal of attention from both the academic and industrial communities.

Nowadays, apart from traditional user-item rating matrix data, an infinite variety of auxiliary data can be captured and utilized by recommendation systems to enhance recommendation performance, for they can reveal different external context information such as physical context (e.g., time [5], [6], location [7]) and social context (e.g., trust [8]). However, with the growing awareness of personal privacy, using rating matrix only to discover more internal context (latent collaborative pattern)  [9], [10] is a more reliable and perpetually efficient strategy, and still occupies an important place in the research community. Different from context-aware recommendation systems [5], [6], [7], [8], [11] that make recommendations with external context, traditional recommendation systems, typically collaborative filtering recommendation systems [9], [10], [11], [12], [13] only consider internal context that is extracted form ratings assigned by users to items.

A recently proposed model called matrix factorization with multiclass preference context (MF-MPC) [10] is a unified method which combines the two major categories of collaborative filtering – neighborhood-based [11] and model-based [9], [12]. Briefly, MF-MPC is an improved method of SVD [12], which is also a matrix factorization (MF) method. Specifically, MF-MPC strengthens the MF method by modeling the multiclass preference context (MPC) of users as a newly added matrix, which also performs the same functions as the user similarity in a neighborhood-based method.

In this paper, we further introduce two MF models that contain not only user similarity but also item similarity, and collectively referred to as matrix factorization with heterogeneous multiclass preference context (MF-HMPC). More specifically, MF-HMPC consists of matrix factorization with dual multiclass preference context (MF-DMPC) and matrix factorization with pipelined multiclass preference context (MF-PMPC). MF-DMPC combines user-oriented MPC and item-oriented MPC in a concurrent way by joining them into one prediction formula. While in MF-PMPC, user-oriented MPC and item-oriented MPC are presented in different prediction rules for two sequential steps.

For empirical studies, we compare our two models against not only the two referenced models – SVD and MF-MPC, but also two representative deep learning models – RBM [14] and NCF [15]. We also analyze the advantage factors of these two proposed models by studying the independent and joint effect of the two types of MPC. Experimental results show that: (1) it is helpful to introduce MPC into MF models but the influences of user-oriented and item-oriented MPC differ according to the data characteristics; (2) MF-DMPC outperforms two single-oriented MF-MPC; (3) MF-PMPC can obtain better performance than MF-DMPC and it relies more on the first step training; (4) both MF-DMPC and MF-PMPC defeat the two deep learning methods, i.e., RBM and NCF. In general, our MF-HMPC that unifies MF-DMPC and MF-PMPC inherits both high accuracy of model-based recommendation algorithms and good explainability of neighborhood-based algorithms, and further strikes a good balance between user-oriented neighborhood information and item-oriented neighborhood information.

We organize the remainder of this paper as follows. We introduce some matrix factorization based and deep learning based recommendation algorithms that use rating matrix only in Section 2. We describe the rating prediction problem that we focus on, and make an interpretation of MPC in Section 3. We formulate our proposed framework MF-HMPC, which consists of MF with dual MPC in Section 4, and MF with pipelined MPC in Section 5. We conduct experiments of our proposed models and other related models, and analyze the results in Section 6. Finally, we give some concluding remarks in Section 7.

Section snippets

Related work

As mentioned above, discovering recommendation algorithms that exploit rating matrix only is still a research hotspot especially when novel technologies become popular. In this section, we introduce the traditional CF methods that our proposed models based on, as well as some deep learning methods that are in vogue recently.

Preliminaries

Before bringing out our proposed models, we need to formalize the rating prediction problem that we focus on, and make an interpretation of multiclass preference context (MPC), which is also our pointcut to upgrade the MF models.

Matrix factorization with dual multiclass preference context

Inspired by the differences between user-oriented and item-oriented collaborative filtering [11], we can infer that item similarity (item-oriented MPC) can also be introduced to improve the performance of matrix factorization models. Furthermore, thanks to the extendibility of MF models, we can hopefully join both user-oriented MPC and item-oriented MPC into the prediction rule so as to obtain a hybrid model, i.e., matrix factorization with dual multiclass preference context (MF-DMPC).

Matrix factorization with pipelined multiclass preference context

In the above section, user-oriented MPC and item-oriented MPC join together through the addition of  < user-oriented MPC, item >  interaction U¯u·MPCVi·T and  < item-oriented MPC, user >  interaction V¯i·MPCUu·T in an MF prediction rule. Intuitively, it is a concurrent way of combination. However, some studies have found that it can be remarkable if we combine two kinds of methods using the residual training strategy [24], [25]. For instance, we can obtain the predicted ratings through a

Experiments

In this section, we conduct extensive empirical studies in order to evaluate the recommendation performance of our proposed models and as well as to investigate the comparison and complementarity between the two single-oriented MPC.

Conclusions and future works

In this paper, we study a classical recommendation problem, i.e., rating prediction in a user-item matrix. In particular, we focus on exploiting the important but rarely studied internal context beneath users’ ratings, and develop a generic factorization-based framework, i.e., matrix factorization with heterogeneous multiclass preference context (MF-HMPC). We then design two specific variants with different structures, including MF with dual MPC (MF-DMPC) for concurrent structure and MF with

CRediT authorship contribution statement

Jing Lin: Conceptualization, Methodology, Software, Validation, Investigation, Writing - original draft. Weike Pan: Conceptualization, Methodology, Writing - review & editing, Supervision, Funding acquisition. Lin Li: Software, Validation, Investigation. Zixiang Chen: Software, Validation, Investigation. Zhong Ming: Resources, Supervision, Funding acquisition.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgement

We thank the handling Associate Editor and Reviewers for their efforts and constructive expert comments, and Mr. Yunfeng Huang for assistance in code review and helpful discussions. We thank the support of National Natural Science Foundation of China Nos. 61872249, 61836005 and 61672358. Weike Pan and Zhong Ming are the corresponding authors for this work.

Jing Lin received the B.S. degree in Electronic and Information Engineering from the Shenzhen University, Shenzhen, China, in 2018. She is currently a master student in the National Engineering Laboratory for Big Data System Computing Technology and the College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China. Her research interests include collaborative recommendation, deep learning and transfer learning. Contact her at [email protected].

References (28)

  • J. Bobadilla et al.

    Recommender systems survey

    Knowl. Based Syst.

    (2013)
  • J. Lu et al.

    Recommender system application developments

    Decis. Support Syst.

    (2015)
  • J. Lin et al.

    MF-DMPC: Matrix factorization with dual multiclass preference context for rating prediction

    Proceedings of the International Conference on Web Services

    (2018)
  • G. Adomavicius et al.

    Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions

    IEEE Trans. Knowl. Data Eng.

    (2005)
  • P.G. Campos et al.

    Time-aware recommender systems: a comprehensive survey and analysis of existing evaluation protocols

    User Model. User Adapt. Interact.

    (2014)
  • Y. Koren

    Collaborative filtering with temporal dynamics

    Proceedings of the Fifteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

    (2009)
  • X. Liu et al.

    Exploring the context of locations for personalized location recommendations

    Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence

    (2016)
  • B. Yang et al.

    Social collaborative filtering by trust

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2017)
  • Y. Koren

    Factorization meets the neighborhood: a multifaceted collaborative filtering model

    Proceedings of the Fourteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

    (2008)
  • W. Pan et al.

    Collaborative recommendation with multiclass preference context

    IEEE Intell. Syst.

    (2017)
  • B.S. Francesco Ricci

    Recommender Systems Handbook

    (2015)
  • S. Rendle

    Factorization machines with libfm

    ACM Trans. Intell. Syst. Technol.

    (2012)
  • B. Sarwar et al.

    Item-based collaborative filtering recommendation algorithms

    Proceedings of the Tenth International Conference on World Wide Web

    (2001)
  • R. Salakhutdinov et al.

    Restricted boltzmann machines for collaborative filtering

    Proceedings of the Twenty-Fourth International Conference on Machine Learning

    (2007)
  • Cited by (0)

    Jing Lin received the B.S. degree in Electronic and Information Engineering from the Shenzhen University, Shenzhen, China, in 2018. She is currently a master student in the National Engineering Laboratory for Big Data System Computing Technology and the College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China. Her research interests include collaborative recommendation, deep learning and transfer learning. Contact her at [email protected].

    Weike Pan received the Ph.D. degree in Computer Science and Engineering from the Hong Kong University of Science and Technology, Kowloon, Hong Kong, China, in 2012. He is currently an associate professor with the National Engineering Laboratory for Big Data System Computing Technology and the College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China. His research interests include transfer learning, recommender systems and machine learning. He has been active in professional services. He has served as an editorial board member of Neurocomputing, a co-guest editor of a special issue on big data of IEEE Intelligent Systems (2015–2016), an information officer of ACM Transactions on Intelligent Systems and Technology (2009–2015), and journal reviewer and conference/workshop PC member for dozens of journals, conferences and workshops. Contact him at [email protected].

    Lin Li received the B.S. degree in Computer Science and Technology from the Shenzhen University, Shenzhen, China, in 2017. She is currently a master student in the National Engineering Laboratory for Big Data System Computing Technology and the College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China. Her research interests include collaborative recommendation, deep learning and transfer learning. Contact her at [email protected].

    Zixiang Chen received the B.S. degree in Software Engineering from the Ludong University, Yantai, China, in 2016. He is currently a master student in the National Engineering Laboratory for Big Data System Computing Technology and the College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China. His research interests include collaborative recommendation and deep learning. Contact him at [email protected].

    Zhong Ming received the Ph.D. degree in Computer Science and Technology from the Sun Yat-Sen University, Guangzhou, China, in 2003. He is currently a professor with the National Engineering Laboratory for Big Data System Computing Technology and the College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China. His research interests include software engineering and web intelligence. Contact him at [email protected].

    This work is an extension of our previous work [1]. In this paper, we have added the following new contents: (i) we have developed a new method, i.e., MF-PMPC, in Section 5; (ii) we have included new experimental results and associated analyses in Section 6; (iii) we have added more related works and more detailed descriptions of baseline methods in Sections 24; and (iv) we have made many improvements throughout the whole paper including figure illustrations and linguistic quality.

    View full text