Elsevier

Knowledge-Based Systems

Volume 213, 15 February 2021, 106426
Knowledge-Based Systems

Neural graph personalized ranking for Top-N Recommendation

https://doi.org/10.1016/j.knosys.2020.106426Get rights and content

Abstract

Personalized recommendation has been widely applied to many real-world services. Many of recent studies focus on collaborative filtering (CF) by deep neural networks, which pursue to predict users’ preference on items based on the past user–item interactions (e.g., a user rates an item). A general CF approach consists of two key modules, embedding representation learning and interaction modeling. In most existing methods, the embedding module is followed by the interaction modeling module, and the user–item interaction information is only emploited in interaction modeling directly. Existing methods, however, defectively overlook the correlation between users and items, as well as the inherent connection between embedding learning and the interaction information. To fill this gap, we propose neural graph personalized ranking (NGPR) which directly makes use of the user–item interaction information in embedding learning by incorporating the user–item interaction graph in embedding learning. Specifically, we construct the user–item interaction graph using de facto interaction between a user and an item. Correlation between users and items can also be reserved by concatenating representations of users and items in the entire procedure of embedding learning. Moreover, more complicated structures like multilayer perceptron (MLP) can be used in interaction modeling to make the most use of the representations, rather than simple linear transformation. We conduct extensive experiments on three public benchmarks and demonstrate the superior performance of the proposed NGPR model on personalized ranking task. In addition, our ablation studies verify that our novel design to incorporate the user–item interaction graph in embedding learning is effective.

Introduction

As the increasingly large scale of collected data from users, personalized recommendation [1], [2] plays a crucial role in alleviating information overload. It has been widely applied to many realistic services, ranging from E-commerce [3], advertising [4] to social media platforms [5]. Recently, due to the significant success of deep learning in many applications, collaborative filtering (CF) by deep neural networks has also drawn much attention. It aims to predict users’ preference on items based on the past user–item interactions. The pivotal challenge is therefore how to accurately model the patterns of users’ preference. Armed with deep neural networks [6], [7], [8], CF has gained more prevalence [9], [10], [11].

In general, a typical CF model [12], [13], [14] consists of two key components, i.e., the embedding learning module and the interaction modeling module. Embedding learning maps users and items to vectorized representations, while interaction modeling matches historical user–item interactions based on the representations. For example, matrix factorization (MF), a classic CF method, maps users and items into a common representation space directly and then simply adopts inner product to model user–item interaction. Recent advancement is to integrate deep neural network techniques into these two key components. For example, some of them build the embedding learning module by a deep neural network [15], [16], [17]. To this end, the collaborative knowledge base embedding method [15] and the collaborative deep learning method [16] improve the embedding learning by incorporating the deep representations from rich side information, while deep matrix factorization [17] uses a two path-way neural network architecture to replace the classic linear embedding module used in vanilla MF. Other examples work on the interaction modeling module [9], [11]. For instance, neural collaborative filtering in [9] replaces the inner product operation in classic MF with nonlinear neural networks. Deep collaborative filtering [11] utilizes multi-layer neural network to extract the embeddings and model the user–item interaction.

Neural network-based CF has indeed achieved success, but these methods suffer from two limitations. First, correlation between users and items cannot be directly encoded into embedding representation learning. Existing methods learn representations of users and items in a somewhat isolated way, leading to the loss of correlation between users and items. In fact, nevertheless, such correlation is essential, by which the interaction information can be thus directly used to learn representations. Second, connection between embedding learning and interaction modeling has been overlooked. Most existing methods only apply an inner product or bilinear decoder in the interaction modeling. Given that the embedding is represented by the neural network model, to sufficiently make use of the discriminative embedding features, interaction modeling can also apply more complicated structures rather than a simple inner product.

Several recent efforts have attempted to leverage the collaborative signals for embedding learning. These works either utilize user (or item) historical interaction vector (a column or a row of user–item interaction matrix) to learn its embedding, or construct the embedding by propagating the information based on user–item interaction graph. The former blends some noisy information, since the absence of historical interaction vector is a mixture of real negative feedback (the user dislike the item) and missing values (the user did not notice the item). In addition, using the whole historical interaction vector can cause efficiency problem (the number of users and items is usually huge). The latter learns embeddings based on the embedding propagation of user–item interaction graph. Such methods only use the interacted users or items to capture the collaborative signals, which avoids the redundant information. However, the existing graph-based methods mainly rely on the accumulation of the interacted user (or item) embeddings during embedding propagation process, without considering the latent affinity between the user (or item) embedding and the interacted neighbors’ embeddings. In this way, the learned embeddings are suboptimal. Furthermore, these methods neglect the key inherent relevance and use linear transformation to model user–item interaction (e.g., inner product and bilinear decoder). Such interaction modeling methods assume the latent representation space of embeddings is linear, which cannot be always guaranteed and cannot effectively extract the non-linear collaborative signals between user embeddings and item embeddings.

To address the above limitations, we propose a new recommendation model named neural graph personalized ranking (NGPR), which particularly incorporates the correlation between users and items into embedding learning and interaction modeling. Specifically, we devise a complementary embedding propagation layer, which updates a user’s (or an item’s) embedding by combining the aggregation of the interacted item (or user) embeddings and the latent affinity between the user (or item) embedding and the interacted neighbors’ embeddings. Then we utilize a multi-layer perceptron model to endow the interaction modeling with nonlinear transformation, which can effectively extract the nonlinear collaborative signals between user’s embedding and item’s embedding. We conduct extensive experiments on three public benchmarks, demonstrating the effectiveness of the proposed NGPR model. To summarize, this work makes the following main contributions

  • We highlight the critical significance of the inherent correlation between embedding learning and interaction modeling for utilizing the collaborative signal. The former needs to encode the user–item interaction information into embeddings, while the latter needs to decode the interaction information accurately to assist the prediction.

  • We propose a novel model named neural graph personalized ranking (NGPR), which captures the user–item geometric structure by exploiting rich complementary embedding propagation and endows the interaction modeling with nonlinear collaborative signals by imposing multi-layer neural network into embeddings.

  • We conduct empirical studies on three real-world data sets. Extensive results demonstrate the effectiveness of NGPR and verify the rationality of the design of embedding learning and interaction modeling.

The rest of the paper is organized as follows. We review related work in Section 2. We present neural graph personalized ranking (NGPR) in Section 3. We report the experimental results in Section 4, and conclude the paper in Section 5.

Section snippets

Related work

We review existing works on deep-based CF and graph-based CF, which are most relevant with this work. Here we highlight the differences with our NGPR.

The general CF model [3], [18], [19] consists of two key components: embedding learning and interaction modeling. The former aims to transform users and items to vectorized representations, while the latter aims to reconstruct historical user–item interaction based on the user and item embeddings. Recently, some studies have been proposed to

Methodology

In this section, we present our NGPR as illustrated in Fig. 1. The proposed model consists of two components: embedding learning and interaction modeling. The embedding learning module maps user and item IDs to embedding representations by aggregating the embeddings based on user–item graph, while the interaction modeling module matches the interaction of a user–item pair based on the learned embeddings. We then introduce the details and analyze the connection with existing methods.

Experiments

In this section, we conduct extensive experiments with the aim of answering the following research questions:

  • RQ1: Do our proposed NGPR method outperform the state-of-the-art collaborative filtering methods?

  • RQ2: How does NGPR method benefit from the embedding propagation layer and the interaction modeling layer?

  • RQ3: How do different hyper-parameter settings (e.g., number of negative samples, depth of interaction modeling layer, message dropout and node dropout) affect NGPR?

Datasets Description.

Conclusion

In this work, we investigate the inherent connection between embedding learning and the user–item interaction information in CF models. We devise a new framework NGPR, which directly fills the above connection by leveraging the user–item interaction graph and nonlinear interaction modeling. The key of NGPR is the newly proposed embedding propagation layer, based on which we allow the embedding representations of users and items to interact with each other to harvest the collaborative signals.

CRediT authorship contribution statement

Zhibin Hu: Conceptualization, Methodology, Software, Validation, Investigation, Writing - original draft, Writing - review & editing, Visualization. Jiachun Wang: Conceptualization, Methodology, Software, Validation, Investigation, Writing - original draft, Writing - review & editing, Visualization. Yan Yan: Conceptualization, Methodology, Writing - original draft, Writing - review & editing. Peilin Zhao: Conceptualization, Methodology, Writing - original draft, Writing - review & editing,

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

Prof. Chen is supported by the National Natural Science Foundation of China (Grant No. 62072186), the Guangdong Basic and Applied Basic Research Foundation, China (Grant No. 2019B1515130001), the Guangdong Special Branch Plans Young Talent with Scientific and Technological Innovation, China (Grant No. 2016TQ03X445), the Guangzhou Science and Technology Planning Project, China (Grant No. 201904010197) and the Opening Project of Guangdong Key Laboratory of Big Data Analysis and Processing, China

Zhibin Hu is currently a PhD candidate in the School of Software Engineering, South China University of Technology, China. He received the B.S. degree in Computer Science from South China Normal University. His research interests include data mining, large-scale machine learning, and deep learning.

References (36)

  • BergR.v.d. et al.

    Graph convolutional matrix completion

    (2017)
  • DengZ.-H. et al.

    Deepcf: A unified framework of representation learning and matching function learning in recommender system

    (2019)
  • KorenY. et al.

    Matrix factorization techniques for recommender systems

    IEEE Comput.

    (2009)
  • JamaliM. et al.

    A matrix factorization technique with trust propagation for recommendation in social networks

    (2010)
  • ZhangF. et al.

    Collaborative knowledge base embedding for recommender systems

  • WangH. et al.

    Collaborative deep learning for recommender systems

  • H.-J. Xue, X. Dai, J. Zhang, S. Huang, J. Chen, Deep matrix factorization models for recommender systems, in: IJCAI,...
  • MaH.

    An experimental study on implicit social recommendation

  • Cited by (16)

    • Ranking-based contrastive loss for recommendation systems

      2023, Knowledge-Based Systems
      Citation Excerpt :

      In this section, we briefly review related studies, which include top-k recommendation, contrastive loss and methods for handling negative samples. The GCN-based CF model has also been widely studied and has recently achieved good performance [15,23–26]. For example, neural graph collaborative filtering (NGCF) [23] exploits the structure of the user–item graph and its high-order connectivity by propagating embeddings on it.

    • A light heterogeneous graph collaborative filtering model using textual information

      2021, Knowledge-Based Systems
      Citation Excerpt :

      Further, considering the DNN-based representation learning and matching function learning suffered from two fundamental flaws, i.e., the limited expressiveness of inner product and the weakness in capturing low-rank relations respectively, Deng et al. [14] propose DeepCF, which combines the strengths of neural representation learning and neural matching function learning, to overcome such flaws. Most recently, based on a bipartite graph, NGPR (neural graph personalized ranking) [21] is proposed to capture the user–item geometric structure by exploiting rich complementary embedding propagation and endowing the interaction modeling with neural collaborative signals into embeddings by an MLP. Inspired by the success of NGPR and DeepCF, to better exploit textual features and learn a complex matching function, we design a new framework that combines graph-based representation learning with neural matching function learning.

    View all citing articles on Scopus

    Zhibin Hu is currently a PhD candidate in the School of Software Engineering, South China University of Technology, China. He received the B.S. degree in Computer Science from South China Normal University. His research interests include data mining, large-scale machine learning, and deep learning.

    Jiachun Wang is pursuing the Master degree in Software Engineering from South China University of Technology, Guangzhou, China. She received the B.S. degree in Network Engineering from Jinan University. Her research interests include recommendation systems, natural language processing and deep learning.

    Yan Yan is currently an assistant professor at School of Electrical Engineering and Computer Science, Washington State University. He receive PhD degree from University of Technology Sydney in 2018 and Bachelor degree from Tianjin University in 2013.

    Peilin Zhao is currently a Principal Researcher at Tencent AI Lab, China. Previously, he has worked at Rutgers University, Institute for Infocomm Research (I2R), Ant Financial Services Group. His research interests include: Online Learning, Recommendation System, Automatic Machine Learning, Deep Graph Learning, and Reinforcement Learning etc. He has published over 100 papers in top venues, including JMLR, ICML, KDD, etc. He has been invited as a PC member, reviewer or editor for many international conferences and journals, such as ICML, JMLR, etc. He received his bachelor’s degree from Zhejiang University, and his Ph.D. degree from Nanyang Technological University.

    Jian Chen is currently a Professor of the School of Software Engineering at South China University of Technology where she started as an Assistant Professor in 2005. She received her B.S. and Ph.D. degrees, both in Computer Science, from Sun Yat-Sen University, China, in 2000 and 2005 respectively. Her research interests can be summarized as developing effective and efficient data analysis techniques for complex data and the related applications.

    Jin Huang received his M.S. and Ph.D. degrees, both in Computer Science, from Sun Yat-Sen University, China, in 2006 and 2010. He is currently associate professor in the School of Computer Science at South China Normal University. His current research interests are in database, data mining and information retrieval.

    1

    Equal Contribution.

    View full text