Elsevier

Pattern Recognition

Volume 128, August 2022, 108628
Pattern Recognition

Personalized knowledge-aware recommendation with collaborative and attentive graph convolutional networks

https://doi.org/10.1016/j.patcog.2022.108628Get rights and content

Highlights

  • A new framework (COAT) is proposed for personalized knowledge-aware recommendation.

  • Collaborative and attentive GNNs are designed to jointly model the UI and KG graphs.

  • Novel attention mechanisms are designed to achieve personalization.

  • An efficient graph convolutional layer is employed to tackle the sparsity issue.

  • COAT outperforms 10 state-of-the-art recommendation methods on benchmark datasets.

Abstract

Knowledge graphs (KGs) are increasingly used to solve the data sparsity and cold start problems of collaborative filtering. Recently, graph neural networks (GNNs) have been applied to build KG-based recommender systems and achieved competitive performance. However, existing GNN-based methods are either limited in their ability to capture fine-grained semantics in a KG, or insufficient in effectively modeling user-item interactions. To address these issues, we propose a novel framework with collaborative and attentive graph convolutional networks for personalized knowledge-aware recommendation. Particularly, we model the user-item graph and the KG separately and simultaneously with an efficient graph convolutional network and a personalized knowledge graph attention network, where the former aims to extract informative collaborative signals, while the latter is designed to capture fine-grained semantics. Collectively, they are able to learn meaningful node representations for predicting user-item interactions. Extensive experiments on benchmark datasets demonstrate the effectiveness of the proposed method compared with state-of-the-arts.

Introduction

The information explosion is creating serious challenges in various aspects of modern society, especially in online environments such as search engines, E-commerce, multimedia platforms, and news portals. People are overwhelmed by the huge number of choices and options before they can benefit from the abundance of information. To deal with these challenges, recommender systems have become a necessity for web applications.

Collaborative filtering has been widely adopted in recommender systems to predict user interests [1], [2], [3]. However, it suffers from the sparsity issue of user-item interaction data and the cold-start problem, which may greatly affect the quality of recommendation. To address these issues, many studies seek to leverage the rich side information of items such as properties of items.

A variety of existing methods use item attributes independently [5], which may not be sufficient to capture item relations. To improve performance, some recent studies [6], [7], [8] propose to model items and their attributes using the readily available knowledge graphs (KGs) such as YAGO, NELL, and DBpedia. KGs explicitly encode relations among entities (e.g., items and their attributes) in the form of triples: (head entity, relation, tail entity), and each triple represents a fact or some concept relation, e.g., (Interstellar, DirectedBy, Nolan). A large number of triples then form a heterogeneous graph containing rich semantic relations and complex connectivity patterns, which can be used to extract item relations and improve recommendation quality.

For KG-based recommendation, it is important to explicitly model high-order relations in both the bipartite graph of user-item interactions and the KG. For example, consider a movie recommendation task illustrated in Fig. 1. User u1 may want to watch movie i3 (“Interstellar”) for either one of the following reasons: 1) User u2, who has watched the same movie i2 (“Contact”) as u1, also watched movie i3 (path u1i2u2i3); 2) Both movies i2 and i3 are science fiction and share a common leading actor Matthew (paths u1i2e5/e7i3). Modeling high-order relations allows to capture these collaborative signals and informative KG relations for recommendation. On the other hand, it is necessary to design mechanisms to capture user-specific fine-grained semantics of the KG for recommendation. For example, user u3 might be more interested in movie i1 (path u3i3e6i1) than movie i2 (path u3i3e5i2) because both i1 and i3 are directed by his/her favorite director Nolan and he/she cares more about the director (Nolan) than the leading actor (Matthew) when selecting a movie. Modeling user-specific relation preference is therefore critical to achieve personalized recommendation. Furthermore, entities might have a different importance weight in representing their common neighbor. For instance, entity e5 (Matthew) as the leading actor should weigh in more than the supporting actor e4 (Michael) in characterizing movie i3 (“Interstellar”) in general. However, how to design effective mechanisms to capture these fine-grained semantics remains a challenging problem.

In recent years, substantial efforts have been devoted to designing knowledge-aware recommender systems [6], [8], [9]. Some studies [10], [11] adopt path-based approaches to explicitly model high-order relations among users and items. However, they require manually designed meta-paths for each specific heterogeneous graph and thus are heavily dependent on domain knowledge. Some regularization-based methods [6], [12] enhance collaborative filtering models with KG embedding techniques so as to capture the complex relationships among items. For example, CKE [6] uses TransR [13] to encode the translational property of KG embeddings. Albeit being flexible, these methods only implicitly refine item embeddings with a KG embedding loss, which are insufficient to deal with KG-based recommendation tasks.

Most recently, recommendation methods based on graph neural networks have achieved state-of-the-art performance and attracted significant attention. KGCN [7] and KGNN-LS [14] obtain the embedding of an entity in KG by uniformly aggregating the embeddings of its same-relation neighbors. Although they aggregate neighbors of different relations with different weights, they completely ignore intra-relation semantics. Besides, they do not explicitly model the user-item graph and hence might be inadequate in extracting collaborative signals. KGAT [8] incorporates user-item interactions into a KG by representing each interaction as a triple (user, Interact, item). It then models high-order relations with graph attention networks on the extended KG. However, its attention mechanism does not model user-specific preferences on entities and relations. Also, the collaborative signals from user-item interactions may be overwhelmed by irrelevant attributes from the KG, especially in scenarios where user-item interactions are sparse (as demonstrated in Sections 5.2 and 5.3 empirically).

To address these issues, in this paper, we propose a novel framework with collaborative and attentive graph convolutional networks (COAT) for personalized knowledge-aware recommendation. To effectively capture both the collaborative signals and the attribute-based relations, COAT models the user-item graph and the KG separately and simultaneously with two different graph neural networks (GNNs). Since the user-item graph and the KG have distinct structural properties and connectivity patterns (e.g., community structure and degree distribution), they need to be modeled by GNNs with different design features.

In this regard, our COAT framework offers greater flexibility than existing models, especially KGAT that forms an extended KG by integrating user-item interactions and models it with a single graph neural network. It also has clear advantages over KGCN and KGNN-LS, which fail to model high-order user-item interactions. In particular, on the user-item graph, an efficient graph convolutional network is leveraged to encode high-order collaborative signals for learning effective user and item embeddings and tackling the sparsity issue.

This gives another advantage of our COAT framework in handling very sparse user-item graphs (as shown in Section 5.3). On the KG, a knowledge graph attention network is proposed for learning entity embeddings, where each layer is equipped with a novel personalized KG-aware attention mechanism for capturing user-specific fine-grained semantics.

The proposed attention mechanism and variants jointly consider user features, target entity, entity relation, and neighbor entity to achieve personalization. Compared with the attention mechanisms used in KGCN and KGAT, it has the advantage in capturing more fine-grained semantic information for modeling user preferences. By making same entities (items) in the user-item graph and the KG share the same embeddings during training (Section 4.2), our model is capable to effectively capture high-order relations across the user-item graph and the KG with the cooperation of the two GNNs.

The main contributions of this paper include the following:

  • We propose a new framework COAT based on collaborative and attentive graph convolutional networks for knowledge-aware recommendation, which effectively addresses the limitations of existing GNN-based methods and combines their advantages in a holistic system.

  • We design novel personalized knowledge-aware attention mechanisms to capture user-specific fine-grained semantics in the KG to achieve more personalized recommendation.

  • We conduct extensive experiments to evaluate our model COAT on four benchmark datasets for top-K recommendation and click-through rate prediction. The results demonstrate its competitive advantages over a variety of state-of-the-art methods.

The organization of this paper is as follows. In Section 2, we provide a review of current literature. In Section 3, we formulate the research problem. We present a detailed description of the proposed method in Section 4. The experimental results and analysis are provided in Section 5. Finally, a short summary is included in Section 6.

Section snippets

Graph neural networks

Graph neural networks, particularly graph convolutional neural networks generalize the convolutional operator defined on Euclidean data such as image (2D grid) and text (1D sequence) to non-regular graph-structured data [15]. They can be mainly categorized as spectral-based methods [16], [17] and spatial-based methods [18], [19]. Spectral-based methods define graph convolution in the Fourier domain based on graph signal processing theory [20], [21]. Spectral CNN [16] exploits the spectrum of

User-item graph (UI)

We consider implicit feedback data as in many recommendation methods [6], [8]. Specifically, there are a set of users U={u1,u2,,uN} and a set of items I={i1,i2,,iM}. The interactions between users and items are represented as a rating matrix YRN×M, where ynm=1 if user un has engaged with item im such as listening, reading or watching, otherwise ynm=0. Y can be considered as the biadjacency matrix of a bipartite graph between users and items. Based on this, we can construct a graph termed

Model overview

In this paper, we propose a new framework with collaborative and attentive graph convolutional networks (COAT) to jointly model the user-item graph and the knowledge graph for recommendation. The model framework is shown in Fig. 2. It consists of an embedding learning component and a prediction component. In the embedding learning component, an efficient graph convolutional network preserves collaborative signals in the refined user and item embeddings from the UI, while a personalized

Experiments

In this section, we empirically evaluate our proposed method COAT in two scenarios including click-through rate (CTR) prediction and top-K recommendation on benchmark datasets. We aim to answer the following research questions through experiments:

  • RQ1

    How does COAT perform compared with state-of-the-art methods?

  • RQ2

    How do different variants of COAT perform under scenarios with very sparse user-item interactions?

  • RQ3

    How does the performance contribution of the proposed personalized knowledge-aware attention

Conclusion

In this paper, we have proposed a novel and effective framework with collaborative and attentive graph convolutional networks (COAT) for personalized knowledge-aware recommendation. COAT is composed of two embedding learning modules: an efficient graph convolutional module for extracting collaborative signals from the user-item graph and a personalized knowledge graph attention module for capturing fine-grained semantics in the KG.

The strengths of our method over existing works lie in its

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This research was supported by the grants of P0001175 (ZVJJ) and P0030935 (ZVPY) funded by PolyU (UGC) and P0034058 (ZGAL) and P0038850 (ZGD1) funded by [[1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44], [45]] Alibaba Group.

Quanyu Dai is currently a researcher at Huawei Noah’s Ark Lab. He received the Ph.D. degree at The Hong Kong Polytechnic University. His research interests include graphbased algorithms and recommender systems. He has publications appeared in the toptier journals and conferences, such as TKDE, TNNLS, IJCAI, AAAI and WWW.

References (45)

  • X. Yu et al.

    Personalized entity recommendation: a heterogeneous information network approach

    WSDM

    (2014)
  • B. Hu et al.

    Leveraging meta-path based context for top- N recommendation with A neural co-attention model

    KDD

    (2018)
  • Y. Cao et al.

    Unifying knowledge graph learning and recommendation: towards a better understanding of user preferences

    WWW

    (2019)
  • Y. Lin et al.

    Learning entity and relation embeddings for knowledge graph completion

    AAAI

    (2015)
  • H. Wang et al.

    Knowledge-aware graph neural networks with label smoothness regularization for recommender systems

    KDD

    (2019)
  • Z. Zhang et al.

    Deep learning on graphs: a survey

    IEEE Trans. Knowl. Data Eng.

    (2020)
  • J. Bruna et al.

    Spectral networks and deep locally connected networks on graphs

    ICLR

    (2014)
  • T.N. Kipf et al.

    Semi-supervised classification with graph convolutional networks

    ICLR

    (2017)
  • W.L. Hamilton et al.

    Inductive representation learning on large graphs

    NIPS

    (2017)
  • P. Velickovic et al.

    Graph attention networks

    ICLR

    (2018)
  • D.I. Shuman et al.

    The emerging field of signal processing on graphs: extending high-dimensional data analysis to networks and other irregular domains

    IEEE Signal Process. Mag.

    (2013)
  • S. Chen et al.

    Discrete signal processing on graphs: sampling theory

    IEEE Trans. Signal Process.

    (2015)
  • Cited by (21)

    View all citing articles on Scopus

    Quanyu Dai is currently a researcher at Huawei Noah’s Ark Lab. He received the Ph.D. degree at The Hong Kong Polytechnic University. His research interests include graphbased algorithms and recommender systems. He has publications appeared in the toptier journals and conferences, such as TKDE, TNNLS, IJCAI, AAAI and WWW.

    Xiao-Ming Wu is currently an assistant professor at the Department of Computing, The Hong Kong Polytechnic University. She works on machine learning and artificial intelligence. Her research has contributed to new theoretical insights of machine learning algorithms and novel methods for artificial intelligence applications in various fields. She publishes in leading venues including NeurIPS, CVPR, EMNLP, IJCAI and AAAI.

    Lu Fan is currently working toward the M.Phil. degree at the Department of Computing, The Hong Kong Polytechnic University. Her research interests include natural language understanding and graph-based applications.

    Qimai Li is currently a Ph.D. student at the Department of Computing, The Hong Kong Polytechnic University. He obtained B.Eng. degree from the College of Computer Science and Technology, Zhejiang University in 2017. His main area of interest includes machine learning on graph, semi-supervised Learning and graph signal processing.

    Han Liu is currently an associate professor in the School of Software, Dalian University of Technology. He has published over 20 papers on journals and conferences, including TKDE, Neural Networks, ACL, CVPR, IJCAI, AAAI, etc. His research interests include machine learning, artificial intelligence and data mining.

    Xiaotong Zhang is currently an associate research professor in the School of Software, Dalian University of Technology. She has published over 20 papers on journals and conferences, including TKDE, Neural Networks, ACL, CVPR, IJCAI, AAAI, etc. Her research interests include machine learning, artificial intelligence and data mining.

    Dan Wang is currently an associate professor at Department of Computing, The Hong Kong Polytechnic University. He is an expert in computer networking, and he is recently working in the inter-discipline domains of smart energy systems, industry 4.0. He publishes extensively in top networking conferences, such as SIGCOMM, SIGMETRICS, INFOCOM and in top inter-discipline conference, such as ACM e-Energy, ACM Buildsys.

    Guli Lin is currently an Engineer at Alibaba Group. His research interests are mainly in applying machine learning for Recommendation and Product Search. He received his Ph.D. degree in computer science from South China University of Technology.

    Keping Yang is currently a senior algorithm expert at Search and Recommendation Department of Alibaba. She is working on machine learning and artificial intelligence since she received her Master degree from Zhejiang University. Her team publishes many papers in top conferences, such as KDD, SIGMOD, WWW, IJCAI, AAAI, etc.

    1

    This work was done while the author was a student in the Department of Computing, The Hong Kong Polytechnic University.

    View full text