Personalized knowledge-aware recommendation with collaborative and attentive graph convolutional networks
Introduction
The information explosion is creating serious challenges in various aspects of modern society, especially in online environments such as search engines, E-commerce, multimedia platforms, and news portals. People are overwhelmed by the huge number of choices and options before they can benefit from the abundance of information. To deal with these challenges, recommender systems have become a necessity for web applications.
Collaborative filtering has been widely adopted in recommender systems to predict user interests [1], [2], [3]. However, it suffers from the sparsity issue of user-item interaction data and the cold-start problem, which may greatly affect the quality of recommendation. To address these issues, many studies seek to leverage the rich side information of items such as properties of items.
A variety of existing methods use item attributes independently [5], which may not be sufficient to capture item relations. To improve performance, some recent studies [6], [7], [8] propose to model items and their attributes using the readily available knowledge graphs (KGs) such as YAGO, NELL, and DBpedia. KGs explicitly encode relations among entities (e.g., items and their attributes) in the form of triples: (head entity, relation, tail entity), and each triple represents a fact or some concept relation, e.g., (Interstellar, DirectedBy, Nolan). A large number of triples then form a heterogeneous graph containing rich semantic relations and complex connectivity patterns, which can be used to extract item relations and improve recommendation quality.
For KG-based recommendation, it is important to explicitly model high-order relations in both the bipartite graph of user-item interactions and the KG. For example, consider a movie recommendation task illustrated in Fig. 1. User may want to watch movie (“Interstellar”) for either one of the following reasons: 1) User , who has watched the same movie (“Contact”) as , also watched movie (path ); 2) Both movies and are science fiction and share a common leading actor Matthew (paths ). Modeling high-order relations allows to capture these collaborative signals and informative KG relations for recommendation. On the other hand, it is necessary to design mechanisms to capture user-specific fine-grained semantics of the KG for recommendation. For example, user might be more interested in movie (path ) than movie (path ) because both and are directed by his/her favorite director Nolan and he/she cares more about the director (Nolan) than the leading actor (Matthew) when selecting a movie. Modeling user-specific relation preference is therefore critical to achieve personalized recommendation. Furthermore, entities might have a different importance weight in representing their common neighbor. For instance, entity (Matthew) as the leading actor should weigh in more than the supporting actor (Michael) in characterizing movie (“Interstellar”) in general. However, how to design effective mechanisms to capture these fine-grained semantics remains a challenging problem.
In recent years, substantial efforts have been devoted to designing knowledge-aware recommender systems [6], [8], [9]. Some studies [10], [11] adopt path-based approaches to explicitly model high-order relations among users and items. However, they require manually designed meta-paths for each specific heterogeneous graph and thus are heavily dependent on domain knowledge. Some regularization-based methods [6], [12] enhance collaborative filtering models with KG embedding techniques so as to capture the complex relationships among items. For example, CKE [6] uses TransR [13] to encode the translational property of KG embeddings. Albeit being flexible, these methods only implicitly refine item embeddings with a KG embedding loss, which are insufficient to deal with KG-based recommendation tasks.
Most recently, recommendation methods based on graph neural networks have achieved state-of-the-art performance and attracted significant attention. KGCN [7] and KGNN-LS [14] obtain the embedding of an entity in KG by uniformly aggregating the embeddings of its same-relation neighbors. Although they aggregate neighbors of different relations with different weights, they completely ignore intra-relation semantics. Besides, they do not explicitly model the user-item graph and hence might be inadequate in extracting collaborative signals. KGAT [8] incorporates user-item interactions into a KG by representing each interaction as a triple (user, Interact, item). It then models high-order relations with graph attention networks on the extended KG. However, its attention mechanism does not model user-specific preferences on entities and relations. Also, the collaborative signals from user-item interactions may be overwhelmed by irrelevant attributes from the KG, especially in scenarios where user-item interactions are sparse (as demonstrated in Sections 5.2 and 5.3 empirically).
To address these issues, in this paper, we propose a novel framework with collaborative and attentive graph convolutional networks (COAT) for personalized knowledge-aware recommendation. To effectively capture both the collaborative signals and the attribute-based relations, COAT models the user-item graph and the KG separately and simultaneously with two different graph neural networks (GNNs). Since the user-item graph and the KG have distinct structural properties and connectivity patterns (e.g., community structure and degree distribution), they need to be modeled by GNNs with different design features.
In this regard, our COAT framework offers greater flexibility than existing models, especially KGAT that forms an extended KG by integrating user-item interactions and models it with a single graph neural network. It also has clear advantages over KGCN and KGNN-LS, which fail to model high-order user-item interactions. In particular, on the user-item graph, an efficient graph convolutional network is leveraged to encode high-order collaborative signals for learning effective user and item embeddings and tackling the sparsity issue.
This gives another advantage of our COAT framework in handling very sparse user-item graphs (as shown in Section 5.3). On the KG, a knowledge graph attention network is proposed for learning entity embeddings, where each layer is equipped with a novel personalized KG-aware attention mechanism for capturing user-specific fine-grained semantics.
The proposed attention mechanism and variants jointly consider user features, target entity, entity relation, and neighbor entity to achieve personalization. Compared with the attention mechanisms used in KGCN and KGAT, it has the advantage in capturing more fine-grained semantic information for modeling user preferences. By making same entities (items) in the user-item graph and the KG share the same embeddings during training (Section 4.2), our model is capable to effectively capture high-order relations across the user-item graph and the KG with the cooperation of the two GNNs.
The main contributions of this paper include the following:
- •
We propose a new framework COAT based on collaborative and attentive graph convolutional networks for knowledge-aware recommendation, which effectively addresses the limitations of existing GNN-based methods and combines their advantages in a holistic system.
- •
We design novel personalized knowledge-aware attention mechanisms to capture user-specific fine-grained semantics in the KG to achieve more personalized recommendation.
- •
We conduct extensive experiments to evaluate our model COAT on four benchmark datasets for top- recommendation and click-through rate prediction. The results demonstrate its competitive advantages over a variety of state-of-the-art methods.
The organization of this paper is as follows. In Section 2, we provide a review of current literature. In Section 3, we formulate the research problem. We present a detailed description of the proposed method in Section 4. The experimental results and analysis are provided in Section 5. Finally, a short summary is included in Section 6.
Section snippets
Graph neural networks
Graph neural networks, particularly graph convolutional neural networks generalize the convolutional operator defined on Euclidean data such as image (2D grid) and text (1D sequence) to non-regular graph-structured data [15]. They can be mainly categorized as spectral-based methods [16], [17] and spatial-based methods [18], [19]. Spectral-based methods define graph convolution in the Fourier domain based on graph signal processing theory [20], [21]. Spectral CNN [16] exploits the spectrum of
User-item graph (UI)
We consider implicit feedback data as in many recommendation methods [6], [8]. Specifically, there are a set of users and a set of items . The interactions between users and items are represented as a rating matrix , where if user has engaged with item such as listening, reading or watching, otherwise . can be considered as the biadjacency matrix of a bipartite graph between users and items. Based on this, we can construct a graph termed
Model overview
In this paper, we propose a new framework with collaborative and attentive graph convolutional networks (COAT) to jointly model the user-item graph and the knowledge graph for recommendation. The model framework is shown in Fig. 2. It consists of an embedding learning component and a prediction component. In the embedding learning component, an efficient graph convolutional network preserves collaborative signals in the refined user and item embeddings from the UI, while a personalized
Experiments
In this section, we empirically evaluate our proposed method COAT in two scenarios including click-through rate (CTR) prediction and top- recommendation on benchmark datasets. We aim to answer the following research questions through experiments:
- RQ1
How does COAT perform compared with state-of-the-art methods?
- RQ2
How do different variants of COAT perform under scenarios with very sparse user-item interactions?
- RQ3
How does the performance contribution of the proposed personalized knowledge-aware attention
Conclusion
In this paper, we have proposed a novel and effective framework with collaborative and attentive graph convolutional networks (COAT) for personalized knowledge-aware recommendation. COAT is composed of two embedding learning modules: an efficient graph convolutional module for extracting collaborative signals from the user-item graph and a personalized knowledge graph attention module for capturing fine-grained semantics in the KG.
The strengths of our method over existing works lie in its
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This research was supported by the grants of P0001175 (ZVJJ) and P0030935 (ZVPY) funded by PolyU (UGC) and P0034058 (ZGAL) and P0038850 (ZGD1) funded by [[1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44], [45]] Alibaba Group.
Quanyu Dai is currently a researcher at Huawei Noah’s Ark Lab. He received the Ph.D. degree at The Hong Kong Polytechnic University. His research interests include graphbased algorithms and recommender systems. He has publications appeared in the toptier journals and conferences, such as TKDE, TNNLS, IJCAI, AAAI and WWW.
References (45)
- et al.
A cross-domain collaborative filtering algorithm with expanding user and item features via the latent factor space of auxiliary domains
Pattern Recognit.
(2019) - et al.
Joint interaction with context operation for collaborative filtering
Pattern Recognit.
(2019) The area above the ordinal dominance graph and the area below the receiver operating characteristic graph
J. Math. Psychol.
(1975)- et al.
Collaborative filtering for implicit feedback datasets
ICDM
(2008) - et al.
Fast context-aware recommendations with factorization machines
SIGIR
(2011) - et al.
Neural factorization machines for sparse predictive analytics
SIGIR
(2017) - et al.
Collaborative knowledge base embedding for recommender systems
KDD
(2016) - et al.
Knowledge graph convolutional networks for recommender systems
WWW
(2019) - et al.
KGAT: knowledge graph attention network for recommendation
KDD
(2019) - et al.
RippleNet: propagating user preferences on the knowledge graph for recommender systems
CIKM
(2018)
Personalized entity recommendation: a heterogeneous information network approach
WSDM
Leveraging meta-path based context for top- N recommendation with A neural co-attention model
KDD
Unifying knowledge graph learning and recommendation: towards a better understanding of user preferences
WWW
Learning entity and relation embeddings for knowledge graph completion
AAAI
Knowledge-aware graph neural networks with label smoothness regularization for recommender systems
KDD
Deep learning on graphs: a survey
IEEE Trans. Knowl. Data Eng.
Spectral networks and deep locally connected networks on graphs
ICLR
Semi-supervised classification with graph convolutional networks
ICLR
Inductive representation learning on large graphs
NIPS
Graph attention networks
ICLR
The emerging field of signal processing on graphs: extending high-dimensional data analysis to networks and other irregular domains
IEEE Signal Process. Mag.
Discrete signal processing on graphs: sampling theory
IEEE Trans. Signal Process.
Cited by (21)
AG-Meta: Adaptive graph meta-learning via representation consistency over local subgraphs
2024, Pattern RecognitionGraph Convolutional Network with elastic topology
2024, Pattern RecognitionTowards fair and personalized federated recommendation
2024, Pattern RecognitionExploring global information for session-based recommendation
2024, Pattern RecognitionPersonalized recommendation via inductive spatiotemporal graph neural network
2024, Pattern Recognition
Quanyu Dai is currently a researcher at Huawei Noah’s Ark Lab. He received the Ph.D. degree at The Hong Kong Polytechnic University. His research interests include graphbased algorithms and recommender systems. He has publications appeared in the toptier journals and conferences, such as TKDE, TNNLS, IJCAI, AAAI and WWW.
Xiao-Ming Wu is currently an assistant professor at the Department of Computing, The Hong Kong Polytechnic University. She works on machine learning and artificial intelligence. Her research has contributed to new theoretical insights of machine learning algorithms and novel methods for artificial intelligence applications in various fields. She publishes in leading venues including NeurIPS, CVPR, EMNLP, IJCAI and AAAI.
Lu Fan is currently working toward the M.Phil. degree at the Department of Computing, The Hong Kong Polytechnic University. Her research interests include natural language understanding and graph-based applications.
Qimai Li is currently a Ph.D. student at the Department of Computing, The Hong Kong Polytechnic University. He obtained B.Eng. degree from the College of Computer Science and Technology, Zhejiang University in 2017. His main area of interest includes machine learning on graph, semi-supervised Learning and graph signal processing.
Han Liu is currently an associate professor in the School of Software, Dalian University of Technology. He has published over 20 papers on journals and conferences, including TKDE, Neural Networks, ACL, CVPR, IJCAI, AAAI, etc. His research interests include machine learning, artificial intelligence and data mining.
Xiaotong Zhang is currently an associate research professor in the School of Software, Dalian University of Technology. She has published over 20 papers on journals and conferences, including TKDE, Neural Networks, ACL, CVPR, IJCAI, AAAI, etc. Her research interests include machine learning, artificial intelligence and data mining.
Dan Wang is currently an associate professor at Department of Computing, The Hong Kong Polytechnic University. He is an expert in computer networking, and he is recently working in the inter-discipline domains of smart energy systems, industry 4.0. He publishes extensively in top networking conferences, such as SIGCOMM, SIGMETRICS, INFOCOM and in top inter-discipline conference, such as ACM e-Energy, ACM Buildsys.
Guli Lin is currently an Engineer at Alibaba Group. His research interests are mainly in applying machine learning for Recommendation and Product Search. He received his Ph.D. degree in computer science from South China University of Technology.
Keping Yang is currently a senior algorithm expert at Search and Recommendation Department of Alibaba. She is working on machine learning and artificial intelligence since she received her Master degree from Zhejiang University. Her team publishes many papers in top conferences, such as KDD, SIGMOD, WWW, IJCAI, AAAI, etc.
- 1
This work was done while the author was a student in the Department of Computing, The Hong Kong Polytechnic University.