Knowledge graph enhanced neural collaborative recommendation
Introduction
As the amount of data from different platforms grows at an unprecedented rate, the recommender system is becoming more and more crucial to help users discover personalised content of interest from these ever-growing corpus of data. For example, there are 11,895 movies released around the world in 2018.1 It is impossible for users to watch all available movies to identify their interested ones. Due to their extraordinary capability of exploiting collective wisdom and experiences, Collaborative Filtering (CF) algorithms, especially Matrix Factorisation (MF) algorithms — a technique that predicts users’ personalised preference from user–item interactions only — has proven to be the most widely used recommendation technology (Wang, Deng et al., 2018). Matrix factorisation assumes that there are some latent factors that exist behind the interactions between users and items. Though widely studied in the past, traditional matrix factorisation suffers from the severe sparsity problem of user–item interaction (Sun et al., 2019).
Inspired by the recent popularity of deep learning, Neural Collaborative Filtering (NCF) is proposed, as a new class of CF methods, cast the traditional MF algorithm into an overall neural framework (Deng et al., 2019, He, Du et al., 2018, He et al., 2017). Generally speaking, there are two key components in learnable NCF models: (1) embedding, which transforms users and items to vectorised representations, and (2) interaction modelling, which reconstructs historical interactions based on the user/item embeddings. For example, in He et al. (2017), the authors proposed to replace the MF interaction function of inner product with nonlinear deep neural networks. The translation-based CF models were proposed to use Euclidean distance metric as the interaction function (Tay et al., 2018). However, these methods may not be sufficient to yield satisfactory embeddings for CF. The possible reason is that most existing methods build the embedding function with the descriptive features only (e.g., ID or attributes). As a result, when the interaction function goes deeper to capture complex user–item relationship, these methods tend to overfit and make the sparsity problem even worse.
Therefore, researchers proposed using auxiliary data in recommender systems to enrich the semantics of the user or item representation, such as social networks, attributes, and multimedia (Hu, Zhang et al., 2018, Luo et al., 2019, Qian et al., 2018, Symeonidis and Malakoudis, 2019, Zheng et al., 2018). Recently, Knowledge Graphs (KGs) have attracted increasing attention, which usually consist of fruitful connected facts about items (Cao et al., 2019, Sourabh and Chowdary, 2019, Wang, Zhang, Zhao et al., 2019). KG introduces semantic relatedness among items, which is a useful tool to help mine their latent connections and improve the quality of recommendation. Extra connectivity information derived from KG endows recommender systems the ability of reasoning and alleviates sparsity problem (Wang, He et al., 2019). Taking movie recommendation as an example in Fig. 1, a user is connected to the movie “The Revenant” since she likes the movie “Inception” by the same actor “Leonardo DiCaprio”. Such connectivity helps to reason about unseen user–item interactions (i.e., a potential recommendation) by synthesising information from long paths.
The prior works on KG-based recommendation can be roughly categorised into two categories considering the path and embedding fashion. (1) Path-based methods use predefined meta paths with specific connectivity patterns to capture different semantic similarities between users and items carried in KGs. Path-based methods exploit KG in an intuitive way, but they fail to automatically uncover and reason on unseen connectivity patterns, since they rely heavily on hand-crafted meta-paths. (2) Another widely researched line is embedding-based methods, which pre-process a KG with knowledge graph embedding (KGE) (Wang, Mao et al., 2017) algorithms, such as TransE (Bordes et al., 2013) and TransR (Lin et al., 2015), and then leverage the learned entity embeddings to regularise the representations of items. The high-order connectivity denotes the path that reaches users from any entity node with the path length larger than 1 as the above movie example shows. However, these methods only consider direct relations between entities, rather than the high-order connectivity paths.
Recent developments of graph neural networks (Hamilton et al., 2017, Kipf and Welling, 2017, Ying et al., 2018) try to automatically capture high-order structure information in a graph, which has the potential of achieving the goal but has not been explored much for KG-based recommendation. Another key deficiency is that they model each interaction in KG as an independent instance and do not consider their relations, which makes them insufficient to discriminate the various user preference revealed by different relation type in KG. As shown in Fig. 1, movie “Inception” connects to movie “Batman Begins” with two kinds of relations, which can be caused by user’s different interest propensity (prefer a movie with the same “genre” or “director”). Moreover, these methods are proposed for a single recommendation task (He et al., 2017) with only KG data. And they fail to study knowledge enhanced recommendation using deep neural networks in an end-to-end way.
To this end, we investigate how to utilise neural collaborative based model for both user–item and auxiliary knowledge graph data, and enable the knowledge to enrich the CF model with sparsity user–item interaction. Then three key challenges arise in this investigation. (1) how to learn item representation from complex heterogeneous structured KG. Traditional KGE regularisation only considers direct relations between entities, rather than the multi-hop relation paths. (2) how to model user profile from weighted items. Existing user profile modelling methods can be limited by its assumption that all historical items of a user profile contribute equally in estimating the similarity between the user profile and a target item. Intuitively, a user interacts with multiple items in the past, but it may not be true that these interacted items reflect the user’s interest to the same degree. (3) how to capture the complex interactions between users and items, which is not exploited by the prior shallow interaction-based methods. The traditional inner product matching function assumes that the dimensions of users and items embedding are independent with each other, and contribute equally for the prediction of all data points (He et al., 2017). To some extent, this assumption is impractical, since the embedding dimensions could be interpreted as certain properties of items (He, Du et al., 2018, Zhang et al., 2014), which are not necessarily to be independent.
In this paper, we propose a novel Knowledge Graph enhanced Neural Collaborative Recommendation(K-NCR) approach. K-NCR tackles above three challenges in a recommendation scenario as shown in Fig. 3. The proposed K-NCR has three key modules: (1) For item modelling, we exploit the high-order structural proximity among entities in KG instead of the one-hop local neighbour structure. In particular, we capture the influence diffusion process among items with multi-hop based on Graph Convolutional Network (GCN), which is capable of stimulating the propagation of user preferences over the set of knowledge entities by iteratively and automatically extending potential interests along with links in the knowledge graph. Moreover, we employ an attention mechanism to discriminate the importance of the relation type to mine users’ potential preferences. (2) For user modelling, to alleviate the limitation of traditional mean-based aggregator in NCF, the varying importance weights are learned from the interacted items with an attention network. These weights can distinguish historical item in a user profile is playing a more important role for the user preference modelling. (3) To capture the complex interaction between user and item embeddings, our proposal of using outer product above the user and item embedding layer results in a two-dimensional interaction map. The interaction map is rather suitable for the CF task, since it not only subsumes the interaction signal used in MF (its diagonal elements correspond to the intermediate results of inner product), but also learns possible complex dimension correlations with CNN layers. Thus, the auxiliary knowledge information and complex nonlinear interaction are modelled in the proposed unified architecture. Given the proposed neural architecture, we design a joint learning framework that allows the KG embedding part and the neural collaborative recommendation part to enhance each other mutually.
The main contributions of this paper are as follows:
- 1.
We are the first to combine KG structure information with collaborative recommendation in an end-to-end neural style. We highlight the importance of explicitly modelling (1) the high-order connectivity of knowledge graph to provide a better recommendation with item side information and (2) the pairwise correlations between the dimensions of the embedding space in the matching function.
- 2.
We propose a new recommendation framework K-NCR, which encodes the high-order connectivities of KG and complex matching interaction in an explicit and end-to-end manner by performing embedding propagation and outer product-based convolutional NCF, respectively.
- 3.
We conduct empirical studies on three million-size real-world dataset. Extensive experiment results demonstrate the state-of-the-art performance of K-NCR and its effectiveness in improving the embedding and matching quality for the final prediction (see Table 1).
Section snippets
Preliminary
Before introducing the proposed approach, we first define the task of knowledge graph enhanced recommendation, and then shortly recapitulate the standard neural collaborative filtering, highlighting its limitations for dealing with the task.
Framework
The goal of KG-enhanced recommendation is to select relevant information from the knowledge graph to assist the target recommendation prediction. In this paper, we propose a knowledge graph enhanced Neural Collaborative Recommendation (K-NCR), an end-to-end framework that utilises KG to alleviate the sparsity problem of recommender systems. We now present the proposed K-NCR model, and the architecture of which is illustrated in Fig. 3. There are three components in the framework: (1) knowledge
Datasets
We use the following three datasets in our experiments for movie, book and music recommendation, respectively:
- •
MovieLens-20M2 dataset contains almost 20 million ratings (ranging from 1 to 5) on the MovieLens website, which is a widely used benchmark dataset in movie recommendations.
- •
Book-Crossing3 dataset contains 1 million explicit ratings (ranging from 0 to 10) of books in the Book-Crossing
Knowledge graph embedding
The Knowledge Aggregating-based Item Modelling module in K-NCR links to a large number of emerged work in KGE methods. KGE is used to embed entities or relations in a KG into continuous vector spaces, so that KG can be easier to process while still preserving the inherent structural information (Wang, Mao et al., 2017). In general, existing KGE methods can be classified into two categories: (1) Translation-based embedding methods try to model the relations between entities as translations from
Conclusion
In this paper, we develop an end-to-end neural architecture K-NCR that jointly incorporates the knowledge graph structure and user–item interaction in a unified neural network model for recommendation. Specifically, we propose a propagation based knowledge graph embedding model that exploits the high-order structural proximity among entities in KG to enhance the item embedding learning in a unified framework. Besides, an attention-based user modelling part is proposed to learn the varying
CRediT authorship contribution statement
Lei Sang: Conceptualization, Data curation, Software, Writing - original draft. Min Xu: Conceptualization, Methodology, Validation, Writing - review & editing. Shengsheng Qian: Methodology, Formal analysis, Supervision, Writing - review & editing. Xindong Wu: Resources, Project administration, Funding acquisition.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
This work has been supported by the National Key Research and Development Program of China under grant 2016YFB1000901, the China Scholarship Council (CSC), the National Natural Science Foundation of China under grant 91746209 and the Program for Changjiang Scholars and Innovative Research Team in University (PCSIRT) of the Ministry of Education of China under grant IRT17R32.
References (72)
- et al.
Personalized recommendation by matrix co-factorization with tags and time information
Expert Systems with Applications
(2019) - et al.
Topic-enhanced emotional conversation generation with attention mechanism
Knowledge-Based Systems
(2019) - et al.
Peer recommendation in dynamic attributed graphs
Expert Systems with Applications
(2019) - et al.
APL: Adversarial pairwise learning for recommender systems
Expert Systems with Applications
(2019) - et al.
Multi-modal matrix factorization with side information for recommending massive open online courses
Expert Systems with Applications
(2019) - et al.
Serendipitous recommendation in e-commerce using innovator-based collaborative filtering
IEEE Transactions on Cybernetics
(2018) - et al.
Deep hybrid collaborative filtering for web service recommendation
Expert Systems with Applications
(2018) - et al.
Tag-aware dynamic music recommendation
Expert Systems with Applications
(2018) - Bahdanau, D., Cho, K., & Bengio, Y. (2015). Neural machine translation by jointly learning to align and translate. In...
- Bai, T., Wen, J.-R., Zhang, J., & Zhao, W. X. (2017). A neural collaborative filtering model with interaction-based...
Translating embeddings for modeling multi-relational data
Learning shape correspondence with anisotropic convolutional neural networks
Unifying knowledge graph learning and recommendation: Towards a better understanding of user preferences
Convolutional neural networks on graphs with fast localized spectral filtering
Netflix update: Try this at home
Inductive representation learning on large graphs
NAIS: Neural attentive item similarity model for recommendation
IEEE Transactions on Knowledge and Data Engineering
Matrix factorization techniques for recommender systems
Computer
Deep learning
Nature
Analogical inference for multi-relational embeddings
Effective approaches to attention-based neural machine translation
Cited by (59)
Feature interactive graph neural network for KG-based recommendation[Formula presented]
2024, Expert Systems with ApplicationsDeep adaptive collaborative graph neural network for social recommendation
2023, Expert Systems with ApplicationsDeep pairwise learning for user preferences via dual graph attention model in location-based social networks
2023, Expert Systems with ApplicationsA fine-grained and multi-context-aware learning path recommendation model over knowledge graphs for online learning communities
2023, Information Processing and ManagementCollaborative recommendation model based on multi-modal multi-view attention network: Movie and literature cases
2023, Applied Soft Computing