Collaborative filtering with a deep adversarial and attention network for cross-domain recommendation
Introduction
In recent years, the explosive growth of data has become increasingly fierce, which makes it difficult for individuals to find information that suits their demand in time [24]. This phenomenon is collectively referred to as information overload [27] and has attracted considerable social attention. To address this problem, recommender systems have emerged in the last two decades and play a vital and indispensable role in real life, especially on e-commerce websites. For example, on the Amazon website1, the recommender system provides a large pool of possible options and automatically helps users determine products or services based on their individual preferences.
In general, collaborative filtering (CF) is a widely used research thread in various recommender systems given the capability of high efficiency and high accuracy [1], [4]. CF-based approaches recommend a personalized list of items to a user by mining the history of other similar users exclusively based on user-item interactions [19], [38], [36], [16]. However, in real life, a user could merely rate a few items, although the number of items is very large. Using these extremely sparse data matrices for recommendations will make CF-based models predict low-quality results because the CF-based methods are only suitable for single domain recommendation. Some works introduce auxiliary information, such as user reviews and item contents, into CF-based algorithms to solve the data sparsity problem [29], [43]. However, it is generally more challenging to process auxiliary information than past activities. Thus, the model complexity will be greatly increased, and the runtime efficiency will be lower, which is important for a practical recommender system.
One promising solution to alleviate this long-standing sparsity problem is cross-domain recommendation (CDR) [3], [32], [35]. This kind of approach combines the knowledge extracted from several different but related domains to assist recommendations in the target domain. Existing CDR methods assume that some explicit or implicit relationships exist between the source domain and target domain. Based on this fundamental assumption, the most common method to identify different cross-domain recommendation scenarios is to observe whether users or items are shared between two domains [21]. Therefore, recommendation scenarios in CDR can be classified into four categories with respect to user-item overlap: 1. No User-No Item overlap (NU-NI) [25], [26]; 2. User-No Item overlap (U-NI) [18], [36]; 3. No User-Item overlap (NU-I) [30], [47]; 4. User-Item overlap (U-I) [22]. In this paper, we deal with cross-domain recommendations, particularly under the scenario where users or items are completely aligned. For example, as illustrated in Fig. 1, Netflix2 and MovieLens3 are essentially different movie websites. However, there are common movies according to IMDB4 ids. The recommendation performance of MovieLens can be enhanced by integrating the knowledge from Netflix.
Most existing CDR approaches extract the domain-shared features among multiple related domains to make a better recommendation. Some approaches are based on matrix factorization (MF) and its variations [37], [34], and others take advantage of clustering algorithms [25], [26]. However, most existing MF-based and clustering methods can only capture the linear correlation across domains due to the shallow models. Moreover, a risk of negative transfer exists when the latent factors of the target domain are directly replaced or fitted by the corresponding latent factors of the source domain. Fortunately, deep learning (DL) methods are capable of performing knowledge transfer in various recommender tasks [18], [30], [47], [22]. In this paper, we also introduce DL to study the implicit relationships between multiple domains.
Some recent work combines deep learning and domain adaptation to generalize a high-performance learner from the source domain to the target domain [2], [10]. In general, most of these approaches first construct a discriminative classifier or an accurate predictor for the source domain and then transfer it to the target domain. Instead of fine-tuning deep models pretrained on a large-scale domain in traditional domain adaptation methods, another line of works couples adaptative modules with deep architectures to reduce domain discrepancies across domains. In particular, adversarial domain adaptation [10] is an increasingly popular technology for distribution matching that combines adversarial learning with domain adaptation to boost transfer performance within a minimax game. Inspired by this observation, we assume that domain adaptation could also be employed in the area of cross-domain recommendation.
Although there are some essential differences between typical domain adaptation and cross-domain recommendation, a common idea of these two techniques is to model the domain-shared features to bridge two related but different domains to achieve higher quality performance. We note that user preferences are susceptible to the domain. In the U-NI recommendation scenario, users express distinct preferences for source and target items so that the item distribution of each user is different. For example, we have some data from the sports domain and book domain. If a user prefers taking exercise instead of reading books in his spare time, it is highly possible that he will prefer to click the items from the sports domain instead of the book domain so that his item distribution of these two domains is different. Similarly, the user distribution of each item is also different in the NU-I recommendation scenario. According to the above phenomenon, we treat cross-domain recommendation as an instance of domain adaptation, which has poorly been studied in the past decade. Although some recent works, such as DARec [46], have a similar idea and show superior performance in contrast to traditional cross-domain recommendations, most of them cannot completely take domain discrepancies into consideration. There is no guarantee that the knowledge learned from this model is beneficial to the recommendation of the target domain. To address this limitation, we propose a novel adaptation approach for cross-domain recommendation to extract both domain-shared and domain-specific features simultaneously.
In terms of cross-domain recommendations based on matrix factorization and clustering algorithms, the combination of domain-shared and domain-specific knowledge has been proposed to improve recommendation accuracy. However, the combination strategies applied in previous studies [37], [9] fail to flexibly transfer the appropriate proportion of common knowledge, supposing that both domain-shared and domain-specific knowledge characterize the user preference to the same degree. In real-world applications, especially under U-NI settings, we argue that two points need to be considered: 1) the interests revealed from interactive information are generally diverse for different users; 2) the degrees of preference for different items typically vary for the same user across domains. Similarly, in the NU-I scenarios, the same item sharing between two domains makes different contributions to the domain-shared knowledge. To better trade off domain-shared and domain-specific knowledge, we propose an adaptive attention mechanism that can allocate parameters automatically.
In this paper, we devise a generic framework called DAAN for cross-domain recommendation, which tightly couples matrix factorization-based collaborative filtering with deep adversarial domain adaptation via an attention network. Furthermore, we empirically conduct extensive experiments on real-world datasets to verify the effectiveness of our model in terms of several ranking metrics, and the results indicate that DAAN outperforms the state-of-the-art methods on two cross-domain recommendation scenarios.
To the best of our knowledge, our proposed approach is the first deep model that considers both domain-shared and domain-specific knowledge across domains within a domain-adversarial training paradigm for cross-domain recommendation. The major contributions of our work can be summarized as follows:
(1) We argue that domain discrepancy between the source and target domains is a problem that needs to be solved for cross-domain recommendation. To alleviate this problem, we develop a novel framework DAAN that learns domain-specific representations based on matrix factorization and captures domain-shared representations via deep adversarial domain adaptation.
(2) We design an efficient attention network that can automatically balance the degree of importance between domain-shared and domain-specific knowledge. Our proposed method is capable of keeping cross-domain recommendations free from the effect of negative transfer in the training process.
(3) We empirically conduct extensive experiments on several real-world datasets, and the results demonstrate that DAAN is superior to several state-of-the-art baselines. Moreover, some ablation studies verify the effectiveness of three components designed for cross-domain recommendations and the utility of our model in alleviating data sparsity and domain discrepancy problems.
The remainder of this paper is organized as follows. In Section 2, we review some related works. In Section 3, we introduce the notations and preliminaries about our work. In Section 4, we first describe our main idea at a high level and then elaborate three core components of the proposed framework in detail. In Section 5, we demonstrate the superior performance of our model compared with some state-of-the-art baselines by experiments conducted on real-world datasets. Finally, in Section 6, we conclude this paper and present prospects for future work.
Section snippets
Related work
In this section, we present the related work from three different perspectives. The first category is our review about collaborative filtering. The second category covers advanced studies about cross-domain recommendation. The third category briefly describes the work on adversarial learning.
Preliminaries
In this section, we first provide a formal formulation of the problem in Section 3.1. Then, we review two basic models introduced in our framework. In Section 3.2, the matrix factorization model for collaborative filtering is described. In Section 3.3, we briefly recapitulate the generative adversarial net.
Our framework
In this section, we design a generic framework DAAN for cross-domain recommendation. For example, we extract shared users between movies and TV and office products and intend to integrate two interaction matrices for the item recommendation task. From Fig. 2, we find that the user-item distributions of two domains are similar, but the preference distributions of the same user for items in two domains are different. Our DAAN aims to capture both domain-specific and domain-shared features to
Experiments
In this section, we systematically evaluate the recommendation performance of our model in two different scenarios: U-NI and NU-I. We first introduce the experimental settings (Section 5.1). Then, we compare the model with several state-of-the-art methods (Section 5.2) and analyze parameter sensitivity (Section 5.3). Furthermore, we conduct several ablation studies (Section 5.4).
Conclusion and future work
In this paper, we developed a novel DAAN model that considers both domain-shared and domain-specific knowledge across domains for cross-domain recommendation. Extensive experiments have shown that recommendation performance is vulnerable to data sparsity and domain discrepancy problems. Towards the goal of alleviating the data sparsity issue, we reconstructed the sparse target user-item interaction matrix by utilizing knowledge guidance from a relatively dense source user-item interaction
CRediT authorship contribution statement
Huiting Liu: Conceptualization, Methodology, Writing - review & editing. Lingling Guo: Software, Validation, Formal analysis, Data curation, Writing - original draft, Visualization. Peipei Li: Investigation. Peng Zhao: Supervision, Resources. Xindong Wu: Supervision.
Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This research has been supported by the National Key Research and Development Program of China No. 2016YFB1000901, the National Natural Science Foundation of China Nos. 61202227 and 61602004, Natural Science Foundation of Anhui Province, No. 2008085MF219 and Provincial Natural Science Foundation of Anhui Higher Education Institution of China, No. KJ2018A0013.
References (50)
- et al.
LSCD: Low-rank and sparse cross-domain recommendation
Neurocomputing
(2019) A survey of transfer learning for collaborative recommendation with auxiliary data
Neurocomputing
(2016)- et al.
A cross-domain recommender system with consistent information transfer
Dec. Support Syst.
(2017) - et al.
Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions
IEEE Trans. Knowl. Data Eng.
(2005) - Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, Dumitru Erhan, Domain Separation Networks....
- Paolo Cremonesi, Antonio Tripodi, Roberto Turrin, Cross-Domain Recommender Systems. ICDM Workshops 2011:...
- et al.
A survey of collaborative filtering-based recommender systems: from traditional methods to hybrid methods based on social networks
IEEE Access
(2018) - Xu Chen, Yongfeng Zhang, Hongteng Xu, Zheng Qin, Hongyuan Zha, Adversarial Distillation for Efficient Recommendation...
- et al.
Item-based top-N recommendation algorithms
ACM Trans. Inf. Syst.
(2004) - Wenqi Fan, Tyler Derr, Yao Ma, Jianping Wang, Jiliang Tang, Qing Li. Deep Adversarial Social Recommendation. IJCAI...