Although, widely applied deep learning models show promising performance in recommender systems, little effort has been devoted to exploring ranking learning in recommender systems. It is important to generate a high quality ranking list for recommender systems, whose ultimate goal is to recommend a ranked list of items for users. Also, the latent features learned from Matrix Factorization (MF) based methods do not take into consideration any deep interactions between the latent features; therefore, they are insufficient to capture user–item latent structures. To address these problems, we propose a novel model, DeepRank, which uses neural networks to improve personalized ranking quality for Collaborative Filtering (CF). This is a general architecture that can not only be easily extended to further research and applications, but also be simplified for pair-wise learning to rank. Finally, we perform extensive experiments on three data sets. Results demonstrate that our proposed models significantly outperform the state-of-the-art approaches. Our projects are available at: https://github.com/XiuzeZhou/deeprank.
Introduction
Recommender systems have been successfully applied to many fields, such as e-commerce, music, and news. These systems model users’ preferences and predict the best-suited services or products for users to help them discover useful information from a plethora of options [1], [2], [3], [4]. In practice, recommender systems aim at recommending items that users may be interested in a ranking list. Therefore, the effects from ranking-oriented methods are more suitable than the accuracy performance from rating prediction methods for recommender systems [5], [6].
To achieve a better quality of recommender systems and improve their ranking performance, various approaches have been proposed. For example, Rendle et al. [7] proposed a Bayesian ranking framework, which compares ordered pairs of items to decide which is preferred over another in the recommendation list. Park et al. [8] adopted some external information, such as user profiles and item contents, to solve the cold-start problem for pair-wise preference regression. Shi et al. [9] proposed a method, ListRank-MF, which combined a list-wise ranking method with Matrix Factorization (MF) for Collaborative Filtering (CF). However, one major limitation of those traditional solutions is that they are unable to fully capture complex structures and deeper information from user–item interactions.
Recently, all kinds of deep learning models have achieved remarkable success in various fields, such as Computer Vision (CV), speech recognition, and Natural Language Processing (NLP). These deep learning models are also widely applied to recommender systems to improve the quality of recommendation. For example, Kim et al. [10] applied Convolutional Neural Networks (CNN) to reviews to obtain contextual information and then combined it with MF for recommendation. Wang et al. [11] presented a collaborative deep learning method, which uses a Stacked Denoising Auto-encoder (SDA) to obtain information from reviews to alleviate the data sparse problem of CF. McAuley et al. [12] used the latent features learned from images by neural networks for the style and appearance of products to catch visual relationships between the products. However, these methods have some shortcomings. First, they cannot be applied to fields with little or no additional information, such as textual and image information [13]. Second, capturing and processing that auxiliary data requires extensive time and effort.
Although many methods based on MF have achieved good performance on recommendation, they cannot effectively learn user and item representations, which leads them to have poor ability to capture complicated and deeper information about the interaction between users and items. To solve this problem, and inspired by the great success of deep learning methods applied to ranking learning, we propose DeepRank, a list-wise ranking method with neural networks. Point-wise methods, rather than focusing on the personalized ranking of a set of items, focus only on predicting an accurate rating value of an item. In practice, users tend to pay more attention to the ranking order of an item than to its accurate ratings. For example, when wanting to see a movie online, a user often cares less about its rating and chooses the movie at the top of the recommended list. Rather than predicting a rating, the goal of DeepRank is to use predicted scores to rank the position of an item.
Then, we reduce our model from top-n list-wise to a simpler structure: top-one list-wise, and then to the simplest structure: a pair-wise learning method, which is one of the most popular ranking-oriented methods in recommender systems. Pair-wise methods, in which each user is represented as a set of pair-wise preferences over items, help users understand their preferences more than do point-wise methods [14], [15].
Finally, the users’ preference features and characteristic features of the items are not directly related. Therefore, to further improve the generation performance of our model, we set the latent features of users and items at different sized dimensions. In most recommendation approaches, the interaction between user latent features and item latent features is used to predict the rating or ranking score [16], [17]. But, the difference in the number of latent features between them is rarely taken into account. To the best of our knowledge, this is the first work that aims at setting the number of latent features for users and items at different values.
We demonstrate that our proposed DeepRank has several attractive advantages:
(1)
When using DeepRank to make predictions, it achieves better ranking performance. To the best of our knowledge, this is the first list-wise work based on neural network to rank learning;
(2)
It has a simple and flexible structure, which can be simplified from top-n list-wise to top-one list-wise and pair-wise ranking learning for efficiency;
(3)
This is the first time the effect of setting the number of latent features for users and items at different values has been investigated.
The rest of the paper is organized as follows: Section 2 briefly reviews the background and some related work. Section 3presents our proposed models in detail. Section 4 describes experimental results for several data sets to show the performance of our models. Section 5 gives the conclusion and provides future directions.
Section snippets
Related work
In this section, we briefly introduce some background information and related works. First, we introduce some ranking-oriented approaches and deep learning models for recommendations. Then, we introduce ListRank-MF, which inspired us to propose our method.
Methods
In this section, we introduce our proposed model: DeepRank, which is a top-n list-wise model for implicit feedback. First, we discuss the problem definition and the notations used throughout the paper. Then, we introduce our model in detail. Finally, we show that our model can be simplified to a top-one list-wise, and further to a pair-wise method for ranking.
Experiments
First, we introduce the data sets used in our experiments. Then, we present the baselines we compared with our model and the metrics we adopted for evaluation. Finally, we conduct the experiments in detail and then answer the following research questions:
-RQ1: How does DeepRank perform compared with other methods?
-RQ2: How do the different dimension sizes between user and item embedding affect the performance of the model?
-RQ3: How does the depth of the model affect DeepRank?
Conclusion
We proposed a novel method, DeepRank, for ranking recommender systems. DeepRank, a promising tool for recommendation, provides new insight into CF models for ranking learning. Compared with existing ranking-oriented methods, our method achieves better performance and presents higher quality recommendations. In addition, our model has several outstanding advantages: (1) It captures user and item latent features in a complicated and nonlinear architecture; (2) Because it has a simple and flexible
CRediT authorship contribution statement
Ming Chen: Supervision, Writing - review & editing. Xiuze Zhou: Conceptualization, Methodology, Writing - original draft, Investigation, Software, Validation.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgment
The authors thank Michael McAllister for proofreading this paper.
For Lithium-ion (Li-ion) batteries, problems such as material aging and capacity decay lead to battery performance degradation or even catastrophic events. Predicting Remaining Useful Life (RUL) is an effective way to indicate the health of Li-ion batteries, which helps to improve the reliability and safety of battery-powered systems. We propose a novel neural network, AttMoE, which combines an attention mechanism with Mixture of Experts (MoE), to capture the capacity fade trend for battery RUL prediction. When facing the problem that raw data collected from sensors are always full of noise, AttMoE uses a dropout mask to denoise the raw data. For RUL prediction, one key idea is that the attention mechanism captures the long-term dependencies between elements in a sequence and more attention is paid to the important features that contain more degradation information; another key idea is that MoE uses many experts to increase model capacity to achieve better representations. Finally, we conducted experiments using two public data sets to show that AttMoE is effective in RUL prediction and achieves up to 10%–20% improvement in terms of Relative Error (RE). Our projects are all open source and are available at https://github.com/XiuzeZhou/RUL.
Course recommendation technology plays a key role in online learning services. However, there are still two key issues that remain unresolved in practice. First, it is difficult to characterize learners’ multiple preferences in various courses, while existing course recommendation methods are incompetent to address it. Second, there is a critical challenge to learn effective recommendation strategies due to the few labeled data, since online course resources usually include insufficient information about learners’ behaviors (e.g., lack of ratings and comments). To deal with these issues, we propose a joint learning framework, namely multi-scale Reinforced Profile for Personalized Recommendation (RPPR). Specifically, we design a multi-scale deep reinforcement learning method to construct learners’ multiple profiles according to coarse-grained and fine-grained semantics. Moreover, to adaptively adjust the recommendation strategies from limited interactive information, we develop an attention-based recommendation model with multivariable perceptron, which leverages related parameters of both neural networks and data structure to control the direction of loss optimization. Empirical results demonstrate that our RPPR significantly outperforms some competitive baselines on two real-world MOOCs (Massive Open Online Courses) datasets.
This paper introduces a novel method aimed at enhancing online-to-offline (O2O) services recommendations by utilizing two-layer knowledge networks. The primary objective of this method is to assist consumers in efficiently navigating the myriad of options available when choosing O2O services. Using co-occurrence relationships, we construct a two-layer knowledge network system, comprising a service knowledge network based on service usage information as the first layer and a consumer knowledge network, built on “co-used” behaviors as the second layer. The former is established upon service use data, while the latter is founded on “co-used” behaviors among consumers. The features and information of these two knowledge networks can complement each other to produce precise and effective recommendations. Empirical findings gained from our experiments demonstrate that: (1) the proposed recommendation method outperforms widely-used and state-of-the-art recommendation methods; (2) both the service knowledge network and consumer knowledge network play an equally significant role in O2O service recommendations; (3) the location of O2O services is an essential factor in consumers’ choices for services. Notably, this research also identifies the optimal parameter settings for the proposed recommendation method.
Recommender systems (RSs) have been employed for many real-world applications including search engines, social networks, and information retrieval systems as powerful intelligent techniques for creating personalized recommendations. RSs mainly utilize different data resources such as user-item ratings, tag data, and social relationships in their recommendation process to overcome the data sparsity issue. One of the main challenges is how to integrate the results obtained by these data resources. Moreover, most of the existing RSs focus on the recommendation accuracy, and do not consider the reliability of the recommendations. To address these challenges, in this study, a reliable recommendation method is developed, which employs deep neural networks and reinforcement learning. The proposed method firstly uses a stacked denoising autoencoder model to obtain the latent features from the rating, tag, and trust data. The similarity value between each pair of users is then calculated in three different aspects according to the extracted latent features. An ensemble approach is developed by integrating the three similarities using reinforcement learning to determine the final similarity value. The ensemble similarity values are used in the rating prediction process, and the reliability of the predicted ratings is evaluated using an effective measure. Finally, a recommendation strategy is developed based on the integration of the predicted ratings and their reliability values. We evaluated the effectiveness of the proposed method by conducting numerous experiments on three datasets where the results show that the proposed method is far superior to other recommendation models in making more accurate recommendations.
Differing from most existing Actor–Critic algorithms, we improve the Actor–Critic algorithm in two aspects: (i) sequentially add through long-term incremental evaluation; (ii) parameterize by both reinforced loss constraint and RL loss in a self-supervised manner. Recommender systems aim to help users to find what they want from a huge number of items [44–47]. KG is an effective tool for alleviating the problems of cold-start and data sparsity.
To improve the recommendation accuracy and offer explanations for recommendations, Reinforcement Learning (RL) has been applied to path reasoning over knowledge graphs. However, in recommendation tasks, most existing RL methods learn the path-finding policy using only a short-term or single reward, leading to a local optimum and losing some potential paths. To address these issues, we propose a Self-Supervised Reinforcement Learning (SSRL) framework combined with dual-reward for knowledge-aware recommendation reasoning over knowledge graphs. Then, we improve Actor–Critic algorithm by using a dual-reward driven strategy, which combines short-term reward with long-term incremental evaluation. The improved algorithm helps the policy guide path reasoning in an overall situation. In addition, to find the most potential paths, in the improved Actor–Critic algorithm, a loss constraint of each sample is used as a reinforced signal to update the gradients. With some improvements against baselines, experimental results demonstrate the effectiveness of our framework.
Course recommendation technology is an effective way to address this challenge [2]. Differing from other recommendation scenarios [3–6], course recommender systems mainly aim to recommend the target courses that users prefer and should learn according to the most relevant courses that they have enrolled in. In other words, the major goal is to improve the recommendation accuracy based on the contributing historical courses.
Online course recommendation is an extremely relevant ingredient for the efficiency of e-learning. The current recommendation methods cannot guarantee the effectiveness and accuracy of course recommendation, especially when a user has enrolled in many different courses. Because these methods fail to distinguish the most relevant historical courses, which can contribute to predicting the target course that indeed reflects the user’s interests from her sequential learning behaviors. In this paper, we propose a context-aware reinforcement learning method, named Hierarchical and Recurrent Reinforcement Learning (HRRL), to efficiently reconstruct user profiles for course recommendation. The key ingredient of our scheme is the novel interaction between an attention-based recommendation model and a profile reviser with Recurrent Reinforcement Learning (RRL) that exploits temporal context. To this aim, a contextual policy gradient with approximation is proposed for RRL. By employing RRL in hierarchical tasks of revising user profiles, the proposed HRRL model enables reliable convergence in revising policy learning and improves the recommendation accuracy. We demonstrate the effectiveness of our proposed method by experiments on two open online courses datasets. Empirical results show that HRRL significantly outperforms state-of-the-art baselines.
