Representation learning with collaborative autoencoder for personalized recommendation
Introduction
In the age of information explosion, recommendation systems have played an increasing important role in tackling information overload problem, which have been widely applied in many online services like electronic commerce (Hu et al., 2017) and social networks (Botangen et al., 2020). The main intuition behind the personalized recommendation is to profile the users’ preferences on items by utilizing the user–item interaction information, which is known as Collaborative Filtering (CF) (Yao et al., 2016). In recent decades, Collaborative Filtering methods have become one of the most pervasive tools in recommendation and attained much attention from multiple disciplines. The advances in CF have far-ranging consequences in practical applications for personalized recommendation.
In the traditional collaborative filtering approaches, the matrix factorization methods have achieved highly competitive performance and are widely used due to their effectiveness and efficiency (Luo et al., 2020). These methods decompose the rating matrix into user-specific matrix and item-specific matrix, and the decomposed matrix can be exploited to leverage the information in recommendation system for further prediction (Koren, 2008). However, traditional matrix factorization techniques have inherent limitations in representation learning and may contain non-helpful feature attributes from the input space prior to training, which can be detrimental to the performance of personalized recommendation (Zhuang et al., 2017b). In recent years, deep learning methods are proposed to project the raw data into higher-level feature space (Yi et al., 2018), and there have been some efforts to apply deep-based methods for recommendation systems already (Ali et al., 2020, Chae et al., 2019, Gao et al., 2020). He et al. proposed a general framework called neural network-based Collaborative Filtering (He et al., 2017), which focused on implicit feedback and utilized a multi-layer perceptron to learn the user–item interaction function. Wang et al. proposed a collaborative deep learning model which simultaneously learned the feature representations from content and extracted the relationship between item and user from the rating matrix (Wang et al., 2015b). Among all the deep recommendation methods, autoencoder models have been applied widely for the advantages of not requiring labeling and effectiveness. Zhuang et al. proposed a collaborative ranking framework via representation learning with pair-wise constraints (Zhuang et al., 2017a), which introduced a two encoding layers autoencoder for representation learning only on the rating matrix without auxiliary information.
Though better feature representations for personalized recommendation can be obtained by the afore-mentioned deep methods, there are two main problems that prevent the further development of these methods. The first is the structure of deep model. Most of these methods relied on the identical autoencoder structure, which pay far less attention to capture different features of the user-based and item-based data. The second problem is the data sparsity of rating matrix, which may incur the significant performance degradation of recommendation. Most previous methods introduced additional information from both users and items to address the problem of data sparsity. However, due to the increasing awareness of privacy protection, it is very difficult to obtain additional information about user attributes.
To address these problems, we propose a novel representation learning method for personalized recommendation based on autoencoder model. The proposed method, called Collaborative Autoencoder for Personalized Recommendation (CAPR for short), is shown in Fig. 1. To address the first problem with different features of the user-based and item-based data, a representation learning model with two different autoencoders are proposed to capture different features of the data with two types of autoencoders. To the best of our knowledge, this is the first time that two different autoencoder models have been used to learn the characteristics of users and items respectively for recommendation. More specifically, firstly, the model of AutoEncoder with Graph Regularization (AEGR for short) is applied to extract user-based features, which aims to construct a graph connecting similar observations for high-level representations learning of users. Secondly, a semi-autoencoder model is introduced to import the side information for item-based feature representations learning. Finally, the user-based and item-based features are used to restructure the approximated representation matrix. To address the second problem of data sparsity, a semi-autoencoder model is employed to integrate item attributes for reconstructing the rating matrix. The semi-autoencoder model breaks the limitation that the input and output layer are required to be consistent in the traditional autoencoder model, which can flexibly introduce additional information of items to solve the data sparsity problem. Experimental results on several datasets demonstrate the effectiveness of our proposed method compared to other state-of-the-art matrix factorization methods and deep-based methods.
In summary, the main contributions of our work can be distilled to the following.
- •
Different from the existing works, which focus on learning the user-based and item-based feature representations with identical models, we provide a novel viewpoint for personalized recommendation by taking two different types of feature representations models into consideration. In this way, the proposed method can extract richer feature representations with different characteristics of the user-based and item-based data and further improve the recommendation performance.
- •
We design a novel autoencoder model to learn robust feature representations for users, which is referred to as autoencoder with graph regularization (AEGR). It is helpful for constructing connections in higher-level feature space.
- •
The extensive experiments on several datasets are conducted to confirm the effectiveness of the proposed method compared to other state-of-the-art matrix factorization methods and deep-based methods.
Section snippets
Related work
Personalized recommendation systems estimate users’ preferences for items and recommend items to users that they might like. Based on how the recommendation is performed, existing methods for recommendation systems can roughly be categorized into three categories: content-based, collaborative filtering, and hybrid methods. In addition, based on how the features of collaborative filtering are learned, matrix factorization and deep neural network have provoked a vast amount of attention and
Preliminaries and methods
Before introducing the details of our proposed CAPR, the preliminaries that applied in the method will be reviewed as follows.
Collaborative autoencoder for personalized recommendation
In this section, the details of our proposed CAPR are given, and the whole framework is illustrated as Fig. 1.
Experiments
In this section, we conduct extensive experiments on three datasets to evaluate the effectiveness of our proposed method for personalized recommendation. In the following, firstly, three datasets used in the experiments are introduced in detail. Secondly, we introduced the compared methods and evaluation metrics of our experiments. Then we presented the results of comparative experiments and their observations. Finally, the main parameters of our proposed CAPR are analyzed on certain dataset.
Conclusion
In this paper, we propose a representation learning method with Collaborative Autoencoder for Personalized Recommendation, called CAPR, to overcome the shortcomings of the state-of-the-art methods and achieve more desirable performance. CAPR has the following three advantages for gaining performance improvement: (1) it extracts different characteristics of the user-based and item-based data; (2) a novel representation learning method with two different autoencoders are designed to learn the
CRediT authorship contribution statement
Yi Zhu: Conceptualization, Methodology, Software, Writing - original draft. Xindong Wu: Supervision, Formal analysis, Project administration. Jipeng Qiang: Visualization, Writing - review & editing. Yunhao Yuan: Data curation, Visualization. Yun Li: Software, Writing - review & editing, Validation.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This research is supported by the National Natural Science Foundation of China under grant 61906060, National Key Research and Development Program of China under grant 2016YFC0801406 and the Program for Changjiang Scholas and Innovative Research Team in University (PCSIRT) of the Ministry of Education of China under grant IRT17R32.
References (39)
- et al.
Deep learning in citation recommendation models survey
Expert Systems with Applications
(2020) - et al.
Geographic-aware collaborative filtering for web service recommendation
Expert Systems with Applications
(2020) - et al.
Autoencoder-based personalized ranking framework unifying explicit and implicit feedback for accurate top-N recommendation
Knowledge-Based Systems
(2019) - et al.
Hybrid collaborative recommendation of co-embedded item attributes and graph features
Neurocomputing
(2021) - et al.
A fast deep AutoEncoder for high-dimensional and sparse matrices in recommender systems
Neurocomputing
(2020) - et al.
Representation learning via dual-autoencoder for recommendation
Neural Networks
(2017) Learning deep architectures for AI
Foundations and Trends® in Machine Learning
(2009)- Chen, L., Wu, L., Hong, R., Zhang, K., & Wang, M. (2020). Revisiting graph based collaborative filtering: A linear...
- et al.
Hybrid collaborative recommendation via dual-autoencoder
IEEE Access
(2020) - et al.
DMRAE: discriminative manifold regularized auto-encoder for sparse and robust feature learning
Progress in Artificial Intelligence
(2020)
Hybrid microblog recommendation with heterogeneous features using deep neural network
Expert Systems with Applications
DeepFM: a factorization-machine based neural network for CTR prediction
Locality preserving projections
Advances in Neural Information Processing Systems
Neural collaborative filtering
Factorization meets the neighborhood: a multifaceted collaborative filtering model
PREA: Personalized recommendation algorithms toolkit
Journal of Machine Learning Research
Cited by (22)
HyNCF: A hybrid normalization strategy via feature statistics for collaborative filtering: A hybrid normalization strategy for collaborative filtering
2024, Expert Systems with ApplicationsMeta-path based graph contrastive learning for micro-video recommendation
2023, Expert Systems with ApplicationsGraph convolutional network combining node similarity association and layer attention for personalized recommendation
2023, Engineering Applications of Artificial IntelligenceDAEM: Deep attributed embedding based multi-task learning for predicting adverse drug–drug interaction
2023, Expert Systems with ApplicationsGraph-embedding-inspired article recommendation model
2023, Expert Systems with ApplicationsCitation Excerpt :Therefore, we adopt PMF to update the “User-Article” correlation matrix and the article feature matrix to boost the final commendation results. AutoEncoder (AE) is one of the core models of deep learning (Bellini et al., 2019), which consists of Encoder and Decoder (Zhu et al., 2021). The Encoder transforms the original feature, X, into low-dimensional latent embeddings through multiple hidden layers, Z: Z = f(X).
Social recommendation via deep neural network-based multi-task learning
2022, Expert Systems with ApplicationsCitation Excerpt :Other works are also trying to make use of deep learning techniques to model the nonlinear interaction between users and items. CAPR (Zhu et al., 2021) learned user-based and item-based feature representations by two different autoencoders for capturing different features of the data, while (Ying et al., 2018) generated embeddings of items by GCN to incorporate both user–item graph structure as well as node feature information. As to social recommendations, deep learning has also been developed in this field.