Representation learning with collaborative autoencoder for personalized recommendation

https://doi.org/10.1016/j.eswa.2021.115825Get rights and content

Highlights

  • Two different autoencoders are used to capture characteristics for users and items.

  • Manifold regularization is integrated into autoencoder for user’s features learning.

  • The comprehensive experiments evaluate the effectiveness of our proposed method.

Abstract

In the past decades, recommendation systems have provided lots of valuable personalized suggestions for the users to address the problem of information over-loaded. Collaborative Filtering (CF) is one of the most commonly applied and successful recommendation approaches, which refers to using the preferences of groups with similar interests to recommend information to other users. Recently, in addition to the traditional matrix factorization techniques, deep learning methods have been proposed to learn more abstract and higher-level representations for recommendation. However, most previous deep recommendation methods learn the higher-level feature representations of users and items through an identical model structure, which ignores the different characteristics of the user-based and item-based data. In addition, the rating matrix is usually sparse which may result in a significant degradation of recommendation performance. To address these problems, we propose a representation learning method with Collaborative Autoencoder for Personalized Recommendation (CAPR for short). In this method, user-based and item-based feature representations are learned by two different autoencoders for capturing different features of the data. Meanwhile, items’ attributions are combined into the feature representations with semi-autoencoder for alleviating the sparsity problem. Extensive experimental results confirm the effectiveness of our proposed method compared to other state-of-the-art matrix factorization methods and deep recommendation methods.

Introduction

In the age of information explosion, recommendation systems have played an increasing important role in tackling information overload problem, which have been widely applied in many online services like electronic commerce (Hu et al., 2017) and social networks (Botangen et al., 2020). The main intuition behind the personalized recommendation is to profile the users’ preferences on items by utilizing the user–item interaction information, which is known as Collaborative Filtering (CF) (Yao et al., 2016). In recent decades, Collaborative Filtering methods have become one of the most pervasive tools in recommendation and attained much attention from multiple disciplines. The advances in CF have far-ranging consequences in practical applications for personalized recommendation.

In the traditional collaborative filtering approaches, the matrix factorization methods have achieved highly competitive performance and are widely used due to their effectiveness and efficiency (Luo et al., 2020). These methods decompose the rating matrix into user-specific matrix and item-specific matrix, and the decomposed matrix can be exploited to leverage the information in recommendation system for further prediction (Koren, 2008). However, traditional matrix factorization techniques have inherent limitations in representation learning and may contain non-helpful feature attributes from the input space prior to training, which can be detrimental to the performance of personalized recommendation (Zhuang et al., 2017b). In recent years, deep learning methods are proposed to project the raw data into higher-level feature space (Yi et al., 2018), and there have been some efforts to apply deep-based methods for recommendation systems already (Ali et al., 2020, Chae et al., 2019, Gao et al., 2020). He et al. proposed a general framework called neural network-based Collaborative Filtering (He et al., 2017), which focused on implicit feedback and utilized a multi-layer perceptron to learn the user–item interaction function. Wang et al. proposed a collaborative deep learning model which simultaneously learned the feature representations from content and extracted the relationship between item and user from the rating matrix (Wang et al., 2015b). Among all the deep recommendation methods, autoencoder models have been applied widely for the advantages of not requiring labeling and effectiveness. Zhuang et al. proposed a collaborative ranking framework via representation learning with pair-wise constraints (Zhuang et al., 2017a), which introduced a two encoding layers autoencoder for representation learning only on the rating matrix without auxiliary information.

Though better feature representations for personalized recommendation can be obtained by the afore-mentioned deep methods, there are two main problems that prevent the further development of these methods. The first is the structure of deep model. Most of these methods relied on the identical autoencoder structure, which pay far less attention to capture different features of the user-based and item-based data. The second problem is the data sparsity of rating matrix, which may incur the significant performance degradation of recommendation. Most previous methods introduced additional information from both users and items to address the problem of data sparsity. However, due to the increasing awareness of privacy protection, it is very difficult to obtain additional information about user attributes.

To address these problems, we propose a novel representation learning method for personalized recommendation based on autoencoder model. The proposed method, called Collaborative Autoencoder for Personalized Recommendation (CAPR for short), is shown in Fig. 1. To address the first problem with different features of the user-based and item-based data, a representation learning model with two different autoencoders are proposed to capture different features of the data with two types of autoencoders. To the best of our knowledge, this is the first time that two different autoencoder models have been used to learn the characteristics of users and items respectively for recommendation. More specifically, firstly, the model of AutoEncoder with Graph Regularization (AEGR for short) is applied to extract user-based features, which aims to construct a graph connecting similar observations for high-level representations learning of users. Secondly, a semi-autoencoder model is introduced to import the side information for item-based feature representations learning. Finally, the user-based and item-based features are used to restructure the approximated representation matrix. To address the second problem of data sparsity, a semi-autoencoder model is employed to integrate item attributes for reconstructing the rating matrix. The semi-autoencoder model breaks the limitation that the input and output layer are required to be consistent in the traditional autoencoder model, which can flexibly introduce additional information of items to solve the data sparsity problem. Experimental results on several datasets demonstrate the effectiveness of our proposed method compared to other state-of-the-art matrix factorization methods and deep-based methods.

In summary, the main contributions of our work can be distilled to the following.

  • Different from the existing works, which focus on learning the user-based and item-based feature representations with identical models, we provide a novel viewpoint for personalized recommendation by taking two different types of feature representations models into consideration. In this way, the proposed method can extract richer feature representations with different characteristics of the user-based and item-based data and further improve the recommendation performance.

  • We design a novel autoencoder model to learn robust feature representations for users, which is referred to as autoencoder with graph regularization (AEGR). It is helpful for constructing connections in higher-level feature space.

  • The extensive experiments on several datasets are conducted to confirm the effectiveness of the proposed method compared to other state-of-the-art matrix factorization methods and deep-based methods.

Section snippets

Related work

Personalized recommendation systems estimate users’ preferences for items and recommend items to users that they might like. Based on how the recommendation is performed, existing methods for recommendation systems can roughly be categorized into three categories: content-based, collaborative filtering, and hybrid methods. In addition, based on how the features of collaborative filtering are learned, matrix factorization and deep neural network have provoked a vast amount of attention and

Preliminaries and methods

Before introducing the details of our proposed CAPR, the preliminaries that applied in the method will be reviewed as follows.

Collaborative autoencoder for personalized recommendation

In this section, the details of our proposed CAPR are given, and the whole framework is illustrated as Fig. 1.

Experiments

In this section, we conduct extensive experiments on three datasets to evaluate the effectiveness of our proposed method for personalized recommendation. In the following, firstly, three datasets used in the experiments are introduced in detail. Secondly, we introduced the compared methods and evaluation metrics of our experiments. Then we presented the results of comparative experiments and their observations. Finally, the main parameters of our proposed CAPR are analyzed on certain dataset.

Conclusion

In this paper, we propose a representation learning method with Collaborative Autoencoder for Personalized Recommendation, called CAPR, to overcome the shortcomings of the state-of-the-art methods and achieve more desirable performance. CAPR has the following three advantages for gaining performance improvement: (1) it extracts different characteristics of the user-based and item-based data; (2) a novel representation learning method with two different autoencoders are designed to learn the

CRediT authorship contribution statement

Yi Zhu: Conceptualization, Methodology, Software, Writing - original draft. Xindong Wu: Supervision, Formal analysis, Project administration. Jipeng Qiang: Visualization, Writing - review & editing. Yunhao Yuan: Data curation, Visualization. Yun Li: Software, Writing - review & editing, Validation.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This research is supported by the National Natural Science Foundation of China under grant 61906060, National Key Research and Development Program of China under grant 2016YFC0801406 and the Program for Changjiang Scholas and Innovative Research Team in University (PCSIRT) of the Ministry of Education of China under grant IRT17R32.

References (39)

  • GaoJ. et al.

    Hybrid microblog recommendation with heterogeneous features using deep neural network

    Expert Systems with Applications

    (2020)
  • GuoH. et al.

    DeepFM: a factorization-machine based neural network for CTR prediction

    (2017)
  • HeX.

    Locality preserving projections

    Advances in Neural Information Processing Systems

    (2003)
  • HeX. et al.

    Neural collaborative filtering

  • Hu, L., Cao, L., Wang, S., Xu, G., Cao, J., & Gu, Z. (2017). Diversifying personalized recommendation with user-session...
  • KorenY.

    Factorization meets the neighborhood: a multifaceted collaborative filtering model

  • Lee, D. D., & Seung, H. S. (2001). Algorithms for non-negative matrix factorization. In International conference on...
  • LeeJ. et al.

    PREA: Personalized recommendation algorithms toolkit

    Journal of Machine Learning Research

    (2014)
  • Li, S., Kawale, J., & Fu, Y. (2015). Deep collaborative filtering via marginalized denoising auto-encoder. In ACM...
  • Cited by (22)

    • Graph-embedding-inspired article recommendation model

      2023, Expert Systems with Applications
      Citation Excerpt :

      Therefore, we adopt PMF to update the “User-Article” correlation matrix and the article feature matrix to boost the final commendation results. AutoEncoder (AE) is one of the core models of deep learning (Bellini et al., 2019), which consists of Encoder and Decoder (Zhu et al., 2021). The Encoder transforms the original feature, X, into low-dimensional latent embeddings through multiple hidden layers, Z: Z = f(X).

    • Social recommendation via deep neural network-based multi-task learning

      2022, Expert Systems with Applications
      Citation Excerpt :

      Other works are also trying to make use of deep learning techniques to model the nonlinear interaction between users and items. CAPR (Zhu et al., 2021) learned user-based and item-based feature representations by two different autoencoders for capturing different features of the data, while (Ying et al., 2018) generated embeddings of items by GCN to incorporate both user–item graph structure as well as node feature information. As to social recommendations, deep learning has also been developed in this field.

    View all citing articles on Scopus
    View full text