Modeling sentimental bias and temporal dynamics for adaptive deep recommendation system

https://doi.org/10.1016/j.eswa.2021.116262Get rights and content

Highlights

  • The proposed method addresses the unbalanced and data sparsity problems.

  • Sentiment bias and temporal dynamics are explored to improve recommendation quality.

  • We design a deep learning method based on long- and short-term user preferences.

  • The performance of the proposed models is evaluated on several Amazon datasets.

Abstract

Recommendation systems rely on the historic data of users' purchases and their feedbacks to profile their preferences and make future recommendations. Most of these systems usually employ Collaborative Filtering (CF) models to analyze users’ ratings and infer the latent factors which represent the user and item features in k-dimensional latent space. However, the historical rating data used for recommendations are usually sparsed and unbalanced. Various approaches have been used to resolve these issues by combining the user’s ratings and reviews to better capture the user’s sentiments and make accurate recommendations. Other challenges comprise changes in users’ preferences and items’ perceptions over time. Therefore, this paper presents a new Sentiment Scoring Model (SSM) based on Long-/Short-Term Memory and a combination function that catches the sentiment bias between user rating and review to relieve the sparsity and unbalanced dataset. Next, we proposed an Adaptive LSTM (ALSTM) method that can model the drifting of user and item features to improve the recommendation accuracy. We show the performance of our model on the three real-world rating datasets from Amazon reviews, which comprises Fine Food, Baby, and Cell-phone & Accessories categories. The result shows the superiority of our proposed model over the existing static and dynamic models. The statistical test shows that all the performance gains are significant at p < 0.05.

Introduction

Recommender systems are efficient tools for online applications that rely on the purchase history, ratings, and reviews given by online users to profile their interests and make a personalized recommendation (Da’u, A., Salim, N., Rabiu, I., & Osman, A., 2020b; Kumar, 2016; Rabiu et al., 2020; Torrent-Fontbona and Lopez, 2019, Zhang et al., 2016, Zhang et al., 2017). Generally, the personalized item recommendations follow two approaches, namely, the content-based (CB) and collaborative filtering (CF) techniques (Da’u et al., 2020b; Da’u, Salim, & Idris, 2020a; Isinkaye, Folajimi, & Ojokoh, 2015). CB involves matching up the user’s attributes whose interests and preferences are stored against the target item attributes to make new recommendations to the user (Li, Zheng, Yang, & Li, 2014). While CF-based method utilizes the user's profile and the identical choice of similar users to build the model and make a personalized prediction of new products to users (Isinkaye et al., 2015; Murali et al., 2019; Rabiu, Salim, Da’u, Osman, & Nasser, 2020). In CF-based approach, the matrix factorization method (MF) is commonly used to make a personalized item recommendation (Matuszyk and Spiliopoulou, 2014, Wangwatcharakul and Wongthanavasu, 2020). Nevertheless, the accuracy of the state-of-the-art recommendation models decreases due to the sparsity of the rating matrix (Beutel et al., 2018, Idrissi and Zellou, 2020).

In recent years, review texts created by the users have been exploited to extract more information about user preferences and interests to address the rating sparsity problem and improve the performance of RS (Adomavicius and Tuzhilin, 2015, Chen et al., 2015, Yang et al., 2014). It is believed that such comments generated by users can provide a useful source to derive explanatory information which does not only improve the users’ understanding of the basic criterion but a mechanism that leads to an accurate recommendation (Wangwatcharakul & Wongthanavasu, 2020). On this note, many researchers have tried to use sentiment classification methods to extract user’s sentiments based on review texts which could either be positive or negative, and subsequently considered for appropriate recommendations (Champiri et al., 2015, Ning et al., 2017, Panniello et al., 2014). Other works exploit review topics and latent factors based on hidden factors as topics (HFT) to enhance the personalized predictions (McAuley & Leskovec, 2013b), while Xu et al., (2019) utilized both the users’ ratings and reviews in two-step neural networks to extract the users’ true sentiments in unbalanced datasets to resolve the sparsity and enhance the rating predictions. They employed a combination function for merging the sentiment scores and ratings which can filtered opinion bias between reviews and ratings while building a fine-grained user-item rating matrix. However, the challenges of the aforementioned approaches is that they ignore the change of user preferences that occur over time, which may lead to poor recommendation performance. On this note, several methods have been designed to effectively model user behavioural patterns and preferences as they change over time (McAuley & Leskovec, 2013a).

Lately, dynamics of user preferences and other sub-problems, such as the data sparsity and cold start problems have been considered (Li et al., 2019, Rabiu et al., 2020, Rafailidis et al., 2017, Wu et al., 2018, Xia et al., 2021). For example, a Temporal Collective Matrix Factorization (TCMF) (Rafailidis et al., 2017) model is proposed to address temporal dynamics and to reduce data sparsity by combining the rating scores and multimodal information, while a dynamic approach (OCF-DR) is proposed by (Li et al., 2019), which uses a neighbourhood factor model to track the users’ preference drift. Similarly, Liu, Liu, Shen, & Li, (2018) developed a rating prediction method that jointly addresses sparsity problem and tracks the dynamics of user preferences in a single learning phase. In this approach, the model is designed by grouping review texts into interim and intrinsic words in each time period, which are subsequently linked into the short-term item features and long-term item features to increase the prediction accuracy. Wangwatcharakul & Wongthanavasu, (2020) proposed a model to track changes in user preferences through the latent user factor transitions and the incorporation of latent topic evolutions. To improve the prediction accuracy, authors extended the existing model by introducing a forgetting time function to resolve the rating shift rate and a multiple transition factor to capture the evolution of preferences over time (Wangwatcharakul & Wongthanavasu, 2021). A similar approach based on Multi-Trans matrix factorization (MTMF) model was proposed by (Zhang & Lu, 2020) with an improved forgetting function to capture temporal dynamics. In this approach, a time weight is first introduced that utilized the forgetting curve and item similarity to decrease the impact of old information. Then, the retained preference information is used to model temporal dynamics by learning the multiple transitions through the user and item factors in the latent space between the past and current time periods. However, the above approaches have the merit that they are easy to implement, specifically, decay-based models. They can’t capture the refined temporal characteristics properly.

Furthermore, with the introduction of deep learning-based methods, some deep learning approaches, such as the User Word Composition Vector Model (UWCVM), Adaptive Deep learning-based Recommendation System (ADRS) and Temporal Deep Semantic Structured Model (TDSSM) have been proposed (Da’u, Salim, & Idris, 2021; Song et al., 2016, Tang et al., 2015). However, the UWCVM (Tang, et al., 2015) and the ADRS (Da'u, et al., 2021) ignore users’ changing preferences and item properties. The TDSSM (Song et al., 2016) only takes into account the changing preferences of users, and neglects the changed item features. Similarly, (Yu, Lian, Mahmoody, Liu, & Xie, 2019) proposed an improved RNN model by incorporating the time-aware and content-aware controllers such that the temporal information can be well used to model the state transitions. Zhu, Xu, & Zhu, (2020) proposed a novel Parallel Attention Network model (PAN) for session-based recommender system to captures the users’ evolving interests. However, most of the above methods performed poorly by only utilizing the last clicked items to represent the user’s short-term interest but fails to capture the overall user preference broadly, or attaching equal importance to both the long-term and short-term interests which are known to be user-specific.

Considering the aforementioned issues among others, the above temporal models are characterized by lower performances for two reasons: (1) they do not properly uncover the important information hidden from review texts by extracting the user’s true opinion in each time step to address the sparse and unbalanced situations. (2) they either ignore the changed interest of users and changes item properties simultaneously or they can’t capture the refined temporal characteristics properly. Motivated by the above inspirations, it is interesting to learn how recommender systems can properly utilize both the ratings and reviews provided by online users to resolve the sparsity problems and the dynamics of users’ preferences to enhance recommendation accuracy. Thus, in this paper, we propose an adaptive deep learning-based RS by modeling Sentimental Bias and Temporal Dynamics. The proposed model is comprised of three modules: LSTM-based sentiment classification method, Adaptive LSTM (ALSTM) method, and prediction layer for item recommendation. Specifically, the LSTM-based sentiment classification method is designed to extract the user’s true opinion from review comments to address the sparsity problem and enhance the rating matrix. Next, we fed enhanced rating mating to ALSTM to model the dynamic user and item features. To capture the dynamic user features, we proposed a new model that combines the temporal features at different rates, i.e., short-term and long-term features, each of which is modeled separately. While for the dynamic item features we use LSTM whose output is concatenated with users’ network. Finally, the Multilayer Perceptron (MLP) model is employed at the prediction layer to calculate the rating predictions based on the dynamic user and item representations. The contributions of this research work are summarized as follows:

  • we proposed a deep learning method using an LSTM-based Semantic Scoring Model (SSM) to extract the users’ sentiments which addresses the unbalanced and data sparsity problems by merging the rating scores and a semantic classification score while filtering the opinion bias through a novel combination function to enhance the rating matrix.

  • we designed an adaptive-based LSTM model for tracking the changed interest of users and changes in item properties simultaneously, through the enhanced rating matrix for better predictions. To handle the dynamic user behaviours, we exploited the long-term and short-term interests and incorporate the time features to model state transitions from different intervals.

  • through series of experiments, we show the superiority of our model compare to the existing static-based and temporal-based models on three real-world datasets.

The rest of this paper is organized as follows: a brief discussion of the related works is provided in Section 2. In Section 3, we present the research preliminaries, and the proposed model is discussed in Section 4. In Section 5, we present the experimental results of the proposed model and how it is compared with the existing models. Finally, Section 6 presents the conclusions and future works of the research.

Section snippets

Related works

Here, we provide a literature review of the related works on two research areas, which includes (1) the works that exploit ratings and reviews for recommendations, (2) the temporal-based collaborative filtering models that track the dynamic change of user preferences in recommendation systems. A brief discussion on each of these approaches is presented as follows.

Preliminary

In this section, some basics and problem definitions of RSs are presented which motivates our work on modeling the sentimental bias and temporal dynamics for adaptive RS.

Proposed model

In this section, we briefly introduce the framework of the proposed model as shown in Fig. 1. The proposed model is comprised of three modules: LSTM-based Sentiment Scoring Model (SSM) for sentiment classification, Adaptive LSTM (ALSTM) method that captures dynamic changes in user and item latent features, and prediction layer for item recommendation. Similar to (Xu et al., 2019), we first perform the sentiment analysis by taking all the users’ review text as input and produces the sentiment

Experiments

In this section, we conduct different experiments to evaluate the performance of our proposed model Adaptive LSTM on three real-world datasets. Firstly, we begin by interpreting the datasets, followed by baselines and the metrics used for evaluations. And finally, we present the results of experiments and discussions.

Conclusion

In this paper, we proposed a novel ALSTM model to enhance a dynamic recommender system under volatile conditions in which both users’ preferences and item properties change over time. We notice that modeling users’ preference dynamics brings unique challenges that required a more sensitive approach that can make better distinctions between transient effects and long-term patterns. Thus, we introduced a time gate to make the classical LSTM more suitable for dynamic user behavior changes.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

This research was funded by the Universiti Teknologi Malaysia, through the grant number: Q.J13000.2551.21H38-Novel Deep Learning, Concept Drift, and Hybrid Models Research Funding Program.

References (75)

  • F.O. Isinkaye et al.

    Recommendation systems: Principles, methods and evaluation

    Egyptian Informatics Journal

    (2015)
  • L. Li et al.

    Modeling and broadening temporal user interest in personalized news recommendation

    Expert Systems with Applications

    (2014)
  • Q. Zhang et al.

    A cross-domain recommender system with consistent information transfer

    Decision Support Systems

    (2017)
  • Adomavicius, G., & Tuzhilin, A. (2015). Context-aware recommender systems. In Recommender Systems Handbook, Second...
  • C.C. Aggarwal

    An Introduction to Recommender Systems. In: Recommender Systems

    (2016)
  • M. Al-Ghossein et al.

    Adaptive collaborative topic modeling for online recommendation

  • Y. Bao et al.

    TopicMF: Simultaneously exploiting ratings and reviews for recommendation

    Proceedings of the National Conference on Artificial Intelligence

    (2014)
  • A. Beutel et al.

    Latent cross: Making use of context in recurrent recommender systems

  • J. Bobadilla et al.

    Recommender systems survey

    Knowledge-Based Systems

    (2013)
  • Z.D. Champiri et al.

    A systematic review of scholar context-aware recommender systems

    Expert Systems with Applications

    (2015)
  • L. Chen et al.

    Recommender systems based on user reviews: The state of the art

    User Modeling and User-Adapted Interaction

    (2015)
  • S. Chen et al.

    Matrix factorization for recommendation with explicit and implicit feedback

    Knowledge-Based Systems

    (2018)
  • J. Cheng et al.

    A new recommendation algorithm based on user’s dynamic information in complex social network

    Mathematical Problems in Engineering

    (2015)
  • J.Y. Chin et al.

    ANR: Aspect-based Neural Recommender

  • W.S. Chin et al.

    A fast parallel stochastic gradient method for matrix factorization in shared memory systems

    ACM Transactions on Intelligent Systems and Technology

    (2015)
  • F.C.T. Chua et al.

    Modeling temporal adoptions using dynamic matrix factorization

    Proceedings - IEEE International Conference on Data Mining, ICDM.

    (2013)
  • A. Da’U et al.

    Sentiment-Aware Deep Recommender System with Neural Attention Networks

    IEEE Access

    (2019)
  • A. Da’u et al.

    An adaptive deep learning method for item recommendation system Knowledge-Based Systems

    Knowledge-Based System

    (2021)
  • A. Da’u et al.

    Recommender system exploiting aspect-basd opinion mining with deep learning method

    Information Sciences

    (2020)
  • A. Da’u et al.

    Weighted aspect-based opinion mining using deep learning for recommender system

    Expert Systems with Applications

    (2020)
  • Q. Diao et al.

    Jointly modeling aspects, ratings and sentiments for movie recommendation (JMARS)

  • A. Felfernig et al.

    Evaluating Group Recommender Systems.

    (2018)
  • E. Ferreira Jose et al.

    ADADRIFT: An Adaptive Learning Technique for Long-history Stream-based Recommender Systems

    IEEE Transactions on Systems, Man, and Cybernetics: Systems.

    (2020)
  • C. He et al.

    Interactive recommender systems: A survey of the state of the art and future research challenges and opportunities

    Expert Systems with Applications

    (2016)
  • X. He et al.

    TriRank: Review-aware explainable recommendation by modeling aspects

    International Conference on Information and Knowledge Management, Proceedings.

    (2015)
  • X. He et al.

    Neural factorization machines for sparse predictive analytics

  • He, X., Liao, L., Zhang, H., Nie, L., Hu, X., & Chua, T. S. (2017). Neural collaborative filtering. 26th International...
  • Y. Hu et al.

    Collaborative filtering for implicit feedback datasets

    Proceedings - IEEE International Conference on Data Mining, ICDM.

    (2008)
  • N. Idrissi et al.

    A systematic literature review of sparsity issues in recommender systems

    Social Network Analysis and Mining

    (2020)
  • Z. Jin et al.

    Leveraging the dynamic changes from items to improve recommendation

    Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

    (2018)
  • Y. Koren

    Collaborative filtering with temporal dynamics

    Communications of the ACM

    (2010)
  • Y. Koren et al.

    Advances in collaborative filtering. Recommender systems handbook

    Springer

    (2015)
  • B. Kumar

    A novel latent factor model for recommender system

    Journal of Information Systems and Technology Management

    (2016)
  • K. Li et al.

    Sparse online collaborative filtering with dynamic regularization

    Information Sciences

    (2019)
  • G. Ling et al.

    Ratings meet reviews, a combined approach to recommend

  • P. Liu et al.

    Recurrent neural network for text classification with multi-task learning

    IJCAI International Joint Conference on Artificial

    (2016)
  • Y. Liu et al.

    Long-and short-term preference model based on graph embedding for sequential recommendation. In International Conference on Database Systems for Advanced Applications

    Springer, Cham.

    (2020)
  • Cited by (8)

    View all citing articles on Scopus
    View full text