Modeling sentimental bias and temporal dynamics for adaptive deep recommendation system
Introduction
Recommender systems are efficient tools for online applications that rely on the purchase history, ratings, and reviews given by online users to profile their interests and make a personalized recommendation (Da’u, A., Salim, N., Rabiu, I., & Osman, A., 2020b; Kumar, 2016; Rabiu et al., 2020; Torrent-Fontbona and Lopez, 2019, Zhang et al., 2016, Zhang et al., 2017). Generally, the personalized item recommendations follow two approaches, namely, the content-based (CB) and collaborative filtering (CF) techniques (Da’u et al., 2020b; Da’u, Salim, & Idris, 2020a; Isinkaye, Folajimi, & Ojokoh, 2015). CB involves matching up the user’s attributes whose interests and preferences are stored against the target item attributes to make new recommendations to the user (Li, Zheng, Yang, & Li, 2014). While CF-based method utilizes the user's profile and the identical choice of similar users to build the model and make a personalized prediction of new products to users (Isinkaye et al., 2015; Murali et al., 2019; Rabiu, Salim, Da’u, Osman, & Nasser, 2020). In CF-based approach, the matrix factorization method (MF) is commonly used to make a personalized item recommendation (Matuszyk and Spiliopoulou, 2014, Wangwatcharakul and Wongthanavasu, 2020). Nevertheless, the accuracy of the state-of-the-art recommendation models decreases due to the sparsity of the rating matrix (Beutel et al., 2018, Idrissi and Zellou, 2020).
In recent years, review texts created by the users have been exploited to extract more information about user preferences and interests to address the rating sparsity problem and improve the performance of RS (Adomavicius and Tuzhilin, 2015, Chen et al., 2015, Yang et al., 2014). It is believed that such comments generated by users can provide a useful source to derive explanatory information which does not only improve the users’ understanding of the basic criterion but a mechanism that leads to an accurate recommendation (Wangwatcharakul & Wongthanavasu, 2020). On this note, many researchers have tried to use sentiment classification methods to extract user’s sentiments based on review texts which could either be positive or negative, and subsequently considered for appropriate recommendations (Champiri et al., 2015, Ning et al., 2017, Panniello et al., 2014). Other works exploit review topics and latent factors based on hidden factors as topics (HFT) to enhance the personalized predictions (McAuley & Leskovec, 2013b), while Xu et al., (2019) utilized both the users’ ratings and reviews in two-step neural networks to extract the users’ true sentiments in unbalanced datasets to resolve the sparsity and enhance the rating predictions. They employed a combination function for merging the sentiment scores and ratings which can filtered opinion bias between reviews and ratings while building a fine-grained user-item rating matrix. However, the challenges of the aforementioned approaches is that they ignore the change of user preferences that occur over time, which may lead to poor recommendation performance. On this note, several methods have been designed to effectively model user behavioural patterns and preferences as they change over time (McAuley & Leskovec, 2013a).
Lately, dynamics of user preferences and other sub-problems, such as the data sparsity and cold start problems have been considered (Li et al., 2019, Rabiu et al., 2020, Rafailidis et al., 2017, Wu et al., 2018, Xia et al., 2021). For example, a Temporal Collective Matrix Factorization (TCMF) (Rafailidis et al., 2017) model is proposed to address temporal dynamics and to reduce data sparsity by combining the rating scores and multimodal information, while a dynamic approach (OCF-DR) is proposed by (Li et al., 2019), which uses a neighbourhood factor model to track the users’ preference drift. Similarly, Liu, Liu, Shen, & Li, (2018) developed a rating prediction method that jointly addresses sparsity problem and tracks the dynamics of user preferences in a single learning phase. In this approach, the model is designed by grouping review texts into interim and intrinsic words in each time period, which are subsequently linked into the short-term item features and long-term item features to increase the prediction accuracy. Wangwatcharakul & Wongthanavasu, (2020) proposed a model to track changes in user preferences through the latent user factor transitions and the incorporation of latent topic evolutions. To improve the prediction accuracy, authors extended the existing model by introducing a forgetting time function to resolve the rating shift rate and a multiple transition factor to capture the evolution of preferences over time (Wangwatcharakul & Wongthanavasu, 2021). A similar approach based on Multi-Trans matrix factorization (MTMF) model was proposed by (Zhang & Lu, 2020) with an improved forgetting function to capture temporal dynamics. In this approach, a time weight is first introduced that utilized the forgetting curve and item similarity to decrease the impact of old information. Then, the retained preference information is used to model temporal dynamics by learning the multiple transitions through the user and item factors in the latent space between the past and current time periods. However, the above approaches have the merit that they are easy to implement, specifically, decay-based models. They can’t capture the refined temporal characteristics properly.
Furthermore, with the introduction of deep learning-based methods, some deep learning approaches, such as the User Word Composition Vector Model (UWCVM), Adaptive Deep learning-based Recommendation System (ADRS) and Temporal Deep Semantic Structured Model (TDSSM) have been proposed (Da’u, Salim, & Idris, 2021; Song et al., 2016, Tang et al., 2015). However, the UWCVM (Tang, et al., 2015) and the ADRS (Da'u, et al., 2021) ignore users’ changing preferences and item properties. The TDSSM (Song et al., 2016) only takes into account the changing preferences of users, and neglects the changed item features. Similarly, (Yu, Lian, Mahmoody, Liu, & Xie, 2019) proposed an improved RNN model by incorporating the time-aware and content-aware controllers such that the temporal information can be well used to model the state transitions. Zhu, Xu, & Zhu, (2020) proposed a novel Parallel Attention Network model (PAN) for session-based recommender system to captures the users’ evolving interests. However, most of the above methods performed poorly by only utilizing the last clicked items to represent the user’s short-term interest but fails to capture the overall user preference broadly, or attaching equal importance to both the long-term and short-term interests which are known to be user-specific.
Considering the aforementioned issues among others, the above temporal models are characterized by lower performances for two reasons: (1) they do not properly uncover the important information hidden from review texts by extracting the user’s true opinion in each time step to address the sparse and unbalanced situations. (2) they either ignore the changed interest of users and changes item properties simultaneously or they can’t capture the refined temporal characteristics properly. Motivated by the above inspirations, it is interesting to learn how recommender systems can properly utilize both the ratings and reviews provided by online users to resolve the sparsity problems and the dynamics of users’ preferences to enhance recommendation accuracy. Thus, in this paper, we propose an adaptive deep learning-based RS by modeling Sentimental Bias and Temporal Dynamics. The proposed model is comprised of three modules: LSTM-based sentiment classification method, Adaptive LSTM (ALSTM) method, and prediction layer for item recommendation. Specifically, the LSTM-based sentiment classification method is designed to extract the user’s true opinion from review comments to address the sparsity problem and enhance the rating matrix. Next, we fed enhanced rating mating to ALSTM to model the dynamic user and item features. To capture the dynamic user features, we proposed a new model that combines the temporal features at different rates, i.e., short-term and long-term features, each of which is modeled separately. While for the dynamic item features we use LSTM whose output is concatenated with users’ network. Finally, the Multilayer Perceptron (MLP) model is employed at the prediction layer to calculate the rating predictions based on the dynamic user and item representations. The contributions of this research work are summarized as follows:
- •
we proposed a deep learning method using an LSTM-based Semantic Scoring Model (SSM) to extract the users’ sentiments which addresses the unbalanced and data sparsity problems by merging the rating scores and a semantic classification score while filtering the opinion bias through a novel combination function to enhance the rating matrix.
- •
we designed an adaptive-based LSTM model for tracking the changed interest of users and changes in item properties simultaneously, through the enhanced rating matrix for better predictions. To handle the dynamic user behaviours, we exploited the long-term and short-term interests and incorporate the time features to model state transitions from different intervals.
- •
through series of experiments, we show the superiority of our model compare to the existing static-based and temporal-based models on three real-world datasets.
The rest of this paper is organized as follows: a brief discussion of the related works is provided in Section 2. In Section 3, we present the research preliminaries, and the proposed model is discussed in Section 4. In Section 5, we present the experimental results of the proposed model and how it is compared with the existing models. Finally, Section 6 presents the conclusions and future works of the research.
Section snippets
Related works
Here, we provide a literature review of the related works on two research areas, which includes (1) the works that exploit ratings and reviews for recommendations, (2) the temporal-based collaborative filtering models that track the dynamic change of user preferences in recommendation systems. A brief discussion on each of these approaches is presented as follows.
Preliminary
In this section, some basics and problem definitions of RSs are presented which motivates our work on modeling the sentimental bias and temporal dynamics for adaptive RS.
Proposed model
In this section, we briefly introduce the framework of the proposed model as shown in Fig. 1. The proposed model is comprised of three modules: LSTM-based Sentiment Scoring Model (SSM) for sentiment classification, Adaptive LSTM (ALSTM) method that captures dynamic changes in user and item latent features, and prediction layer for item recommendation. Similar to (Xu et al., 2019), we first perform the sentiment analysis by taking all the users’ review text as input and produces the sentiment
Experiments
In this section, we conduct different experiments to evaluate the performance of our proposed model Adaptive LSTM on three real-world datasets. Firstly, we begin by interpreting the datasets, followed by baselines and the metrics used for evaluations. And finally, we present the results of experiments and discussions.
Conclusion
In this paper, we proposed a novel ALSTM model to enhance a dynamic recommender system under volatile conditions in which both users’ preferences and item properties change over time. We notice that modeling users’ preference dynamics brings unique challenges that required a more sensitive approach that can make better distinctions between transient effects and long-term patterns. Thus, we introduced a time gate to make the classical LSTM more suitable for dynamic user behavior changes.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
This research was funded by the Universiti Teknologi Malaysia, through the grant number: Q.J13000.2551.21H38-Novel Deep Learning, Concept Drift, and Hybrid Models Research Funding Program.
References (75)
- et al.
Recommendation systems: Principles, methods and evaluation
Egyptian Informatics Journal
(2015) - et al.
Modeling and broadening temporal user interest in personalized news recommendation
Expert Systems with Applications
(2014) - et al.
A cross-domain recommender system with consistent information transfer
Decision Support Systems
(2017) - Adomavicius, G., & Tuzhilin, A. (2015). Context-aware recommender systems. In Recommender Systems Handbook, Second...
An Introduction to Recommender Systems. In: Recommender Systems
(2016)- et al.
Adaptive collaborative topic modeling for online recommendation
- et al.
TopicMF: Simultaneously exploiting ratings and reviews for recommendation
Proceedings of the National Conference on Artificial Intelligence
(2014) - et al.
Latent cross: Making use of context in recurrent recommender systems
- et al.
Recommender systems survey
Knowledge-Based Systems
(2013) - et al.
A systematic review of scholar context-aware recommender systems
Expert Systems with Applications
(2015)
Recommender systems based on user reviews: The state of the art
User Modeling and User-Adapted Interaction
Matrix factorization for recommendation with explicit and implicit feedback
Knowledge-Based Systems
A new recommendation algorithm based on user’s dynamic information in complex social network
Mathematical Problems in Engineering
ANR: Aspect-based Neural Recommender
A fast parallel stochastic gradient method for matrix factorization in shared memory systems
ACM Transactions on Intelligent Systems and Technology
Modeling temporal adoptions using dynamic matrix factorization
Proceedings - IEEE International Conference on Data Mining, ICDM.
Sentiment-Aware Deep Recommender System with Neural Attention Networks
IEEE Access
An adaptive deep learning method for item recommendation system Knowledge-Based Systems
Knowledge-Based System
Recommender system exploiting aspect-basd opinion mining with deep learning method
Information Sciences
Weighted aspect-based opinion mining using deep learning for recommender system
Expert Systems with Applications
Jointly modeling aspects, ratings and sentiments for movie recommendation (JMARS)
Evaluating Group Recommender Systems.
ADADRIFT: An Adaptive Learning Technique for Long-history Stream-based Recommender Systems
IEEE Transactions on Systems, Man, and Cybernetics: Systems.
Interactive recommender systems: A survey of the state of the art and future research challenges and opportunities
Expert Systems with Applications
TriRank: Review-aware explainable recommendation by modeling aspects
International Conference on Information and Knowledge Management, Proceedings.
Neural factorization machines for sparse predictive analytics
Collaborative filtering for implicit feedback datasets
Proceedings - IEEE International Conference on Data Mining, ICDM.
A systematic literature review of sparsity issues in recommender systems
Social Network Analysis and Mining
Leveraging the dynamic changes from items to improve recommendation
Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Collaborative filtering with temporal dynamics
Communications of the ACM
Advances in collaborative filtering. Recommender systems handbook
Springer
A novel latent factor model for recommender system
Journal of Information Systems and Technology Management
Sparse online collaborative filtering with dynamic regularization
Information Sciences
Ratings meet reviews, a combined approach to recommend
Recurrent neural network for text classification with multi-task learning
IJCAI International Joint Conference on Artificial
Long-and short-term preference model based on graph embedding for sequential recommendation. In International Conference on Database Systems for Advanced Applications
Springer, Cham.
Cited by (8)
Improved literature recommendation system through the fusion mode of conformity bias elimination and keyword preference
2024, Expert Systems with ApplicationsApplication of Recommendation Algorithms Based on Social Relationships and Behavioral Characteristics in Music Online Teaching
2024, International Journal of Web-Based Learning and Teaching TechnologiesE-commerce Product’s Trust Prediction Based on Customer Reviews
2023, Lecture Notes in Networks and SystemsA Systematic Review of Recommendation System Based on Deep Learning Methods
2023, Lecture Notes in Networks and Systems