Hybrid recommendation approaches for multi-criteria collaborative filtering

Highlights

• We propose new recommendation methods using ANFIS and SOM clustering.

• We extend fuzzy approaches for multi-criteria collaborative filtering.

• We improve the predictive accuracy of multi-criteria collaborative filtering.

Abstract

Recommender systems are software tools and techniques for suggesting items in an automated fashion to users tailored their preferences. Collaborative Filtering (CF) techniques, which attempt to predict what information will meet a user’s needs from the neighborhoods of like-minded people, are becoming increasingly popular as ways to overcome the information overload. The multi-criteria based CF presents a possibility to provide accurate recommendations by considering the user preferences in multiple aspects and several methods have been proposed for improving the accuracy of these systems. However, the problem of multi-criteria recommendations with a single and overall rating is still considered an optimization problem. In addition, increasing the accuracy in predicting the appropriate items tailored to the users’ preferences is on of the main challenges in these systems. Hence, in this research new recommendation methods using Adaptive Neuro-Fuzzy Inference Systems (ANFIS) and Self-Organizing Map (SOM) clustering are proposed to improve predictive accuracy of criteria CF. In this research, SOM enables us to generate high quality clusters of dataset and ANFIS is used for discovering knowledge (fuzzy rules) from users’ ratings in multi-criteria dataset, generating appropriate membership functions (MFs), overall rating prediction and input selection. Using exhaustive search method for input selection, the effective inputs are determined to build the ANFIS models in all generated clusters. Furthermore, new fuzzy-based algorithms, Weighted Fuzzy MC-CF (WFuMC-CF), Fuzzy Euclidean MC-CF (FuEucMC-CF) and Fuzzy Average MC-CF (FuAvgMC-CF), are presented for prediction task in multi-criteria CF. FuEucMC-CF and FuAvgMC-CF algorithms uses the fuzzy-based Euclidian distance and fuzzy-based average similarity, respectively, the WFuMC-CF algorithm uses fuzzy-based user- and item-based prediction in a weighted approach. Experimental results on real-world dataset demonstrate that the proposed hybrid methods remarkably improve the accuracy of multi-criteria CF in relation to the previous methods based on multi-criteria ratings.

Introduction

Recommender systems support the online customer in his/her decision-making and buying process (Jannach, 2006, Jannach et al., 2012). The implementation of recommender systems in the Internet has increased in the diverse areas (Park et al., 2012a, Park et al., 2012b). The most common research papers are focused on movie recommendation studies (Anand and Mampilli, 2014, Carrer-Neto et al., 2012, Winoto and Tang, 2010); however, a great volume of literature for recommender systems is centered on different topics, such as music (Bogdanov et al., 2013, Hyung et al., 2014, Lee et al., 2010, Palanivel and Siavkumar, 2010, Tan et al., 2011), news articles (Das, Datar, Garg, & Rajaram, 2007), hotels (Jannach, Karakaya, & Gedikli, 2012), television (Barragáns-Martínez et al., 2010, Yao and Zhang, 2009), books (Núñez-Valdéz et al., 2012, Crespo et al., 2011), restaurant (Park, Park, & Cho, 2008), documents (Porcel and Herrera-Viedma, 2010, Porcel et al., 2009, Porcel et al., 2012, Serrano-Guerrero et al., 2011), elearning (Bobadilla et al., 2009, Zaíane, 2002), e-commerce (Castro-Schez et al., 2011, Huang et al., 2007), applications in markets (Costa-Montenegro, Barragáns-Martínez, & Rey-López, 2012), tourism (Fuchs & Zanker, 2012) and web search (McNally, O’Mahony, Coyle, Briggs, & Smyth, 2011), among others.

A recommender system consists of two basic entities: users and items, where users provide their opinions (ratings) about items. Items indicated by I (i₁, i₂, … , i_N) are the objects that are recommended. In all domain items are stored in set I. The possibly unique item identifiers can either be proprietary product codes from an ecommerce site such as Amazon.com’s ASINs or globally accepted codes such as ISBNs, ISSNs and so on. Items can be identified through their value and utility or their complexity. In the recommendation process the value for an item can be perfected as positive (usefulness) or negative (inappropriate) by the users. Users indicated by U (u₁, u₂, … , u_M) comprise all the users that have browsed items or had a contribution for giving rating to items. According to Adomavicius and Tuzhilin (2005), the recommendation problem can be formulated as: Let U be the set of all registered users in a recommender system, and let I be the set of all possible items users have access to in the system. Let g: U × I → R, where R is a totally ordered set (e.g., non-negative integers or real numbers within a certain range), be a utility function such that g (u_m, i_n) measures the gain or usefulness of item i_n to user u_m. Then, for each user u_m ∈ U, the aim is to choose an item $i^{\max \text{,}{u}_m}\in I$, unknown to the user, which maximizes the utility function g. More formally:

$$\forall u_m\in U\text{,}{i}^{\max \text{,}{u}_m}=\arg {\displaystyle \underset{i_n\in I}{\max }}g(u_m\text{,}{i}_n)$$

In recommender systems, the utility of an item is usually represented by a rating, which measures how much a specific user is interested in the item. Depending on the application, the ratings can either be specified by the users, or be computed by the system.

Identifying those users who have similar taste to the active user in the past is crucial for successful application of Collaborative Filtering (CF). The two most frequently used approaches are Pearson correlation and cosine-based approach (Liu et al., 2013, Nilashi et al., 2013). The similarity based on Pearson correlation is calculated as follows:

$$\mathit{sim}(u\text{,}{u}^{\prime })=\frac{{\sum }_{i\in I}(r_{u\text{,}i}-{\overline{r}}_u)(r_{u^{\prime }\text{,}i}-{\overline{r}}_{u^{\prime }})}{\sqrt{{\sum }_{i\in I}{(r_{u\text{,}i}-{\overline{r}}_u)}^2}\sqrt{{\sum }_{i\in I}{(r_{u^{\prime }\text{,}i}-{\overline{r}}_{u^{\prime }})}^2}}$$

where I represents the set of common items rated by user u and user u′. The value of any of the two approaches ranges from −1 to +1. The greater the value, the more similar these two users are. Thus, −1 means that the two users have exact opposite taste, and +1 means they have exactly the same taste. r_u,i and $r_{u^{\prime }\text{,}i}$ denotes the ratings users u and u′ to items i. Also, ${\overline{r}}_u$ and ${\overline{r}}_{u^{\prime }}$ indicate each user’s average ratings.

Recommender systems based on CF recommender systems (Barragáns-Martínez et al., 2010, Bobadilla et al., 2012, Bobadilla et al., 2012, Cechinel et al., 2013, de Campos et al., 2010, Lee et al., 2010, Lika et al., 2014, Liu et al., 2013, Luo et al., 2013, Tsai and Hung, 2012) are particularly popular and used by large online retailers. CF techniques are used for producing personalized recommendations by computing the similarity between the current user and other users with similar choices. Examples of these systems include the GroupLens system (Konstan et al., 1997), and Ringo (www.ringo.com). CF algorithms can be divided into two categories: memory-based algorithms and model based algorithms (Adomavicius and Tuzhilin, 2005, Deshpande and Karypis, 2004). Memory-based algorithms exploit the entire item-user database. A set of similar users are identified for the current user, and rating predictions are generated based on ratings in the neighborhood of the current user (Ghazanfar & Prügel-Bannett, 2013a). Memory-based CF is easy to implement and new data addition is easy to add incrementally. In the memory-based category, the most popular non-probabilistic approach is the K-Nearest Neighbor algorithm (K-NN). Memory-based approaches can be further classified into user-based and item-based. In the user-based approach, the recommendation algorithm searches for users similar to the user that requires a recommendation (called the active user) in terms of ratings for the previously seen items. Then, the rating for the unknown item is predicted based on what ratings were assigned to this item by other, similar users (Deshpande and Karypis, 2004, Sarwar et al., 2000, Schein et al., 2002). Item-based CF proposed by Sarwar, Karypis, Konstan, and Riedl (2001) is as an alternative style of CF where the item–item similarity is utilized to make the prediction. It avoids the scalability bottleneck associated with the traditional user-based algorithm. The bottleneck arises from the search for neighbors in a population of users that is continuously growing.

In contrast to the heuristics that are based mostly on information retrieval methods, model-based approaches use mathematical models to reduce the dimensionality of the data (Koren, Bell, & Volinsky, 2009). In addition, model-based methods adopt an eager learning strategy (de Campos et al., 2010, Ghazanfar and Prügel-Bannett, 2013a, Ghazanfar and Prugel-Bennett, 2013b) such as Higher-Order Singular Value Decomposition (HOSVD) (Symeonidis, Nanopoulos, & Manolopoulos, 2008), SVD (Ghazanfar and Prugel-Bennett, 2013b, Kurucz et al., 2007; Vozalis & Margaritis, 2007), Bayesian networks (Breese, Heckerman, & Kadie, 1998), fuzzy logic (Luo, Liu, Zhang, Ye, & Xu, 2008) clustering (Al Mamunur Rashid et al., 2006, Park and Tuzhilin, 2008, Sarwar et al., 2002, Xue et al., 2005), and Kernel-mapping recommender (Ghazanfar et al., 2012, Ghazanfar et al., 2011) for predicting or recommending content, where a model of the data, i.e. the users, items and their ratings for those items, is pre-computed.

The rating provided by a user on the items in recommender system is primal in dealing with recommender system. Interestingly though, rating makes available/supplies the required data or information about the quality and the requisite preference in respect of a user that provide the rating of the item. Often times recommender systems are primarily designed only for single-valued ratings. This means that, for each user, there is a single rating indicating barely how much the user liked the item.

According to Adomavicius and Kwon (2007), pure CF-based recommender systems rely solely on product ratings provided by a large user community to generate personalized recommendation lists for each individual online user. In traditional CF systems the assumption is that customers provide an overall rating for the items which they have purchased, for example, using a 5-star rating system. However, given the value of customer feedback to the business, customers in some domains are nowadays given the opportunity to provide more fine-grained feedback and to rate products and services along various dimensions (Jannach et al., 2012, Adomavicius et al., 2011). Adomavicius and Kwon (2007) stated that single-rating CF recommenders are indicated as systems that attempt to estimate a rating function R that has a form users × items → R₀ for predicting a rating for any had given user-item pair. R₀ is totally ordered set, typically composed of real-valued numbers inside a certain range. They further disclosed that, in multi-criteria recommender systems, in comparison, the rating function R₀ has got the form users ×  items → R₀ × R₁ × ⋯ × R_k. Therefore, the system has to predict a general rating R₀ in addition to k additional criteria ratings. This means that, multi-criteria system provides more information about user’s preferences than a single-rating system. In addition, by adopting a decision theory, multi-criteria systems can provide rich tools for system designer to build more interesting systems as well (Lakiotaki, Matsatsinis, & Tsoukiàs, 2011). Refer to Adomavicius and Kwon, 2007, Jannach et al., 2012 for more information.

The algorithm for a multi-criteria CF recommender system can be extended from a single-rating recommender system. In multi-criteria CF problem, there are m users, n items and k criteria in addition to the overall rating. Users have provided a number of explicit ratings for items; a general rating R₀ must be predicted in addition to k additional criteria ratings (R₁, … ,R_k). It can be configured to push new items to users in two ways, either by producing a Top-N list of recommendations for a given target, or by predicting the target user’s likely utility (or rating) for a particular unseen item. We will refer to these as the recommendation task and the rating prediction task in multi-criteria CF, respectively. Fig. 1 demonstrates the multi-criteria CF problem in case of prediction and recommendation tasks for an active user U_a and item I_a. Recommendation is a list of N products, TP_r = {T_p1, T_p2, … , T_pN}, that the active user will like the most. The recommended list usually consists of the products not already purchased by the active customer. This output interface of multi-criteria CF algorithms is also known as Top-N recommendation. Fig. 1 shows the schematic diagram of the multi-criteria CF process. Multi-criteria CF algorithms represent the entire m × n × k user-item-criteria data as a tensor of ratings, A. Each entry a_i,j in tensor A as shown in Fig. 1 represents the preference score (ratings) of the ith user on the jth item as overall preference in addition to criteria ratings in the 3rd dimension. Each overall and criteria rating is within a numerical scale and it can as well be 0, indicating that the user has not yet rated that product.

In the context of personalization applications, traditional single-rating CF have been highly successful however, the research area regarding of the CF with multi-criteria ratings for items has been rarely touched and fairly this issue is largely unexplored. According to Adomavicius and Kwon (2007), the problem of multi-criteria recommendations with a single and overall rating is still considered an optimization problem.

Usually in CF systems, the user’s preference ratings are represented in terms of numerical values. Many aspects in the real world cannot be assessed in a quantitative form, but rather in a qualitative form with vague or imprecise knowledge (Martinez et al., 2008, Zenebe and Norcio, 2009). In addition, according to Zenebe and Norcio (2009), users’ behavior on features of items is imprecise, subjective and vague that consequently leads to uncertainty in reasoning on perception of user on items’ features. Concerning the user behavior and perception for example interest, the uncertainty is connected to how user interest precisely can be represented and measured. Multi-criteria CF face this problem from the two sides, in the criteria and overall ratings. Thus, this problem in multi-criteria CF has to be taken into consideration. It has been shown that the use of fuzzy set techniques in recommender systems is a proper method to model the natural complexity of human behavior and to handle the uncertainty and fuzziness on user preferences (Anand and Mampilli, 2014, Cornelis et al., 2007, Lu et al., 2012, Nilashi and Ibrahim, 2013). In multi-criteria CF, the fuzzy set can better solve this problem instead of relying solely on crisp approaches such as clustering methods, dimensionality reduction techniques, and also regression approaches. Fuzzy methods can predict the exact user preferences based on items’ features in multi-criteria CF. This means, they can better solve the sparsity problem with accurate rating prediction.

Other issue in multi-criteria CF is discovering the knowledge from the user preferences that can be used in the prediction and recommendation processes. In fuzzy logic, it is usually difficult to determine the correct set of rules and membership functions from the users’ preferences in multi-criteria CF. In addition, fine-tuning a fuzzy solution is even more difficult and takes longer. In addition, fuzzy logic controllers do not possess capabilities for automated learning. Thus, this is an important issue that has to be considered. For solving this problem, recently neural networks have been employed to real-world problems (Park et al., 2012a, Park et al., 2012b) because of its capability of automated learning (Nyongesa, 1998). It learns system behavior by using system input–output data. Neural networks have good generalization capabilities. The learning and generalization capabilities of neural networks enable it to more effectively address real-world problems. Thus, neural networks can solve many problems that are either unsolved or inefficiently solved by existing techniques, including fuzzy logic. However, in neural networks, it is difficult to understand the “Black Box,” i.e., it is incomplete compared to a fuzzy rule based system description (Liu, Liu, Zhang, & Wu, 2004).

Thus, these limitations prevent them from providing efficient solutions for multi-criteria CF problems. An appropriate combination of these two technologies, Neuro-Fuzzy Inference Systems (ANFIS), can effectively solve the problems of fuzzy logic and neural networks and, thus, can more effectively address the multi-criteria CF problems. A Neuro-Fuzzy approach was used to take advantage of the neural network’s ability to learn, and membership degrees and functions of fuzzy logic. The weights of the neural networks are mapped to fuzzy logic rules and member functions. Expressing the weights of the neural network by fuzzy rules also provides a better understanding of the “Black Box” and thus helps the better design of the neural network itself. Thus, while the learning of neural network is parameterized by the variation in input data, the learning of ANFIS is fixed by the rules and membership function values that we define. A Neuro-Fuzzy system is functionally equivalent to a FIS. A FIS mimics a human reasoning process by implementing fuzzy sets and approximate reasoning mechanism that uses numerical values instead of logical values. A Neuro-Fuzzy system can replace the knowledge acquisition process by humans using a training process with a set of input–output training dataset. Thus, instead of dependent on human experts the Neuro-Fuzzy system will determine the parameters associated with the Neuro-Fuzzy system through a training process, by minimizing an error criterion.

Therefore, in this paper, we investigate for discovering the knowledge from users’ preferences rating, solving the uncertainty problem and improving the predictive accuracy of multi-criteria CF that is the main objective of our research. Specifically, we look at four key questions:

• How the predictive accuracy of multi-criteria CF can be improved using Self-Organizing Map (SOM) and ANFIS?

• How the non-stochastic uncertainty emerged from vagueness and imprecision of user perception on items’ features can be solved in the multi-criteria CF?

• How the knowledge can be extracted from users’ preferences rating without the human expert intervention in the multi-criteria CF?

The remainder of this paper is organized as follows. The related work is explained in Section 2. Section 3 introduce the fuzzy techniques, SOM clustering and ANFIS for proposed methods. Section 4 provides research methodology and hybrid proposed methods and algorithms. Section 5 presents the result and discussion and finally, conclusions and future work is presented in Section 6.