Elsevier

Neurocomputing

Volume 204, 5 September 2016, Pages 51-60
Neurocomputing

Tag-aware recommender systems based on deep neural networks

https://doi.org/10.1016/j.neucom.2015.10.134Get rights and content

Abstract

Many researchers have introduced tag information to recommender systems to improve the performance of traditional recommendation techniques. However, user-defined tags will usually suffer from many problems, such as sparsity, redundancy, and ambiguity. To address these problems, we propose a new recommendation algorithm based on deep neural networks. In the proposed algorithm, users׳ profiles are initially represented by tags and then a deep neural network model is used to extract the in-depth features from tag space layer by layer. In this way, representations of the raw data will become more abstract and advanced, and therefore the unique structure of tag space will be revealed automatically. Based on those extracted abstract features, users׳ profiles are updated and used for making recommendations. The experimental results demonstrate the usefulness of the proposed algorithm and show its superior performance over the clustering based recommendation algorithms. In addition, the impact of network depth on the algorithm performance is also investigated.

Introduction

With the surge of Internet users and tremendous development of individual communication terminals, everyday user-generated data makes people have to face huge information, from which, finding out relevant items for a given user is a big problem to be solved urgently. Therefore, as a useful tool to filter out irrelevant information, recommender systems have attracted increasing attention in recent years [1]. Adopting knowledge discovery technology, recommender systems can predict users׳ future preferences and behaviors according to history records. Thus far, recommendation systems have successfully found applications in diverse fields, such as book recommendations in Amazon.com [2], video recommendations in TiVo.com [3], and movie recommendations in Netflix.com [4].

Many different prediction techniques have been proposed to generate personalized recommendations. Due to its simplicity and promising results, Collaborative Filtering (CF) has been one of the most successful and widely used methods in recommender systems [5]. The core assumption of CF is that users who have expressed similar interests in the past will share common interests in the future [6]. As described in [7], CF can be classified into two major types: memory-based CF and model-based CF. According to the exploited information, memory-based CF is categorized as user based or item based CF. In user-based CF, a user׳s rating on a target item is based on the ratings that several similar users have given to that item in the past [8]. In contrast to user-based CF, item-based CF predicts the rating of the target item based on the ratings that the user has given to other similar items [8]. Model-based CF uses the user-item matrix to train prediction models, such as Bayesian network model [9] and matrix factorization [10]. However, in real-life recommender systems, one user will only rate quite a small fraction of all items and one item is usually rated by a small proportion of all users. Hence the rating matrix is so sparse that it is difficult to find similar users or items because of lacking information [11].

To obtain more useful information about users, many websites introduce a kind of web-based systems, known as social tagging systems [12]. The systems allow users to freely label items with arbitrary words [13], namely user-defined tags. By introducing those tags, recommender systems with tag information (tag-aware recommender systems) [14], [15] are more useful and applied to lots of online sites, such as Del.icio.us1 (with tags of bookmark), Flickr2(with tags of images) and Last.Fm3 (with tags of music). User-defined tags can reflect both users׳ preferences and evaluations on items. From this perspective, tags can be seen as bridges between users and items. Especially, different users usually assign different tags even to the same items, which may benefit personalized recommendations a lot. In addition, tag information can help solve the cold-start problem in recommender systems [16].

Nevertheless, not all the tags benefit recommendations [17]. Arbitrarily associated by users, tags may form an uncontrolled vocabulary, which usually leads to two common situations [18]: (i) similar meanings are expressed with different words, which will be treated as different tags in tagging systems, such as “bike” and “bicycle”; (ii) polysemy, for example, “Hummer” may indicate a buzzer, while it can also represent the brand of a famous car from General Motors Corporation. These issues often cause information redundant and ambiguous and prevent from uncovering the underlying structure or relations among tags. To solve those issues, researchers usually apply clustering-based methods to recommender systems. By clustering [19], redundant tags will be aggregated into the same cluster. Discovering the feature of a cluster is more easily than discovering that of a single tag. In addition, the meaning of a tag is shared by the other tags in the same cluster, so the ambiguity can be alleviated at the same time. However, in tag clustering methods, computing tags׳ similarities accurately is an essential precondition, but intractable due to data sparsity. In addition, tag features need to be represented by extra data, such as the information of items, leading to higher complexity. Hence, it is necessary to find a more straightforward and effective method to address the tag information.

Because of its excellent performance on extracting effective representations of raw data [20], deep neural networks have been applied in many fields, such as image classification [21], [22], [23] and natural language processing [24], [25], [26]. Especially, some researchers have tried to exploit deep neural networks to improve the performance of traditional recommendation algorithms recently. For example, in [27], deep belief network (DBN) was used to music recommendation by learning features from audio data. Similarly, convolutional neural network (CNN) was introduced to music recommendation in [28].

As described in [20], deep architectures are more abstract representations and generally invariant to most local changes of the input data. Thus the extracted abstract and invariant features can detect categories in more varied phenomena and potentially have greater predictive power. In social tagging systems, although tags given by a user often change, the user׳s preference is usually invariant. Accordingly, with the help of deep neural networks, it is possible to learn more abstract features from user׳s tag space and then user׳s latent preference will be uncovered by those extracted features. Motivated by this, we propose a recommendation algorithm based on deep neural networks. In the proposed algorithm, a deep neural network model is first used to extract more abstract features form the tag information. Then, we adopt user-based CF to generate recommendations based on these features. In this paper, the sparse autoencoder [29] is selected as the deep neural network model. For convenience, the proposed algorithm is denoted as CFA.

The main contribution of this paper is to exploit a deep neural network for tackling the tag information so as to improve the recommendation performance. The advantages of using the deep neural network are as follows:

  • Through the deep neural network, the latent features, which are much denser, can be extracted layer by layer. Then users׳ profiles are updated with those extracted features. Therefore, using the deep neural network can overcome the difficulty of data sparsity.

  • The number of neurons in the hidden layers is much smaller than that in the input layer. Accordingly, the dimensionality of the data in the hidden layers is lower than that of the input data. In other words, using the deep neural network can reduce the dimensionality of data.

  • By using the deep neural network, we can learn more abstract and representative latent features of varied tag information. Hence the redundancy and ambiguity of tag information can be diminished in some extent.

The proposed algorithm is tested on two real datasets. Experimental results demonstrate that the deep neural network can improve the recommendation performance of traditional CF effectively. Comparison results also show that the proposed algorithm outperforms the clustering-based CF approaches. In addition, the impact of network depth in the sparse autoencoder is analyzed.

The remainder of this paper is organized as follows. Section 2 reviews some background, including some typical tag-aware recommendation models and related work on tag-aware recommendations. In Section 3, the proposed recommendation algorithm based on deep neural networks is described in detail. Section 4 reports the experimental results. Finally, conclusions are given in Section 5.

Section snippets

Tag-aware recommendation models

Formally, a social tagging system typically consists three communities: users, items and tags. A folksonomy in tag-aware recommender systems is usually modeled as a tuple [13]:F=(U,I,T,Y),where U, I, T are finite sets and represent users set, items set and tags set, respectively. Y indicates the ternary relations among users, items, and tags and is denoted by a 3-order tensor (3-dimensional array):Y=(yu,i,t)R|U|×|I|×|T|where yu,i,t=1 if user u has assigned tag t to item i, and yu,i,t=0

Deep neural networks based tag-aware recommendation algorithm

In this section, we will describe the proposed recommendation algorithm based on deep neural networks in detail. More specifically, the proposed algorithm consists three main procedures. At first, the users׳ profiles are modeled as vectors over tags. Then, a deep neural network is used to discover latent features from the users׳ tag space. Finally, aggregate the extracted features and items information to generate recommendations. The illustration of the proposed algorithm is depicted in Fig. 1

Test datasets and parameter settings

In our experiments, we use two real website datasets, including Last.fm and Del.icio.us, which are released in the framework of the 2nd International Workshop on Information Heterogeneity and Fusion in Recommender Systems to make an evaluation [49]. The first dataset is obtained from Last.fm online music system,4 which allows users to tag music tracks and artists. In this dataset, the users are interconnected in a social network generated from Last.fm “friend” relations

Conclusions

Taking advantage of powerful abilities of learning representations in deep neural networks, we have proposed a new tag-aware recommendation algorithm. In the proposed algorithm, a deep neural network model, namely the sparse autoencoder, is used to discover the in-depth features of tag space. Instead of the raw data, the extracted features, which become more abstract, representative, and dense, are used for user-based CF to make recommendations. The performance of the proposed algorithm has

Acknowledgments

This work was supported by the National Natural Science Foundation of China (Grant nos. 61273317, 61422209, 61473215), the National Program for Support of Top-notch Young Professionals of China and the Fundamental Research Fund for the Central Universities (Grant no. K5051202053).

Yi Zuo received the B.S. degree in electronic information engineering from Xidian University, Xi׳an, China, in 2008. He is currently working toward the Ph.D. degree in Pattern Recognition and Intelligent Systems at Xidian University, Xi׳an, China. His research interests include computational intelligence and recommender systems.

References (55)

  • L. Sharma et al.

    A survey of recommendation systemresearch challenges

    Int. J. Eng. Trends Technol. (IJETT)

    (2013)
  • G. Linden et al.

    Amazon. com recommendationsitem-to-item collaborative filtering

    IEEE Internet Comput.

    (2003)
  • K. Ali, W. Van Stam, TiVo: making show recommendations using a distributed collaborative filtering architecture, in:...
  • J. Bennett, S. Lanning, The netflix prize, in: Proceedings of KDD Cup and Workshop, vol. 2007, 2007, p....
  • J.L. Herlocker et al.

    Evaluating collaborative filtering recommender systems

    ACM Trans. Inf. Syst. (TOIS)

    (2004)
  • D. Goldberg et al.

    Using collaborative filtering to weave an information tapestry

    Commun. ACM

    (1992)
  • M.D. Ekstrand et al.

    Collaborative filtering recommender systems

    Found. Trends Hum.-Comput. Interact.

    (2011)
  • Y. Shi et al.

    Collaborative filtering beyond the user-item matrixa survey of the state of the art and future challenges

    ACM Comput. Surv. (CSUR)

    (2014)
  • J.S. Breese, D. Heckerman, C. Kadie, Empirical analysis of predictive algorithms for collaborative filtering, in:...
  • Y. Koren et al.

    Matrix factorization techniques for recommender systems

    Computer

    (2009)
  • X. Su et al.

    A survey of collaborative filtering techniques

    Adv. Artif. Intell.

    (2009)
  • E.H. Chi, T. Mytkowicz, Understanding the efficiency of social tagging systems using information theory, in:Proceedings...
  • A. Hotho et al.

    Information Retrieval in Folksonomies: Search and Ranking

    (2006)
  • F. Sun et al.

    Towards tags ranking for social images

    Neurocomputing

    (2013)
  • S.-L. Huang et al.

    Investigating effectiveness and user acceptance of semantic social tagging for knowledge sharing

    Inf. Process. Manag.

    (2012)
  • Z.K. Zhang et al.

    Solving the cold-start problem in recommender systems with social tags

    Europhys. Lett. (EPL)

    (2010)
  • K. Bischoff, C.S. Firan, W. Nejdl, R. Paiu, Can all tags be used for search? in: Proceedings of the 17th ACM Conference...
  • Z.-K. Zhang et al.

    Tag-aware recommender systemsa state-of-the-art survey

    J. Comput. Sci. Technol.

    (2011)
  • A. Shepitsen, J. Gemmell, B. Mobasher, R. Burke, Personalized recommendation in social tagging systems using...
  • Y. Bengio et al.

    Representation learninga review and new perspectives

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2013)
  • J. Gan et al.

    Deep self-taught learning for facial beauty prediction

    Neurocomputing

    (2014)
  • Y. Bengio et al.

    Greedy layer-wise training of deep networks

    Adv. Neural Inf. Process. Syst.

    (2007)
  • B. Leng et al.

    A 3D model recognition mechanism based on deep Boltzmann machines

    Neurocomputing

    (2015)
  • R. Collobert et al.

    Natural language processing (almost) from scratch

    J. Mach. Learn. Res.

    (2011)
  • S.M. Siniscalchi et al.

    Exploiting deep neural networks for detection-based speech recognition

    Neurocomputing

    (2013)
  • S.M. Siniscalchi et al.

    An artificial neural network approach to automatic speech processing

    Neurocomputing

    (2014)
  • P. Hamel, D. Eck, Learning features from music audio with deep belief networks, in: The International Society for Music...
  • Cited by (113)

    View all citing articles on Scopus

    Yi Zuo received the B.S. degree in electronic information engineering from Xidian University, Xi׳an, China, in 2008. He is currently working toward the Ph.D. degree in Pattern Recognition and Intelligent Systems at Xidian University, Xi׳an, China. His research interests include computational intelligence and recommender systems.

    Jiulin Zeng received the B.S. degree in electronic information engineering from Southwest University, Chongqing, China, in 2013. Now he is working toward the M.S. degree at Xidian University, Xi׳an, China. His current research interests include recommender systems and deep neural networks.

    Maoguo Gong is currently a full professor with Xidian University, Xi׳an, China. He received the B. Eng degree in electronic engineering, and the Ph.D. degree from Xidian University, in 2003 and 2009, respectively. Since 2006, he has been a lecturer of Xidian University. He was promoted to associate professor and full professor in 2008 and 2010, respectively, both with exceptive admission. In 2012, he was appointed as the chief professor (Director) of the OMEGA Innovative Research Team of Shaanxi Province, China.

    Gong׳s research interests are broadly in the area of computational intelligence, with applications to optimization, learning, data mining and image understanding. He has published over 50 papers in journals and conferences, and holds 12 granted patents as the first inventor. He is leading or has completed 10 projects as the PI, funded by the National Natural Science Foundation of China, the National High Technology Research and Development Program of China and others. He was the recipient of the prestigious National Top Young Talent of China (selected by the Central Organization Department of China), the excellent young scientist foundation (selected by the National Natural Science Foundation of China), the New Century Excellent Talent in University (selected by the Ministry of Education of China), the Young Teacher Award by the Fok Ying Tung Education Foundation, the Elsevier SCOPUS Young Researcher Award of China, and the National Natural Science Award of China.

    He is the vice-chair of IEEE CIS Task Force on Memetic Computing, executive committee member of Chinese Association for Artificial Intelligence, senior member of IEEE and Chinese Computer Federation, editorial board member for five journals, reviewer for over 10 journals.

    Licheng Jiao received the B.S. degree from Shanghai Jiaotong University, Shanghai, China, in 1982, the M.S. and Ph.D. degrees from Xi׳an Jiaotong University, Xi׳an, China, in 1984 and 1990, respectively. Since 1992, Jiao has been a Professor in the School of Electronic Engineering at Xidian University, Xi׳an, China. His research interests include image processing, natural computation, machine learning, and intelligent information processing.

    Jiao has charged of about 40 important scientific research projects, and published more than 20 monographs and a hundred papers in international journals and conferences.

    View full text