Tag-aware recommender systems based on deep neural networks
Introduction
With the surge of Internet users and tremendous development of individual communication terminals, everyday user-generated data makes people have to face huge information, from which, finding out relevant items for a given user is a big problem to be solved urgently. Therefore, as a useful tool to filter out irrelevant information, recommender systems have attracted increasing attention in recent years [1]. Adopting knowledge discovery technology, recommender systems can predict users׳ future preferences and behaviors according to history records. Thus far, recommendation systems have successfully found applications in diverse fields, such as book recommendations in Amazon.com [2], video recommendations in TiVo.com [3], and movie recommendations in Netflix.com [4].
Many different prediction techniques have been proposed to generate personalized recommendations. Due to its simplicity and promising results, Collaborative Filtering (CF) has been one of the most successful and widely used methods in recommender systems [5]. The core assumption of CF is that users who have expressed similar interests in the past will share common interests in the future [6]. As described in [7], CF can be classified into two major types: memory-based CF and model-based CF. According to the exploited information, memory-based CF is categorized as user based or item based CF. In user-based CF, a user׳s rating on a target item is based on the ratings that several similar users have given to that item in the past [8]. In contrast to user-based CF, item-based CF predicts the rating of the target item based on the ratings that the user has given to other similar items [8]. Model-based CF uses the user-item matrix to train prediction models, such as Bayesian network model [9] and matrix factorization [10]. However, in real-life recommender systems, one user will only rate quite a small fraction of all items and one item is usually rated by a small proportion of all users. Hence the rating matrix is so sparse that it is difficult to find similar users or items because of lacking information [11].
To obtain more useful information about users, many websites introduce a kind of web-based systems, known as social tagging systems [12]. The systems allow users to freely label items with arbitrary words [13], namely user-defined tags. By introducing those tags, recommender systems with tag information (tag-aware recommender systems) [14], [15] are more useful and applied to lots of online sites, such as Del.icio.us1 (with tags of bookmark), Flickr2(with tags of images) and Last.Fm3 (with tags of music). User-defined tags can reflect both users׳ preferences and evaluations on items. From this perspective, tags can be seen as bridges between users and items. Especially, different users usually assign different tags even to the same items, which may benefit personalized recommendations a lot. In addition, tag information can help solve the cold-start problem in recommender systems [16].
Nevertheless, not all the tags benefit recommendations [17]. Arbitrarily associated by users, tags may form an uncontrolled vocabulary, which usually leads to two common situations [18]: (i) similar meanings are expressed with different words, which will be treated as different tags in tagging systems, such as “bike” and “bicycle”; (ii) polysemy, for example, “Hummer” may indicate a buzzer, while it can also represent the brand of a famous car from General Motors Corporation. These issues often cause information redundant and ambiguous and prevent from uncovering the underlying structure or relations among tags. To solve those issues, researchers usually apply clustering-based methods to recommender systems. By clustering [19], redundant tags will be aggregated into the same cluster. Discovering the feature of a cluster is more easily than discovering that of a single tag. In addition, the meaning of a tag is shared by the other tags in the same cluster, so the ambiguity can be alleviated at the same time. However, in tag clustering methods, computing tags׳ similarities accurately is an essential precondition, but intractable due to data sparsity. In addition, tag features need to be represented by extra data, such as the information of items, leading to higher complexity. Hence, it is necessary to find a more straightforward and effective method to address the tag information.
Because of its excellent performance on extracting effective representations of raw data [20], deep neural networks have been applied in many fields, such as image classification [21], [22], [23] and natural language processing [24], [25], [26]. Especially, some researchers have tried to exploit deep neural networks to improve the performance of traditional recommendation algorithms recently. For example, in [27], deep belief network (DBN) was used to music recommendation by learning features from audio data. Similarly, convolutional neural network (CNN) was introduced to music recommendation in [28].
As described in [20], deep architectures are more abstract representations and generally invariant to most local changes of the input data. Thus the extracted abstract and invariant features can detect categories in more varied phenomena and potentially have greater predictive power. In social tagging systems, although tags given by a user often change, the user׳s preference is usually invariant. Accordingly, with the help of deep neural networks, it is possible to learn more abstract features from user׳s tag space and then user׳s latent preference will be uncovered by those extracted features. Motivated by this, we propose a recommendation algorithm based on deep neural networks. In the proposed algorithm, a deep neural network model is first used to extract more abstract features form the tag information. Then, we adopt user-based CF to generate recommendations based on these features. In this paper, the sparse autoencoder [29] is selected as the deep neural network model. For convenience, the proposed algorithm is denoted as CFA.
The main contribution of this paper is to exploit a deep neural network for tackling the tag information so as to improve the recommendation performance. The advantages of using the deep neural network are as follows:
- •
Through the deep neural network, the latent features, which are much denser, can be extracted layer by layer. Then users׳ profiles are updated with those extracted features. Therefore, using the deep neural network can overcome the difficulty of data sparsity.
- •
The number of neurons in the hidden layers is much smaller than that in the input layer. Accordingly, the dimensionality of the data in the hidden layers is lower than that of the input data. In other words, using the deep neural network can reduce the dimensionality of data.
- •
By using the deep neural network, we can learn more abstract and representative latent features of varied tag information. Hence the redundancy and ambiguity of tag information can be diminished in some extent.
The proposed algorithm is tested on two real datasets. Experimental results demonstrate that the deep neural network can improve the recommendation performance of traditional CF effectively. Comparison results also show that the proposed algorithm outperforms the clustering-based CF approaches. In addition, the impact of network depth in the sparse autoencoder is analyzed.
The remainder of this paper is organized as follows. Section 2 reviews some background, including some typical tag-aware recommendation models and related work on tag-aware recommendations. In Section 3, the proposed recommendation algorithm based on deep neural networks is described in detail. Section 4 reports the experimental results. Finally, conclusions are given in Section 5.
Section snippets
Tag-aware recommendation models
Formally, a social tagging system typically consists three communities: users, items and tags. A folksonomy in tag-aware recommender systems is usually modeled as a tuple [13]:where U, I, T are finite sets and represent users set, items set and tags set, respectively. Y indicates the ternary relations among users, items, and tags and is denoted by a 3-order tensor (3-dimensional array):where if user u has assigned tag t to item i, and
Deep neural networks based tag-aware recommendation algorithm
In this section, we will describe the proposed recommendation algorithm based on deep neural networks in detail. More specifically, the proposed algorithm consists three main procedures. At first, the users׳ profiles are modeled as vectors over tags. Then, a deep neural network is used to discover latent features from the users׳ tag space. Finally, aggregate the extracted features and items information to generate recommendations. The illustration of the proposed algorithm is depicted in Fig. 1
Test datasets and parameter settings
In our experiments, we use two real website datasets, including Last.fm and Del.icio.us, which are released in the framework of the 2nd International Workshop on Information Heterogeneity and Fusion in Recommender Systems to make an evaluation [49]. The first dataset is obtained from Last.fm online music system,4 which allows users to tag music tracks and artists. In this dataset, the users are interconnected in a social network generated from Last.fm “friend” relations
Conclusions
Taking advantage of powerful abilities of learning representations in deep neural networks, we have proposed a new tag-aware recommendation algorithm. In the proposed algorithm, a deep neural network model, namely the sparse autoencoder, is used to discover the in-depth features of tag space. Instead of the raw data, the extracted features, which become more abstract, representative, and dense, are used for user-based CF to make recommendations. The performance of the proposed algorithm has
Acknowledgments
This work was supported by the National Natural Science Foundation of China (Grant nos. 61273317, 61422209, 61473215), the National Program for Support of Top-notch Young Professionals of China and the Fundamental Research Fund for the Central Universities (Grant no. K5051202053).
Yi Zuo received the B.S. degree in electronic information engineering from Xidian University, Xi׳an, China, in 2008. He is currently working toward the Ph.D. degree in Pattern Recognition and Intelligent Systems at Xidian University, Xi׳an, China. His research interests include computational intelligence and recommender systems.
References (55)
- et al.
A survey of recommendation systemresearch challenges
Int. J. Eng. Trends Technol. (IJETT)
(2013) - et al.
Amazon. com recommendationsitem-to-item collaborative filtering
IEEE Internet Comput.
(2003) - K. Ali, W. Van Stam, TiVo: making show recommendations using a distributed collaborative filtering architecture, in:...
- J. Bennett, S. Lanning, The netflix prize, in: Proceedings of KDD Cup and Workshop, vol. 2007, 2007, p....
- et al.
Evaluating collaborative filtering recommender systems
ACM Trans. Inf. Syst. (TOIS)
(2004) - et al.
Using collaborative filtering to weave an information tapestry
Commun. ACM
(1992) - et al.
Collaborative filtering recommender systems
Found. Trends Hum.-Comput. Interact.
(2011) - et al.
Collaborative filtering beyond the user-item matrixa survey of the state of the art and future challenges
ACM Comput. Surv. (CSUR)
(2014) - J.S. Breese, D. Heckerman, C. Kadie, Empirical analysis of predictive algorithms for collaborative filtering, in:...
- et al.
Matrix factorization techniques for recommender systems
Computer
(2009)
A survey of collaborative filtering techniques
Adv. Artif. Intell.
Information Retrieval in Folksonomies: Search and Ranking
Towards tags ranking for social images
Neurocomputing
Investigating effectiveness and user acceptance of semantic social tagging for knowledge sharing
Inf. Process. Manag.
Solving the cold-start problem in recommender systems with social tags
Europhys. Lett. (EPL)
Tag-aware recommender systemsa state-of-the-art survey
J. Comput. Sci. Technol.
Representation learninga review and new perspectives
IEEE Trans. Pattern Anal. Mach. Intell.
Deep self-taught learning for facial beauty prediction
Neurocomputing
Greedy layer-wise training of deep networks
Adv. Neural Inf. Process. Syst.
A 3D model recognition mechanism based on deep Boltzmann machines
Neurocomputing
Natural language processing (almost) from scratch
J. Mach. Learn. Res.
Exploiting deep neural networks for detection-based speech recognition
Neurocomputing
An artificial neural network approach to automatic speech processing
Neurocomputing
Cited by (113)
Knowledge graph-based recommendation system enhanced by neural collaborative filtering and knowledge graph embedding
2024, Ain Shams Engineering JournalA fairness-aware graph contrastive learning recommender framework for social tagging systems
2023, Information SciencesTKGAT: Graph attention network for knowledge-enhanced tag-aware recommendation system
2022, Knowledge-Based SystemsA deep learning based trust- and tag-aware recommender system
2022, NeurocomputingDeep learning based dual encoder retrieval model for citation recommendation
2022, Technological Forecasting and Social ChangeActive tag recommendation for interactive entity search: Interaction effectiveness and retrieval performance
2022, Information Processing and Management
Yi Zuo received the B.S. degree in electronic information engineering from Xidian University, Xi׳an, China, in 2008. He is currently working toward the Ph.D. degree in Pattern Recognition and Intelligent Systems at Xidian University, Xi׳an, China. His research interests include computational intelligence and recommender systems.
Jiulin Zeng received the B.S. degree in electronic information engineering from Southwest University, Chongqing, China, in 2013. Now he is working toward the M.S. degree at Xidian University, Xi׳an, China. His current research interests include recommender systems and deep neural networks.
Maoguo Gong is currently a full professor with Xidian University, Xi׳an, China. He received the B. Eng degree in electronic engineering, and the Ph.D. degree from Xidian University, in 2003 and 2009, respectively. Since 2006, he has been a lecturer of Xidian University. He was promoted to associate professor and full professor in 2008 and 2010, respectively, both with exceptive admission. In 2012, he was appointed as the chief professor (Director) of the OMEGA Innovative Research Team of Shaanxi Province, China.
Gong׳s research interests are broadly in the area of computational intelligence, with applications to optimization, learning, data mining and image understanding. He has published over 50 papers in journals and conferences, and holds 12 granted patents as the first inventor. He is leading or has completed 10 projects as the PI, funded by the National Natural Science Foundation of China, the National High Technology Research and Development Program of China and others. He was the recipient of the prestigious National Top Young Talent of China (selected by the Central Organization Department of China), the excellent young scientist foundation (selected by the National Natural Science Foundation of China), the New Century Excellent Talent in University (selected by the Ministry of Education of China), the Young Teacher Award by the Fok Ying Tung Education Foundation, the Elsevier SCOPUS Young Researcher Award of China, and the National Natural Science Award of China.
He is the vice-chair of IEEE CIS Task Force on Memetic Computing, executive committee member of Chinese Association for Artificial Intelligence, senior member of IEEE and Chinese Computer Federation, editorial board member for five journals, reviewer for over 10 journals.
Licheng Jiao received the B.S. degree from Shanghai Jiaotong University, Shanghai, China, in 1982, the M.S. and Ph.D. degrees from Xi׳an Jiaotong University, Xi׳an, China, in 1984 and 1990, respectively. Since 1992, Jiao has been a Professor in the School of Electronic Engineering at Xidian University, Xi׳an, China. His research interests include image processing, natural computation, machine learning, and intelligent information processing.
Jiao has charged of about 40 important scientific research projects, and published more than 20 monographs and a hundred papers in international journals and conferences.