Elsevier

Neurocomputing

Volume 385, 14 April 2020, Pages 1-12
Neurocomputing

TNAM: A tag-aware neural attention model for Top-N recommendation

https://doi.org/10.1016/j.neucom.2019.11.095Get rights and content

Abstract

Recent work shows that incorporating tag information to recommender systems is promising for improving the recommendation accuracy in social systems. However, existing approaches suffer from less reasonable assignment of tag weights when constructing the user profiles and item characteristics in real-world scenarios, resulting in decreased accuracy in making recommendations. The above issue is specifically summarized into two aspects: 1) the weight of a target item is mainly determined by number of one certain type of tags, and 2) users place equal focus on the same tag for different items. To tackle these problems, we propose a novel model named TNAM, a Tag-aware Neural Attention Model, which accurately captures users’ special attention to tags of items. In the proposed model, we design a tag-based neural attention network by extracting potential tag information to overcome the difficulty of assigning tag weights for personalized users. We combine user-item interactions with tag information to map sparse data to dense vectors in higher-order space. In this way, TNAM acquires more interrelations between users and items to make recommendations more accurate. Extensive experiments of our model on three publicly implicit feedback datasets reveal significant improvements on the metrics of HR and NDCG in Top-N recommendation tasks over several state-of-the-art approaches.

Introduction

Personalized recommendation is a hot topic in recent years [1], [2], [3], and a great variety of recommendation methods have been proposed. These methods are divided into two mainstreams, including neighborhood-based and model-based methods. Collaborative Filtering technology (CF) [4] is the most popular solution in neighborhood-based methods. The core idea of CF is that users who have similar interests in the past will usually share common interests in the future [5]. The model-based methods, like matrix factorization [6], [7], diffusion-based model [8], tensor factorization [9] and random-walk-based algorithm [10], are based on the assumption that data can be described by an underlying model or a specific function to make prediction. At the same time, in order to generate highly recommendation predictions or reliable top-N list, other approaches based on latent factor model (LFM) are proposed to predict the miss data through inherent factors between users and items [11], [12], [13]. In recent years, researchers begin to incorporate side information into the recommender system to advance the performance [14], [15]. For example, social tag systems [16] introduce tags which provide valuable supplementary information by summarizing items properties and reflecting user preferences. The social tag recommendation systems have been successfully applied to many datasets of various fields, such as Delicious1 (a social bookmarking system), LastFM2 (a music website) MovieLens3 (a personalized movie recommendation platform), Weibo4 (a news website) and Amazon5 (a E-commerce site), leading to a significant improvement in accuracy and efficiency.

Social tag systems allow users to freely mark their own tags to be more effective and practical [17]. In this way, tags are regarded as the bridge to establish the implicit relationship between users and items by assigning various personalized tags. In addition, tag information can be used to solve the problem of cold start [18]. However, new problems inevitably arise. These problems include uncontrolled vocabularies, since potential relationships of tags are not reasonably considered, resulting in sparsity, redundancy and ambiguity in tag latent space [19], [20]. To solve the problem of potential relationships being ignored, clustering-based method [1] is introduced to the social tag recommender systems. The core of clustering strategy is to aggregate tags, thus reducing the ambiguity by class in tag latent space instead of finding a single tag. However, computing the similarity of tags is a time-consuming work, since it needs to not only format tags, but also to discover the potential meaning of tags. More importantly, the simple clustering does not provide high-precision recommendation results to meet personalized recommendation demands. Therefore, it is a key point to consider about how to mine the potential relationship of tags.

Currently, deep learning has been widely adopted in the research of the tag-aware recommender systems [21], as it performs well in the feature extraction and function fitting. In ACF [19], the users’ profiles are initially represented by tags, and a deep neural network model is then used to extract the in-depth features from tag latent space layer by layer. Finally, the extracted features are used by user-based collaborative filtering to make recommendations. Although the performance of ACF [19] outperforms the clustering-based collaborative filtering method [1], this method still have the following drawback: no direct correlation between the model learning signal and the objective of personalized recommendation is established. In order to solve the above drawback, Xu et al. proposed DSPR [20], which mapped user profiles and item characteristics to a deep feature space by using the deep-semantic similarity-based neural network. In recent works, researchers focus on associating tag information with natural language processing technique based on deep learning, Liang et al. [22] used pre-trained word embeddings to represent user-defined tags, extracting the latent features of items and users. Moreover, in order to alleviate the problem of high dimension and sparsity of tag information, Li et al. [23] proposed a novel optimization criterion based on Bayesian personalized ranking to serve as the regularization constraint of latent features derived from implicit feedbacks.

Although previous efforts have achieved some success [1], [19], [20], they still suffer from less reasonable assignments of the tag weight. These works either assign weights according to the number of one type of tags, or assign the same weight to all users. In fact, different users often paid different attentions to the same tag for each item. And we list two challenges to clarify this point:

  • 1)

    The number of one type of tags affect the accuracy of recommendations: we select a small number of user tags and item tags from the MovieLens dataset and visualize them as shown in Fig. 1(a)(b). We notice that although tag Drama occurs frequently, it cannot reflect the real preferences of users. Because Drama is a common attribute that many films contain, and the number of tag Drama can not accurately characterize every film. So it is inappropriate to recommend the Crying Game which has fewer tags of comedy and romance to user #12485. Analogously, if movie for the moment is recommended to user #10077 due to tag War, user #10077 may not satisfy with movie for the moment which involves too many tags of Romance.

  • 2)

    The same tag weight results in low-precision personalized recommendation: as shown in Fig. 1(c), previous works generally regard the tags of War, Comedy and Romance with the same weight in the tag latent space. And the latent vector Vexp generated by the above three tags will be used for the next user preference calculation. However, this vector does not reveal the real preferences of the user, since the user will demonstrate different interests among different kinds of movies. For examples, if one user prefers war plots, the vector Vreal will outperform Vexp in representing his/her real preferences.

To solve the problems above, we propose TNAM: a tag-aware neural attention model, which accurately captures different users’ attentions to the same tag for each item. Firstly, focusing on challenge 1, we propose the tag embedding technique which takes both user-item interactions and tag information into account to eliminate the noise of universal tags. As for challenge 2, we design a tag-based neural attention network. It assigns different weights to the same tag of an item for different users. However, if noises or weak features are input into the neural attention network, its power will decline. In order to reduce the effect of noises and extract highly abstract features, we first input compressed vectors from the stacked autoencoder [19], [20], [24] and the intersection of tag vectors. We then introduce a generic Neural Collaborative Filter (NCF) [11] approache to build a neural computing architecture. Finally, a sigmoid function is used to convert the probability output to a binary classification result, which predicts whether the user will like the recommended items. Limited by the number of samples [11], [25], it is impossible to regard the remaining non-interactive samples as negative samples for each positive sample. Therefore, we randomly sample a small number of irrelevant items from unobserved items at a controlled negative sampling rate. These sampled items are regarded as negative samples in each training run, leading to a better performance of item recommendation [20], [26].

To our best knowledge, this is the first Top-N recommendation work that applies neural attention networks to tag-aware recommender systems. The main contributions of this work goes as follows.

  • We propose a general tag-aware architecture which combines tag information with user-item interactions for recommendation to extract user preferences and item characteristics on different aspects.

  • We design a tag-based neural attention network, which extracts potential relationships from tags by using the stacked autoencoder and the interaction of tag vectors.

  • We perform comprehensive experiments on three publicly accessible datasets to demonstrate the effectiveness of our proposed model. The experiment results also show a great improvement brought by the tag-based neural attention network.

The rest of this paper has been briefly organized in the following way. Section 2 gives a brief overview of the related work. Section 3 describes our tag-aware neural attention model in detail. Section 4 reports the experiment results on three real datasets. Finally, our conclusion and future work of this research are presented in Section 5.

Section snippets

Related work

Tag-based Collaborative Filtering (TCF) has been the most common recommendation method for past several years [27]. Since then many improved methods based on TCF have been proposed for better recommendations [17]. Ricci et al.[28] proposed a method with the purpose of recommending items by the relationship among users, tags and items. The core idea of this method is to calculate a series of tag weights with a user-tag projection matrix, and weight candidate tags on items to extract the

Preliminaries

The objective of the Top-N recommendation is to recommend a list of items that the user might be interested in through the implicit feedback information. In general, three components are typically contained in social systems: users, items and tags. A folksonomy [16], [38] is a tuple:F:=(U,I,T,Y), where U, I, T indicate users set, items set and tags set, respectively. Y is the internal relations among them. For the convenience of representation, we often regard Y as a 3-order tensor:Y=y(u,i,t)RM

Experiments

In this section, we first introduce the datasets, evaluation protocols and baselines, and then describe the experiment setup and parameter settings. Finally, we conduct plenty of experiments on three real-world publicly accessible datasets to answer the following three questions, aiming at certifying the effectiveness of our proposed methods:

  • RQ1

    Does our proposed TNAM significantly outperform the state-of-the-art baselines in tag-aware Top-N recommendation methods?

  • RQ2

    How do the key hyper-parameters

Conclusion

In this paper, we proposed a tag-aware neural attention model, which greatly improves the accuracy of tag-aware recommendation by addressing assignment of the tag weights when constructing the user profiles and item characteristics. In our proposed model, tag embedding technique is developed to extract user preferences and item characteristics on different aspects. Moreover, a tag neural attention network is designed in our model to obtain abundant tag semantic information introducing the

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

The paper is supported by the National Natural Science Foundation of China (NSFC) under Grant No. 61672498 and the National Key Research and Development Program of China under Grant No. 2016YFC0302300.

Ruoran Huang received the B.S. in software engineering from Yanshan University in 2016. He is currently working toward the Ph.D. degree in computer system structure at Institute of Computing Technology, University of Chinese Academy of Sciences, Beijing, China. His research interests include data mining, artificial intelligence and cloud computing.

References (53)

  • I. Bayer et al.

    A generic coordinate descent framework for learning from implicit feedback

    Proceedings of the 26th International Conference on World Wide Web

    (2017)
  • X. Luo et al.

    An effective scheme for QOS estimation via alternating direction method-based matrix factorization

    IEEE Trans. Serv. Comput.

    (2016)
  • X. Luo et al.

    Temporal pattern-aware QOS prediction via biased non-negative latent factorization of tensors

    IEEE Trans. Cybern.

    (2019)
  • X. He et al.

    Neural collaborative filtering

    Proceedings of the 26th International Conference on World Wide Web

    (2017)
  • X. Luo et al.

    Generating highly accurate predictions for missing qos data via aggregating nonnegative latent factor models

    IEEE Trans. Neural Netw. Learn. Syst.

    (2015)
  • X. Luo et al.

    Incorporation of efficient second-order solvers into latent factor models for accurate prediction of missing QOS data

    IEEE Trans. Cybern.

    (2017)
  • D. Deng et al.

    Neural gaussian mixture model for review-based rating prediction

    Proceedings of the 12th ACM Conference on Recommender Systems

    (2018)
  • Y. Deldjoo et al.

    Audio-visual encoding of multimedia content for enhancing movie recommendations

  • A. Hotho et al.

    Information retrieval in folksonomies: Search and ranking

    Proceedings of the ESWC

    (2006)
  • L.B. Marinho et al.

    Collaborative Tag Recommendations

    (2008)
  • Z. Zhang et al.

    Solving the cold-start problem in recommender systems with social tags

    EPL

    (2010)
  • Z. Xu et al.

    Tag-aware personalized recommendation using a hybrid deep model

    Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17

    (2017)
  • Z.-K. Zhang et al.

    Tag-aware recommender systems: a state-of-the-art survey

    J. Comput. Sci. Technol.

    (2011)
  • N. Liang et al.

    TRSDL: tag-aware recommender system based on deep learning–intelligent computing systems

    Appl. Sci.

    (2018)
  • H. Li et al.

    Tag-aware recommendation based on bayesian personalized ranking and feature mapping

    Intell. Data Anal.

    (2019)
  • Y. Ouyang et al.

    Autoencoder-based collaborative filtering

    Proceedings of the International Conference on Neural Information Processing

    (2014)
  • Cited by (24)

    • TKGAT: Graph attention network for knowledge-enhanced tag-aware recommendation system

      2022, Knowledge-Based Systems
      Citation Excerpt :

      HTRM [17] uses LSTM to learn from user tagging behaviors. TNAM [18] introduces attention mechanisms into the model in order to analyze the information between user-related tags and item-related tags. TGCN [19] uses GAT [20] on a collaborative tag graph and a prediction layer to learn the embedding vectors end-to-end.

    • Entity knowledge transfer-oriented dual-target cross-domain recommendations

      2022, Expert Systems with Applications
      Citation Excerpt :

      The experimental results show that the proposed model can consistently perform better than the benchmark recommendation models. Furthermore, some neural attention recommendations (Huang et al, 2020; Li et al., 2021; Pradhan et al., 2020) have also been designed to improve the rating prediction accuracy of recommendation systems. In addition, some variant models based on the attention mechanism have also been explored to obtain the competitive recommendation results.

    • A deep learning based trust- and tag-aware recommender system

      2022, Neurocomputing
      Citation Excerpt :

      Moreover, the adversarial learning model is utilized to predict tags for users according to the previous user-generated tags in the system. In [59], a tag-based recommendation approach is developed based on a neural attention network to learn hidden tag information and determine the weights of tags for users. In addition, the data sparsity issue is addressed by integrating tag data and user-item interactions leading to an enhancement in the accuracy of recommendations.

    View all citing articles on Scopus

    Ruoran Huang received the B.S. in software engineering from Yanshan University in 2016. He is currently working toward the Ph.D. degree in computer system structure at Institute of Computing Technology, University of Chinese Academy of Sciences, Beijing, China. His research interests include data mining, artificial intelligence and cloud computing.

    Nian Wang is currently working toward the Ph.D. degree at Institute of Computing Technology, University of Chinese Academy of Sciences, Beijing, China. His current research interests include Internet of Things, Artificial Intelligence of Things and Cyber-Physical System.

    Chuanqi Han received the B.S. in aircraft design engineering from Northwestern Polytechnical University, Xi’an in 2014, and M.S. degree in computer application technology from University of Chinese Academy of Sciences, Beijing, in 2018. He is currently working toward the Ph.D. degree in computer system structure at Institute of Computing Technology, University of Chinese Academy of Sciences, Beijing, China. His research interests include artificial intelligence and data mining.

    Fang Yu is currently working toward the Ph.D. degree at Institute of Computing Technology, University of Chinese Academy of Sciences, Beijing, China. His research focuses on machine learning and computer vision, in particular deep learning, resource efficient learning, CNN efficient inference.

    Li Cui received the BS degrees from Tsinghua University, Beijing, China, in 1985, the M.S. degree from the Institute of Semiconductors of the Chinese Academy of Sciences, Beijing, China in 1988, and the PhD degree from the University of Glasgow, UK in 1999. She is currently a researcher of the Pervasive Computing Research Center, Institute of Computing Technology (ICT), Chinese Academy of Sciences (CAS). Her research interests include artificial intelligence, pervasive computing, wireless sensor network and sensor technology.

    View full text