research-article

Mention Recommendation in Twitter with Cooperative Multi-Agent Reinforcement Learning

Authors:

Xuanjing HuangAuthors Info & Claims

SIGIR'19: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 535 - 544

https://doi.org/10.1145/3331184.3331237

Published: 18 July 2019 Publication History

Abstract

In Twitter-like social networking services, the "@'' symbol can be used with the tweet to mention users whom the user wants to alert regarding the message. An automatic suggestion to the user of a small list of candidate names can improve communication efficiency. Previous work usually used several most recent tweets or randomly select historical tweets to make an inference about this preferred list of names. However, because there are too many historical tweets by users and a wide variety of content types, the use of several tweets cannot guarantee the desired results. In this work, we propose the use of a novel cooperative multi-agent approach to mention recommendation, which incorporates dozens of more historical tweets than earlier approaches. The proposed method can effectively select a small set of historical tweets and cooperatively extract relevant indicator tweets from both the user and mentioned users. Experimental results demonstrate that the proposed method outperforms state-of-the-art methods.

Supplementary Material

MP4 File (cite3-17h20-d2.mp4)

Download
507.96 MB

References

[1]

Kailong Chen, Tianqi Chen, Guoqing Zheng, Ou Jin, Enpeng Yao, and Yong Yu. 2012. Collaborative personalized tweet recommendation. In Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval. ACM, 661--670.

Digital Library

[2]

Zhuoye Ding, Xipeng Qiu, Qi Zhang, and Xuanjing Huang. 2013. Learning Topical Translation Model for Microblog Hashtag Suggestion. In IJCAI. 2078--2084.

Digital Library

[3]

Maxim Egorov. 2016. Multi-agent deep reinforcement learning.

[4]

Jun Feng, Heng Li, Minlie Huang, Shichen Liu, Wenwu Ou, Zhirong Wang, and Xiaoyan Zhu. 2018. Learning to Collaborate: Multi-Scenario Ranking via Multi-Agent Reinforcement Learning. In Proceedings of the 2018 World Wide Web Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 1939--1948.

Digital Library

[5]

Jakob Foerster, Gregory Farquhar, Triantafyllos Afouras, Nantas Nardelli, and Shimon Whiteson. 2017. Counterfactual multi-agent policy gradients. arXiv preprint arXiv:1705.08926 (2017).

[6]

Jakob Foerster, Nantas Nardelli, Gregory Farquhar, Triantafyllos Afouras, Philip H. S. Torr, Pushmeet Kohli, and Shimon Whiteson. 2017. Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning. In International Conference on Machine Learning. 1146--1155.

Digital Library

[7]

Yuyun Gong and Qi Zhang. 2016. Hashtag Recommendation Using Attention-Based Convolutional Neural Network. In IJCAI. 2782--2788.

Digital Library

[8]

Yeyun Gong, Qi Zhang, Xuyang Sun, and Xuanjing Huang. 2015. Who Will You@? In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. ACM, 533--542.

Digital Library

[9]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, Vol. 9, 8 (1997), 1735--1780.

Digital Library

[10]

Haoran Huang, Qi Zhang, Xuanjing Huang, et almbox. 2017. Mention recommendation for Twitter with end-to-end memory network. In Proc. IJCAI, Vol. 17. 1872--1878.

Digital Library

[11]

Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[12]

Vijay R. Konda and John N. Tsitsiklis. 2000. Actor-critic algorithms. In Advances in neural information processing systems. 1008--1014.

Digital Library

[13]

Ioannis Konstas, Vassilios Stathopoulos, and Joemon M. Jose. 2009. On social networks and collaborative recommendation. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval. ACM, 195--202.

Digital Library

[14]

Ralf Krestel, Peter Fankhauser, and Wolfgang Nejdl. 2009. Latent dirichlet allocation for tag recommendation. In Proceedings of the third ACM conference on Recommender systems. ACM, 61--68.

Digital Library

[15]

Quanle Li, Dandan Song, Lejian Liao, and Li Liu. 2015. Personalized mention probabilistic ranking-recommendation on mention behavior of heterogeneous social network. In International Conference on Web-Age Information Management. Springer, 41--52.

[16]

Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh. 2016. Hierarchical question-image co-attention for visual question answering. In NIPS. 289--297.

Digital Library

[17]

Hao Ma, Irwin King, and Michael R. Lyu. 2009. Learning to recommend with social trust ensemble. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval. ACM, 203--210.

Digital Library

[18]

Renfeng Ma, Qi Zhang, Jiawen Wang, Lizhen Cui, and Xuanjing Huang. 2018. Mention Recommendation for Multimodal Microblog with Cross-attention Memory Network. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval (SIGIR '18). ACM, New York, NY, USA, 195--204.

Digital Library

[19]

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013).

[20]

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, et al. 2015. Human-level control through deep reinforcement learning. Nature, Vol. 518, 7540 (2015), 529.

[21]

Seungwhan Moon, Leonardo Neves, and Vitor Carvalho. 2018. Multimodal named entity recognition for short social media posts. arXiv preprint arXiv:1802.07862 (2018).

[22]

Mor Naaman, Jeffrey Boase, and Chih-Hui Lai. 2010. Is it really about me? message content in social awareness streams. In Proceedings of the 2010 ACM conference on Computer supported cooperative work. ACM, 189--192.

Digital Library

[23]

Hyeonseob Nam, Jung-Woo Ha, and Jeonghee Kim. 2017. Dual Attention Networks for Multimodal Reasoning and Matching. In CVPR. 299--307.

[24]

Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011. Scikit-learn: Machine learning in Python. Journal of machine learning research, Vol. 12, Oct (2011), 2825--2830.

Digital Library

[25]

Soumajit Pramanik, Mohit Sharma, Maximilien Danisch, Qinna Wang, Jean-Loup Guillaume, and Bivas Mitra. 2018. Easy-Mention: a model-driven mention recommendation heuristic to boost your tweet popularity. International Journal of Data Science and Analytics (2018), 1--17.

[26]

Richard S. Sutton and Andrew G. Barto. 1998. Introduction to reinforcement learning. Vol. 135. MIT press Cambridge.

Digital Library

[27]

Yuri Takhteyev, Anatoliy Gruzd, and Barry Wellman. 2012. Geography of Twitter networks. Social networks, Vol. 34, 1 (2012), 73--81.

[28]

Ardi Tampuu, Tambet Matiisen, Dorian Kodelja, Ilya Kuzovkin, Kristjan Korjus, Juhan Aru, Jaan Aru, and Raul Vicente. 2017. Multiagent cooperation and competition with deep reinforcement learning. PloS one, Vol. 12, 4 (2017), e0172395.

[29]

Liyang Tang, Zhiwei Ni, Hui Xiong, and Hengshu Zhu. 2015. Locating targets through mention in Twitter. World Wide Web, Vol. 18, 4 (2015), 1019--1049.

Digital Library

[30]

Kagan Tumer and Adrian Agogino. 2007. Distributed agent-based air traffic flow management. In Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems. ACM, 255.

Digital Library

[31]

Beidou Wang, Can Wang, Jiajun Bu, Chun Chen, Wei Vivian Zhang, Deng Cai, and Xiaofei He. 2013. Whom to mention: expand the diffusion of tweets by@ recommendation on micro-blogging systems. In Proceedings of the 22nd international conference on World Wide Web. ACM, 1331--1340.

Digital Library

[32]

Kai Wang, Weiyi Meng, Jiang Bian, Shijun Li, and Sha Yang. 2018. Spatial context-aware user mention behavior modeling for mentionee recommendation. Neural Networks, Vol. 106 (2018), 152--167.

[33]

Kai Wang, Weiyi Meng, Shijun Li, and Sha Yang. 2019. Multi-Modal Mention Topic Model for mentionee recommendation. Neurocomputing, Vol. 325 (2019), 190--199.

[34]

Christopher J. C. H. Watkins and Peter Dayan. 1992. Q-learning. Machine learning, Vol. 8, 3--4 (1992), 279--292.

Digital Library

[35]

Ronald J. Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, Vol. 8, 3-4 (1992), 229--256.

Digital Library

[36]

Serena Yeung, Vignesh Ramanathan, Olga Russakovsky, Liyue Shen, Greg Mori, and Li Fei-Fei. 2017. Learning to Learn from Noisy Web Videos. In 2017 CVPR. IEEE, 7455--7463.

[37]

Dongfei Yu, Jianlong Fu, Tao Mei, and Yong Rui. 2017. Multi-level attention networks for visual question answering. In Computer Vision and Pattern Recognition (CVPR), 2017 IEEE Conference on. IEEE, 4187--4195.

[38]

Wei Zhang, Jianyong Wang, and Wei Feng. 2013. Combining latent factor model with location features for event-based group recommendation. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 910--918.

Digital Library

[39]

Ge Zhou, Lu Yu, Chu Xu Zhang, Chuang Liu, Zi Ke Zhang, and Jianlin Zhang. 2016. A Novel Approach for Generating Personalized Mention List on Micro-Blogging System. In IEEE International Conference on Data Mining Workshop.

Digital Library

Cited By

Mazhar TAsif RMalik MNadeem MHaq IIqbal MKamran MAshraf S(2023)Electric Vehicle Charging System in the Smart Grid Using Different Machine Learning MethodsSustainability10.3390/su1503260315:3(2603)Online publication date: 1-Feb-2023
https://doi.org/10.3390/su15032603
Jiang LLiu KWang YWang DWang PFu YYin M(2023)Reinforced Explainable Knowledge Concept Recommendation in MOOCsACM Transactions on Intelligent Systems and Technology10.1145/357999114:3(1-20)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1145/3579991
Du ZYang NYu ZYu P(2023)Learning From Atypical Behavior: Temporary Interest Aware Recommendation Based on Reinforcement LearningIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2022.314429235:10(9824-9835)Online publication date: 1-Oct-2023
https://doi.org/10.1109/TKDE.2022.3144292
Show More Cited By

Index Terms

Mention Recommendation in Twitter with Cooperative Multi-Agent Reinforcement Learning
1. Human-centered computing
  1. Collaborative and social computing
    1. Collaborative and social computing theory, concepts and paradigms
      1. Social recommendation
2. Information systems
  1. Information retrieval
    1. Users and interactive retrieval
      1. Collaborative search

Recommendations

Mention Recommendation for Multimodal Microblog with Cross-attention Memory Network
SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval

The users of Twitter-like social media normally use the "@'' sign to select a suitable person to mention. It is a significant role in promoting the user experience and information propagation. To help users easily find the usernames they want to mention,...
Whom to mention: expand the diffusion of tweets by @ recommendation on micro-blogging systems
WWW '13: Proceedings of the 22nd international conference on World Wide Web

Nowadays, micro-blogging systems like Twitter have become one of the most important ways for information sharing. In Twitter, a user posts a message (tweet) and the others can forward the message (retweet). Mention is a new feature in micro-blogging ...
Personalized News Recommendation Using Twitter
WI-IAT '13: Proceedings of the 2013 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT) - Volume 03

Online news reading has become a popular way to read news articles from a huge collection of news sources around the globe. News recommender systems help users manage this flood by suggesting articles based on user interests rather than presenting ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR'19: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2019

1512 pages

ISBN:9781450361729

DOI:10.1145/3331184

General Chairs:
Benjamin Piwowarski
CNRS - Sorbonne Universite, France
,
Max Chevalier
Universite de Toulouse, CNRS, France
,
Eric Gaussier
Universite Grenoble Alpes, CNRS, France
,
Program Chairs:
Yoelle Maarek
Amazon Research, Israel
,
Jian-Yun Nie
University of Montreal, Canada
,
Falk Scholer
RMIT University, Australia

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 July 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Shanghai Municipal Science and Technology Major Project
National Natural Science Foundation of China
STCSM
ZJLab
China National Key R&D Program
China National Key R&D

Conference

SIGIR '19

Sponsor:

SIGIR

SIGIR '19: The 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval

July 21 - 25, 2019

Paris, France

Acceptance Rates

SIGIR'19 Paper Acceptance Rate 84 of 426 submissions, 20%;

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
753
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)3

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Mazhar TAsif RMalik MNadeem MHaq IIqbal MKamran MAshraf S(2023)Electric Vehicle Charging System in the Smart Grid Using Different Machine Learning MethodsSustainability10.3390/su1503260315:3(2603)Online publication date: 1-Feb-2023
https://doi.org/10.3390/su15032603
Jiang LLiu KWang YWang DWang PFu YYin M(2023)Reinforced Explainable Knowledge Concept Recommendation in MOOCsACM Transactions on Intelligent Systems and Technology10.1145/357999114:3(1-20)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1145/3579991
Du ZYang NYu ZYu P(2023)Learning From Atypical Behavior: Temporary Interest Aware Recommendation Based on Reinforcement LearningIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2022.314429235:10(9824-9835)Online publication date: 1-Oct-2023
https://doi.org/10.1109/TKDE.2022.3144292
Zhao QLi TDu MJiang YSun QWang ZLiu HXu H(2023)UniMatch: A Unified User-Item Matching Framework for the Multi-purpose Merchant Marketing2023 IEEE 39th International Conference on Data Engineering (ICDE)10.1109/ICDE55515.2023.00254(3309-3321)Online publication date: Apr-2023
https://doi.org/10.1109/ICDE55515.2023.00254
Chen XYao LMcAuley JZhou GWang X(2023)Deep reinforcement learning in recommender systems: A survey and new perspectivesKnowledge-Based Systems10.1016/j.knosys.2023.110335264(110335)Online publication date: Mar-2023
https://doi.org/10.1016/j.knosys.2023.110335
Jiang YDi KHu ZChen FLi PJiang Y(2023)ε-Maximum Critic Deep Deterministic Policy Gradient for Multi-agent Reinforcement LearningParallel and Distributed Computing, Applications and Technologies10.1007/978-981-99-8211-0_17(180-189)Online publication date: 29-Nov-2023
https://doi.org/10.1007/978-981-99-8211-0_17
Afsar MCrump TFar B(2022)Reinforcement Learning based Recommender Systems: A SurveyACM Computing Surveys10.1145/354384655:7(1-38)Online publication date: 15-Dec-2022
https://dl.acm.org/doi/10.1145/3543846
Yang ZYe JWang LLin XHe L(2022)Inferring substitutable and complementary products with Knowledge-Aware Path Reasoning based on dynamic policy network▪Knowledge-Based Systems10.1016/j.knosys.2021.107579235:COnline publication date: 9-Apr-2022
https://dl.acm.org/doi/10.1016/j.knosys.2021.107579
Yi FJiang BWu J(2020)Heterogeneous Information Network Embedding for Mention RecommendationIEEE Access10.1109/ACCESS.2020.29943138(91394-91404)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.2994313

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten