Qi Zhang
ABSTRACT
On Twitter-like social media sites, the re-posting statuses or tweets of other users are usually considered to be the key mechanism for spreading information. How to predict whether a tweet will be retweeted by a user has received increasing attention in recent years. Previous methods studied the problem using various linguistic features, personal information of users, and many other manually constructed features to achieve the task. Usually, feature engineering is a laborious task, we require to obtain the external sources and they are difficult or not always available. Recently, deep learning methods have been used in the industry and research community for their ability to learn optimal features automatically and in many tasks, deep learning methods can achieve state-of-the art performance, such as natural language processing, computer vision, image classification and so on. In this work, we proposed a novel attention-based deep neural network to incorporate contextual and social information for this task. We used embeddings to represent the user, the user's attention interests, the author and tweet respectively. To train and evaluate the proposed methods, we also constructed a large dataset collected from Twitter. Experimental results showed that the proposed method could achieve better results than the previous state-of-the-art methods.
- O. Abdel-Hamid, L. Deng, and D. Yu. Exploring convolutional neural network structures and optimization techniques for speech recognition. In INTERSPEECH, pages 3366--3370, 2013.Google Scholar
- P. Achananuparp, E.-P. Lim, J. Jiang, and T.-A. Hoang. Who is retweeting the tweeters' modeling, originating, and promoting behaviors in the twitter network. TMIS, 3(3):13, 2012. Google ScholarDigital Library
- D. Bahdanau, J. Chorowski, D. Serdyuk, P. Brakel, and Y. Bengio. End-to-end attention-based large vocabulary speech recognition. arXiv preprint arXiv:1508.04395, 2015.Google Scholar
- B. Bi and J. Cho. Modeling a retweet network via an adaptive bayesian approach. In Proceedings of the 25th International Conference on World Wide Web, pages 459--469. International World Wide Web Conferences Steering Committee, 2016. Google ScholarDigital Library
- P. Blunsom, E. Grefenstette, N. Kalchbrenner, et al. A convolutional neural network for modelling sentences. In Proceedings of ACL, 2014.Google Scholar
- A. Bordes, J. Weston, and N. Usunier. Open question answering with weakly supervised embedding models. In Machine Learning and Knowledge Discovery in Databases, pages 165--180. Springer, 2014.Google ScholarDigital Library
- D. Boyd, S. Golder, and G. Lotan. Tweet, tweet, retweet: Conversational aspects of retweeting on twitter. In System Sciences (HICSS), 2010 43rd Hawaii International Conference on, pages 1--10. IEEE, 2010. Google ScholarDigital Library
- E. F. Can, H. Oktay, and R. Manmatha. Predicting retweet count using visual cues. In Proceedings of the 22nd ACM international conference on information & knowledge management, pages 1481--1484. ACM, 2013. Google ScholarDigital Library
- K. Chen, J. Wang, L.-C. Chen, H. Gao, W. Xu, and R. Nevatia. Abc-cnn: An attention based convolutional neural network for visual question answering. arXiv preprint arXiv:1511.05960, 2015.Google Scholar
- J. Chorowski, D. Bahdanau, D. Serdyuk, K. Cho, and Y. Bengio. Attention-based models for speech recognition. arXiv preprint arXiv:1506.07503, 2015.Google Scholar
- P. Cui, S. Jin, L. Yu, F. Wang, W. Zhu, and S. Yang. Cascading outbreak prediction in networks: a data-driven approach. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 901--909, 2013. Google ScholarDigital Library
- P. Cui, F. Wang, S. Liu, M. Ou, S. Yang, and L. Sun. Who should share what?: item-level social influence prediction for users and posts ranking. In Proceeding of the International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2011, Beijing, China, July, pages 185--194, 2011. Google ScholarDigital Library
- M. Denil, A. Demiraj, N. Kalchbrenner, P. Blunsom, and N. de Freitas. Modelling, visualising and summarising documents with a single convolutional neural network. arXiv preprint arXiv:1406.3830, 2014.Google Scholar
- A. Echihabi and D. Marcu. A noisy-channel approach to question answering. In Proceedings of the 41st Annual Meeting on Association for Computational Linguistics-Volume 1, pages 16--23. Association for Computational Linguistics, 2003. Google ScholarDigital Library
- W. Feng and J. Wang. Retweet or not?: personalized tweet re-ranking. In Proceedings of the sixth ACM international conference on Web search and data mining, pages 577--586. ACM, 2013. Google ScholarDigital Library
- J. Gao, L. Deng, M. Gamon, X. He, and P. Pantel. Modeling interestingness with deep neural networks, Dec. 17 2015. US Patent 20,150,363,688.Google Scholar
- K. Gregor, I. Danihelka, A. Graves, and D. Wierstra. Draw: A recurrent neural network for image generation. arXiv preprint arXiv:1502.04623, 2015.Google Scholar
- G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580, 2012.Google Scholar
- L. Hong, O. Dan, and B. D. Davison. Predicting popular messages in twitter. In Proceedings of the 20th international conference companion on World wide web, pages 57--58. ACM, 2011. Google ScholarDigital Library
- B. Hu, Z. Lu, H. Li, and Q. Chen. Convolutional neural network architectures for matching natural language sentences. In Advances in Neural Information Processing Systems, pages 2042--2050, 2014. Google ScholarDigital Library
- S. Ji, W. Xu, M. Yang, and K. Yu. 3d convolutional neural networks for human action recognition. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 35(1):221--231, 2013. Google ScholarDigital Library
- A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei. Large-scale video classification with convolutional neural networks. In CVPR, pages 1725--1732. IEEE, 2014. Google ScholarDigital Library
- Y. Kim. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882, 2014.Google Scholar
- A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097--1105, 2012. Google ScholarDigital Library
- A. Kupavskii, L. Ostroumova, A. Umnov, S. Usachev, P. Serdyukov, G. Gusev, and A. Kustarev. Prediction of retweet cascade size over time. In Proceedings of the 21st ACM international conference on Information and knowledge management, pages 2335--2338. ACM, 2012. Google ScholarDigital Library
- Z. Luo, M. Osborne, S. Petrovic, and T. Wang. Improving twitter retrieval by exploiting structural information. In AAAI, 2012. Google ScholarDigital Library
- Z. Luo, M. Osborne, J. Tang, and T. Wang. Who will retweet me?: finding retweeters in twitter. In Proceedings of SIGIR, pages 869--872. ACM, 2013. Google ScholarDigital Library
- M.-T. Luong, H. Pham, and C. D. Manning. Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025, 2015.Google Scholar
- T. Mikolov, M. Karafiát, L. Burget, J. Cernockỳ, and S. Khudanpur. Recurrent neural network based language model. In INTERSPEECH, 2010.Google ScholarCross Ref
- T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111--3119, 2013. Google ScholarDigital Library
- V. Mnih, N. Heess, A. Graves, et al. Recurrent models of visual attention. In NIPS, 2014. Google ScholarDigital Library
- M. Oquab, L. Bottou, I. Laptev, and J. Sivic. Learning and transferring mid-level image representations using convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1717--1724, 2014. Google ScholarDigital Library
- H.-K. Peng, J. Zhu, D. Piao, R. Yan, and Y. Zhang. Retweet modeling using conditional random fields. In Data Mining Workshops (ICDMW), 2011 IEEE 11th International Conference on, pages 336--343. IEEE, 2011. Google ScholarDigital Library
- S. Petrovic, M. Osborne, and V. Lavrenko. Rt to win! predicting message propagation in twitter. In ICWSM, 2011.Google Scholar
- R. Pfitzner, A. Garas, and F. Schweitzer. Emotional divergence influences information spreading in twitter. ICWSM, 12:2--5, 2012.Google Scholar
- P. H. Pinheiro and R. Collobert. Recurrent convolutional neural networks for scene parsing. arXiv preprint arXiv:1306.2795, 2013.Google Scholar
- A. Severyn and A. Moschitti. Learning to rank short text pairs with convolutional deep neural networks. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 373--382. ACM, 2015. Google ScholarDigital Library
- S. Sharma, R. Kiros, and R. Salakhutdinov. Action recognition using visual attention. arXiv preprint arXiv:1511.04119, 2015.Google Scholar
- Y. Shen, X. He, J. Gao, L. Deng, and G. Mesnil. Learning semantic representations using convolutional neural networks for web search. In Proceedings of the companion publication of the 23rd international conference on World wide web companion, pages 373--374. International World Wide Web Conferences Steering Committee, 2014. Google ScholarDigital Library
- R. Socher, C. C. Lin, C. Manning, and A. Y. Ng. Parsing natural scenes and natural language with recursive neural networks. In Proceedings of ICML, pages 129--136, 2011.Google Scholar
- E. Spiro, C. Irvine, C. DuBois, and C. Butts. Waiting for a retweet: modeling waiting times in information propagation. In 2012 NIPS workshop of social networks and social media conference. http://snap. stanford. edu/social2012/papers/spiro-dubois-butts. pdf. Accessed, volume 12, 2012.Google Scholar
- S. Stieglitz and L. Dang-Xuan. Political communication and influence through microblogging--an empirical analysis of sentiment in twitter messages and retweet behavior. In 2012 45th Hawaii International Conference on System Sciences. Google ScholarDigital Library
- B. Suh, L. Hong, P. Pirolli, and E. H. Chi. Want to be retweeted? large scale analytics on factors impacting retweet in twitter network. In Social computing (socialcom), 2010 ieee second international conference on, pages 177--184. IEEE, 2010. Google ScholarDigital Library
- T. Wang, D. J. Wu, A. Coates, and A. Y. Ng. End-to-end text recognition with convolutional neural networks. In Pattern Recognition (ICPR), 2012 21st International Conference on, pages 3304--3308. IEEE, 2012.Google Scholar
- T. Xiao, Y. Xu, K. Yang, J. Zhang, Y. Peng, and Z. Zhang. The application of two-level attention models in deep convolutional neural network for fine-grained image classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 842--850, 2015.Google Scholar
- F. Xiong, Y. Liu, Z.-j. Zhang, J. Zhu, and Y. Zhang. An information diffusion model based on retweeting mechanism for online social media. Physics Letters A, 376(30):2103--2108, 2012.Google ScholarCross Ref
- K. Xu, J. Ba, R. Kiros, A. Courville, R. Salakhutdinov, R. Zemel, and Y. Bengio. Show, attend and tell: Neural image caption generation with visual attention. arXiv preprint arXiv:1502.03044, 2015.Google Scholar
- M.-C. Yang, J.-T. Lee, S.-W. Lee, and H.-C. Rim. Finding interesting posts in twitter based on retweet graph analysis. In Proceedings of SIGIR, 2012. Google ScholarDigital Library
- Z. Yang, J. Guo, K. Cai, J. Tang, J. Li, L. Zhang, and Z. Su. Understanding retweeting behaviors in social networks. In Proceedings of CIKM. ACM, 2010. Google ScholarDigital Library
- T. R. Zaman, R. Herbrich, J. Van Gael, and D. Stern. Predicting information spreading in twitter. In Workshop on computational social science and the wisdom of crowds, NIPS, 2010.Google Scholar
- M. D. Zeiler. Adadelta: An adaptive learning rate method. arXiv preprint arXiv:1212.5701, 2012.Google Scholar
- J. Zhang, B. Liu, J. Tang, T. Chen, and J. Li. Social influence locality for modeling retweeting behaviors. In Proceedings of AAAI, 2013. Google ScholarDigital Library
- J. Zhang, J. Tang, J. Li, Y. Liu, and C. Xing. Who influenced you? predicting retweet via social influence locality. ACM Transactions on Knowledge Discovery from Data (TKDD), 9(3):25, 2015. Google ScholarDigital Library
- Q. Zhang, Y. Gong, Y. Guo, and X. Huang. Retweet behavior prediction using hierarchical dirichlet process. In AAAI, 2015. Google ScholarDigital Library
Index Terms
- Retweet Prediction with Attention-based Deep Neural Network
Recommendations
Hot Topic-Aware Retweet Prediction with Masked Self-attentive Model
SIGIR'19: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information RetrievalSocial media users create millions of microblog entries on various topics each day. Retweet behaviour play a crucial role in spreading topics on social media. Retweet prediction task has received considerable attention in recent years. The majority of ...
Predicting retweet count using visual cues
CIKM '13: Proceedings of the 22nd ACM international conference on Information & Knowledge ManagementSocial media platforms allow rapid information diffusion, and serve as a source of information to many of the users. Particularly, in Twitter information provided by tweets diffuses over the users through retweets. Hence, being able to predict the ...
Prediction of retweet cascade size over time
CIKM '12: Proceedings of the 21st ACM international conference on Information and knowledge managementRetweet cascades play an essential role in information diffusion in Twitter. Popular tweets reflect the current trends in Twitter, while Twitter itself is one of the most important online media. Thus, understanding the reasons why a tweet becomes ...
Comments