Omar Alonso
Marc Najork
ABSTRACT
Twitter has evolved into a significant communication nexus, coupling personal and highly contextual utterances with local news, memes, celebrity gossip, headlines, and other microblogging subgenres. If we take Twitter as a large and varied dynamic collection, how can we predict which tweets will be interesting to a broad audience in advance of lagging social indicators of interest such as retweets? The telegraphic form of tweets, coupled with the subjective notion of interestingness, makes it difficult for human judges to agree on which tweets are indeed interesting.
In this paper, we address two questions: Can we develop a reliable strategy that results in high-quality labels for a collection of tweets, and can we use this labeled collection to predict a tweet's interestingness? To answer the first question, we performed a series of studies using crowdsourcing to reach a diverse set of workers who served as a proxy for an audience with variable interests and perspectives. This method allowed us to explore different labeling strategies, including varying the judges, the labels they applied, the datasets, and other aspects of the task. To address the second question, we used crowdsourcing to assemble a set of tweets rated as interesting or not; we scored these tweets using textual and contextual features; and we used these scores as inputs to a binary classifier. We were able to achieve moderate agreement (κ = 0.52) between the best classifier and the human assessments, a figure which reflects the challenges of the judgment task.
- Alonso, O., Carson, C., Gerster, D., Ji, X., and Nabar, S. U. Detecting Uninteresting Content in Text Streams. In CSE (July 2010), 39--42.Google Scholar
- André, P., Bernstein, M. S., and Luther, K. Who gives a tweet?: evaluating microblog content value. In CSCW (2012), 471--474. Google ScholarDigital Library
- Aroyo, L., and Welty, C. Harnessing disagreement in crowdsourcing a relation extraction gold standard. Tech. Rep. RC25371 (WAT1304-058), IBM Research, 2013.Google Scholar
- Bowker, G. C., and Star, S. L. Sorting Things Out: Classification and Its Consequences. MIT Press, 1999. Google ScholarDigital Library
- Boyd, D., Golder, S., and Lotan, G. Tweet, tweet, retweet: Conversational aspects of retweeting on twitter. In HICSS (2010), 1--10. Google ScholarDigital Library
- Choudhary, A. N., Hendrix, W., Lee, K., Palsetia, D., and Liao, W.-K. Social media evolution of the Egyptian revolution. CACM 55, 5 (2012), 74--80. Google ScholarDigital Library
- Colton, S., and Bundy, A. On the notion of interestingness in automated mathematical creativity. In AISB (1999), 82--91.Google Scholar
- De Choudhury, M., Diakopoulos, N., and Naaman, M. Unfolding the event landscape on twitter: classification and exploration of user categories. In CSCW (2012), 241--244. Google ScholarDigital Library
- Duan, Y., Jiang, L., Qin, T., Zhou, M., and Shum, H.-Y. An empirical study on learning to rank of tweets. In COLING (2010), 295--303. Google ScholarDigital Library
- Erdelez, S. Information encountering: a conceptual framework for accidental information discovery. In ISIC (1997), 412--421. Google ScholarDigital Library
- Fleiss, J. L. Measuring nominal scale agreement among many raters. Psychol.l Bull. 76, 5 (1971), 378--382.Google Scholar
- Gaffney, D. #iranelection: quantifying online activism. In WebSci10 (2010).Google Scholar
- Gayo-Avello, D. Nepotistic relationships in twitter and their impact on rank prestige algorithms. IPM (2013), 1250--1280.Google Scholar
- Hughes, A. L., and Palen, L. Twitter adoption and use in mass convergence and emergency events. IJEM 6, 3/4 (2009), 248--260.Google ScholarCross Ref
- Hurlock, J., and Wilson, M. L. Searching twitter: Separating the tweet from the chaff. In ICWSM (2011).Google Scholar
- Krippendorff, K. Content Analysis, 2nd ed. Sage Publications, 2003.Google Scholar
- Lempel, R., and Moran, S. Salsa: the stochastic approach for link-structure analysis. TOIS 19, 2 (2001), 131--160. Google ScholarDigital Library
- Lewis, P. Reading the riots: Investigating England's summer of disorder. The Guardian, September 5, 2011.Google Scholar
- Lin, T., Oren, Etzioni, and Fogarty, J. Identifying interesting assertions from the web. In CIKM (2009), 1787--1790. Google ScholarDigital Library
- Marcus, A., Bernstein, M., Badar, O., Karger, D., Madden, S., and Miller, R. Twitinfo: aggregating and visualizing microblogs for event exploration. In CHI (2011), 227--236. Google ScholarDigital Library
- Metzler, D., and Cai, C. USC/ISI at TREC 2011: Microblog track. In TREC (2011).Google Scholar
- Momeni, E., Tao, K., Haslhofer, B., and Houben, G.-J. Identification of useful user comments in social media: A case study on Flickr commons. In JCDL (2013), 1--10. Google ScholarDigital Library
- Morris, M. R., Counts, S., Roseway, A., Hoff, A., and Schwarz, J. Tweeting is believing?: understanding microblog credibility perceptions. In CSCW (2012), 441--450. Google ScholarDigital Library
- Naaman, M., Becker, H., and Gravano, L. Hip and trendy: characterizing emerging trends on twitter. JASIST 62, 5 (2011), 902--918. Google ScholarDigital Library
- Poblete, B., Garcia, R., Mendoza, M., and Jaimes, A. Do all birds tweet the same? characterizing twitter around the world. In CIKM (2011), 1025--1030. Google ScholarDigital Library
- Robertson, S. E., Walker, S., Jones, S., Hancock-Beaulieu, M., and Gatford, M. Okapi at TREC-3. In TREC (1994).Google Scholar
- Sakaki, T., Okazaki, M., and Matsuo, Y. Earthquake shakes twitter users: real-time event detection by social sensors. In WWW (2010), 851--860. Google ScholarDigital Library
- Silvia, P. J. What is interesting? exploring the appraisal structure of interest. Emotion 5 (2005), 89--102.Google ScholarCross Ref
- Strauss, A., and Corbin, J. Basics of Qualitative Research. Sage Publications, 1998.Google Scholar
- Uysal, I., and Croft, W. B. User oriented tweet ranking: a filtering approach to microblogs. In CIKM (2011), 2261--2264. Google ScholarDigital Library
- Yardi, S., Romero, D., Schoenebeck, G., and Boyd, D. Detecting spam in a twitter network. First Monday 15, 1 (2010).Google Scholar
- Zhao, D., and Rosson, M. B. How and why people twitter: the role that micro-blogging plays in informal communication at work. In GROUP (2009), 243--252. Google ScholarDigital Library
- Zubiaga, A., Spina, D., Fresno, V., and Martínez, R. Classifying trending topics: a typology of conversation triggers on twitter. In CIKM (2011), 2461--2464. Google ScholarDigital Library
Index Terms
- Are Some Tweets More Interesting Than Others? #HardQuestion
Recommendations
Inferring the Interesting Tweets in Your Network
CGC '13: Proceedings of the 2013 International Conference on Cloud and Green ComputingAs the demand for quick, live and relevant information increases, more people look to microblogging sites, such as Twitter, as a source of content. Retweeting acts as a filter of useful information for users with more interesting information likely to ...
Analyzing and predicting viral tweets
WWW '13 Companion: Proceedings of the 22nd International Conference on World Wide WebTwitter and other microblogging services have become indispensable sources of information in today's web. Understanding the main factors that make certain pieces of information spread quickly in these platforms can be decisive for the analysis of ...
Analysis of Tweets Related to Cyberbullying: Exploring Information Diffusion and Advice Available for Cyberbullying Victims
The use of Twitter, especially by teenagers and young people, has raised the issue of cyberbullying. There is a lack of research into what types of advice and support are available in tweets for cyberbullying victims, and into the features influencing ...
Comments