Skip to main content
SpringerLink
Log in

Social media signal detection using tweets volume, hashtag, and sentiment analysis

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Social Media is a well-known platform for users to create, share and check the new information. The world becomes a global village because of the utilization of internet and social media. The data present on Twitter contains information of great importance. There is a strong need to extract valuable information from this huge amount of data. A key research challenge in this area is to analyze and process this huge data and detect the signals or spikes. Existing work includes sentiment analysis for Twitter, hashtag analysis, and event detection but spikes/signal detection from Twitter remains an open research area. From this line of research, we propose a signal detection approach using sentiment analysis from Twitter data (tweets volume, top hashtag and sentiment analysis). In this paper, we propose three algorithms for signal detection in tweets volume, tweets sentiment and top hashtag. The algorithms are the- Average moving threshold algorithm, Gaussian algorithm, and hybrid algorithm. The hybrid algorithm is a combination of the average moving threshold algorithm and Gaussian algorithm. The proposed algorithms are tested over real-time data extracted from Twitter and two large publically available datasets- Saudi Aramco dataset and BP America dataset. Experimental results show that hybrid algorithm outperforms the Gaussian and average moving threshold algorithm and achieve a precision of 89% on real-time tweets data, 88% on Saudi Aramco dataset and 81% on BP America dataset with the recall of 100%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Abdelhaq H, Sengstock C, Gertz M (2013) Eventweet: online localized event detection from twitter. Proceedings of the VLDB Endowment 6:1326–1329

    Article  Google Scholar 

  2. Adnan M, Longley P (2013) Analysis of twitter usage in london, paris, and new york city, in 16th AGILE international conference on geographic information science, Leuven, pp. 1–7

  3. Agarwal A, Xie B, Vovsha I, Rambow O, Passonneau R (2011) Sentiment analysis of twitter data, in Proceedings of the workshop on languages in social media, pp. 30–38

  4. Alag S (2008) Collective intelligence in action: Manning Publications Co.

  5. Asur S, Huberman BA (2010) Predicting the future with social media, in Proceedings of the 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology-Volume 01, pp. 492–499

  6. Bagui S, Nguyen LT (2015) Database sharding: to provide fault tolerance and scalability of big data on the cloud. International Journal of Cloud Applications and Computing (IJCAC) 5:36–52

    Article  Google Scholar 

  7. Bastos MT (2015) Shares, pins, and tweets: news readership from daily papers to social media. Journal Stud 16:305–325

    Google Scholar 

  8. Chen L, Roy A (2009) Event detection from flickr data through wavelet-based spatial analysis, in Proceedings of the 18th ACM conference on Information and knowledge management, pp. 523–532

  9. Ellison NB (2007) Social network sites: definition, history, and scholarship. J Comput-Mediat Commun 13:210–230

    Article  Google Scholar 

  10. Ellison N, Steinfield C, Lampe C (2006) Spatially bounded online social networks and social capital, International Communication Association, vol. 36

  11. Ferragina P, Piccinno F, Santoro R (2015) On analyzing hashtags in twitter, in International Conference on Web and Social Media (ICWSM), pp. 110–119

  12. Fung GPC, Yu JX, Yu PS, Lu H (2005) Parameter free bursty events detection in text streams, in Proceedings of the 31st international conference on Very large data bases, pp. 181–192

  13. Gupta B, Agrawal DP, Yamaguchi S (2016) Handbook of research on modern cryptographic solutions for computer and cyber security: IGI Global

  14. He Q, Chang K, Lim E-P (2007) Analyzing feature trajectories for event detection, in Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, pp. 207–214

  15. Herbert A, Nenadic G (2015) Text Mining Twitter Posts For Signs of Social Isolation

  16. Kaplan AM, Haenlein M (2010) Users of the world, unite! The challenges and opportunities of social media. Business horizons 53:59–68

    Article  Google Scholar 

  17. Kleinberg J (2003) Bursty and hierarchical structure in streams. Data Min Knowl Disc 7:373–397

    Article  MathSciNet  Google Scholar 

  18. Kouloumpis E, Wilson T, Moore JD (2011) Twitter sentiment analysis: the good the bad and the omg! Icwsm 11:164

    Google Scholar 

  19. Lazer D, Pentland AS, Adamic L, Aral S, Barabasi AL, Brewer D et al (2009) Life in the network: the coming age of computational social science. Science (New York, NY) 323:721

    Article  Google Scholar 

  20. Li Y, Wang G, Nie L, Wang Q, Tan W (2018) Distance metric optimization driven convolutional neural network for age invariant face recognition. Pattern Recogn 75:51–62

    Article  Google Scholar 

  21. McCormick TH, Lee H, Cesare N, Shojaie A, Spiro ES (2017) Using twitter for demographic and social science research: tools for data collection and processing. Sociol Methods Res 46:390–421

    Article  MathSciNet  Google Scholar 

  22. Mcnee SM (2006) Meeting user information needs in recommender systems: University of Minnesota

  23. Melville P, Sindhwani V, Lawrence R (2009) Social media analytics: channeling the power of the blogosphere for marketing insight. Proc of the WIN 1:1–5

    Google Scholar 

  24. Murphy J, Link MW, Childs JH, Tesfaye CL, Dean E, Stern M et al (2014) Social media in public opinion research: executive summary of the Aapor task force on emerging technologies in public opinion research. Public Opinion Quarterly 78:788–794

    Article  Google Scholar 

  25. O'Connor B, Balasubramanyan R, Routledge BR, Smith NA (2010) From tweets to polls: linking text sentiment to public opinion time series. Icwsm 11:1–2

    Google Scholar 

  26. Ouf S, Nasr M (2015) Cloud computing: the future of big data management. International Journal of Cloud Applications and Computing (IJCAC) 5:53–61

    Article  Google Scholar 

  27. Petrović S, Osborne M, Lavrenko V (2010) Streaming first story detection with application to twitter, in Human language technologies: The 2010 annual conference of the north american chapter of the association for computational linguistics, pp. 181–189

  28. Petrovic S, Osborne M, McCreadie R, Macdonald C, Ounis I, Shrimpton L (2013) Can twitter replace newswire for breaking news?, in ICWSM

  29. Qualman E (2010) Socialnomics: How social media transforms the way we live and do business: John Wiley & Sons

  30. Ramasubramanian C, Ramya R (2013) Effective pre-processing activities in text mining using improved porter’s stemming algorithm. International Journal of Advanced Research in Computer and Communication Engineering 2:2278–1021

    Google Scholar 

  31. Sakaki T, Okazaki M, Matsuo Y (2010) Earthquake shakes Twitter users: real-time event detection by social sensors, in Proceedings of the 19th international conference on World wide web, pp. 851–860

  32. Small TA (2011) What the hashtag? A content analysis of Canadian politics on twitter. Inf Commun Soc 14:872–895

    Article  Google Scholar 

  33. Spencer J, Uchyigit G (2012) Sentimentor: Sentiment analysis of twitter data, in Proceedings of European conference on machine learning and principles and practice of knowledge discovery in databases, pp. 56–66

  34. Trusov M, Bucklin RE, Pauwels K (2009) Effects of word-of-mouth versus traditional marketing: findings from an internet social networking site. J Mark 73:90–102

    Article  Google Scholar 

  35. Wang X, Wei F, Liu X, Zhou M, Zhang M (2011) Topic sentiment analysis in twitter: a graph-based hashtag sentiment classification approach, in Proceedings of the 20th ACM international conference on Information and knowledge management, pp. 1031–1040

  36. Watts DJ (2004) The “new” science of networks. Annu Rev Sociol 30:243–270

    Article  Google Scholar 

  37. Weng J, Lee B-S (2011) Event detection in twitter. ICWSM 11:401–408

    Google Scholar 

  38. Yang Y, Pierce T, Carbonell J (1998) A study of retrospective and on-line event detection, in Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, pp. 28–36

  39. Zhang Z, Gupta BB (2016) Social media security and trustworthiness: overview and new direction, Future Generation Computer Systems

  40. Zhang Z, Sun R, Zhao C, Wang J, Chang CK, Gupta BB (2017) CyVOD: a novel trinity multimedia social network scheme. Multimedia Tools and Applications 76:18513–18529

    Article  Google Scholar 

  41. Zheng L, Wang H, Gao S (2018) Sentimental feature selection for sentiment analysis of Chinese online reviews. Int J Mach Learn Cybern 9:75–84

    Article  Google Scholar 

Download references

Acknowledgments

This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2016R1D1A1A09919551).

Author information

Authors and Affiliations

  1. Department of Software Engineering, University of Engineering and Technology Taxila, Taxila, Pakistan

    Faria Nazir & Mustansar Ali Ghazanfar

  2. Department of Computer Science, COMSATS University Islamabad, Attock Campus, Attock, Pakistan

    Muazzam Maqsood & Farhan Aadil

  3. Department of Media Software, Sungkyul University, Anyang, South Korea

    Seungmin Rho

  4. Department of Software, Sejong University, Seoul, South Korea

    Irfan Mehmood

Authors
  1. Faria Nazir
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mustansar Ali Ghazanfar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Muazzam Maqsood
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Farhan Aadil
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Seungmin Rho
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Irfan Mehmood
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Irfan Mehmood.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nazir, F., Ghazanfar, M.A., Maqsood, M. et al. Social media signal detection using tweets volume, hashtag, and sentiment analysis. Multimed Tools Appl 78, 3553–3586 (2019). https://doi.org/10.1007/s11042-018-6437-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6437-z

Keywords

Search

Navigation