Abstract
Recently there has been a growing attention on the use of web and social data to improve traditional prediction models in politics, finance, marketing and health, but even though a correlation between observed phenomena and related social data has been demonstrated in many cases, yet the effectiveness of the latter for long-term or even mid-term predictions has not been shown. In epidemiological surveillance, the problem is compounded by the fact that infectious diseases models (such as susceptible-infected-recovered-susceptible, SIRS) are very sensitive to current conditions, such that small changes can produce remarkable differences in future outcomes. Unfortunately, current or nearly-current conditions keep changing as data are collected and updated by the epidemiological surveillance organizations. In this paper we show that the time series of Twitter messages reporting a combination of symptoms that match the influenza-like-illness (ILI) case definition represent a more stable and reliable information on “current conditions”, to the point that they can replace, rather than simply integrate, official epidemiological data. We estimate the effectiveness of these data at predicting current and past flu seasons (17 seasons overall), in combination with official historical data on past seasons, obtaining an average correlation of 0.85 over a period of 17 weeks covering the flu season.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Yu, S., Kak, S.: A Survey of Prediction using Social Media (2012), http://arxiv.org/ftp/arxiv/papers/1203/1203.1647.pdf (retrieved)
Asur, S., Huberman, B.H.: Predicting the future with social media, http://www.hpl.hp.com/research/scl/papers/socialmedia/socialmedia.pdf (retrieved)
Radinsky, K., Horvitz, E.: Mining the web to predict future events. In: WSDM 2013, pp. 255–264 (2013)
Ruiz, E.J., Hristidis, V., Castillo, C., Gionis, A., Jaimes, A.: Correlating financial time series with micro-blogging activity. In: Proceedings of the Fifth ACM International Conference on Web Search and Data Mining, WSDM 2012, pp. 513–522. ACM, New York (2012)
Carrifere-Swallow, Y., Labbfe, F.: Nowcasting with Google Trends in an Emerging Market. Central Bank of Chile, Working Papers n. 588 (2010)
Ginsberg, J., Mohebbi, M.H., Patel, R.S., Brammer, L., Smolinski, M.S., et al.: Detecting influenza epidemics using search engine query data. Nature 457, 1012–1014 (2009)
Althouse, B.M., Ng, Y.Y., Cummings, D.A.T.: Prediction of Dengue Incidence Using Search Query Surveillance. PLoS Negl. Trop. Dis. 5(8) (2011)
Xu, D., Liu, Y., Zhang, M., Ma, S., Ciu, A., Ru, L.: Predicting Epidemic Tendency through Search Behaviour Analysis. In: Proc. of 22nd IJCAI (2011)
Lamb, A., Paul, M.J., Dredze, M.: Separating Fact from Fear: Tracking Flu Infections on Twitter. In: NAACL (2013)
Achrekar, H., Gandhe, A., Lazarus, R., Yu, S., Liu, B.: Predicting Flu Trends using Twitter Data. In: CPNS 2011 (2011)
Shaman, J., Karspeck, A., Yang, W., Tamerius, J., Lipsitch, M.: Real-time influenza forecasts during the 2012-2013 season. Nature Communications 4, 2837 (2013), doi:10.1038/ncomms3837
Gesualdo, F., Stilo, G., Agricola, E., Gonfiantini, M.V., Pandolfi, E., Velardi, P., Tozzi, A.E.: Influenza-like illness surveillance on Twitter through automated learning of naïve language. PloS One Public Library of Science One, Journal (2013)
Velardi, P., Stilo, G., Tozzi, A.E., Gesualdo, F.: Twitter mining for fine-grained syndromic surveillance. Artificial Intelligence in Medicine, Special Issue on Text Mining and Information Analysis (in press, 2014)
Stilo, G., De Vincenzi, M., Tozzi, A.E., Velardi, P.: Automated Learning of Everyday Patients’ Language for Medical Blog Analytics. In: Proceedings of the Recent Advances in Natural Language Processing (RANLP 2013) Hissar, Hissar (Bulgaria), September 9-11 (2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Stilo, G., Velardi, P., Tozzi, A.E., Gesualdo, F. (2014). Predicting Flu Epidemics Using Twitter and Historical Data. In: Ślȩzak, D., Tan, AH., Peters, J.F., Schwabe, L. (eds) Brain Informatics and Health. BIH 2014. Lecture Notes in Computer Science(), vol 8609. Springer, Cham. https://doi.org/10.1007/978-3-319-09891-3_16
Download citation
DOI: https://doi.org/10.1007/978-3-319-09891-3_16
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-09890-6
Online ISBN: 978-3-319-09891-3
eBook Packages: Computer ScienceComputer Science (R0)