Ege Cem Kirci
ABSTRACT
What are the worst outages for Internet users? How long do they last, and how wide are they? Such questions are hard to answer via traditional outage detection and analysis techniques, as they conventionally rely on network-level signals and do not necessarily represent users' perceptions of connectivity.
We present SIFT, a detection and analysis tool for capturing user-affecting Internet outages. SIFT leverages users' aggregated web search activity to detect outages. Specifically, SIFT starts by building a timeline of users' interests in outage-related search queries. It then analyzes this timeline looking for spikes of user interest. Finally, SIFT characterizes these spikes in duration, geographical extent, and simultaneously trending search terms which may help understand root causes, such as power outages or associated ISPs.
We use SIFT to collect more than 49 000 Internet outages in the United States over the last two years. Among others, SIFT reveals that user-affecting outages: (i) do not happen uniformly: half of them originate from 10 states only; (ii) can affect users for a long time: 10% of them last at least 3 hours; and (iii) can have a broad impact: 11% of them simultaneously affect at least 10 distinct states. SIFT annotations also reveal a perhaps overlooked fact: outages are often caused by climate and/or power-related issues.
Supplemental Material
- Giuseppe Aceto, Alessio Botta, Pietro Marchetta, Valerio Persico, and Antonio Pescapé. 2018. A Comprehensive Survey on Internet Outages. Journal of Network and Computer Applications 113 (2018), 36--63. Google Scholar
Cross Ref
- Lucas Aulbach. 2021. Thousands Still without Power in Kentucky Following Devastating Tornado Outbreak. https://www.courier-journal.com/story/news/2021/12/11/kentucky-tornado-widespread-power-outages-amid-damage/6475197001/Google Scholar
- Benton Institute. 2020. June 15, 2020 T-Mobile Network Outage Report. https://www.benton.org/headlines/june-15-2020-1-mobile-network-outage-reportGoogle Scholar
- Daniele Besana. 2017. ThousandEyes Review: Outage Detection. https://www.routerfreak.com/thousandeyes-review-outage-detection/Google Scholar
- Boggs. 2021. Several Schools Closed as Thousands Remain without Power. https://spectrumnews1.com/oh/columbus/news/2021/08/12/several-schools-closed-as-thousands-remain-without-powerGoogle Scholar
- Matt Charnock. 2020. Charge Your Power Banks: Rotating Blackouts and Power Shutoffs Possible In Parts of Bay Area This Week. https://sfist.com/2020/09/06/charge-your-power-banks-rotating-bay-area-blackouts-possible-tuesday-and-wednesday/Google Scholar
- Hyunyoung Choi. 2009. Predicting Initial Claims for Unemployment Benefits. SSRN Scholarly Paper 1659307. Social Science Research Network, Rochester, NY. https://papers.ssrn.com/abstract=1659307Google Scholar
- Hyunyoung Choi and Hal Varian. 2012. Predicting the Present with Google Trends. Economic Record 88, s1 (2012), 2--9. Google ScholarCross Ref
- Crisis24. 2020. US: Tropical Storm Zeta Causes Disruptions in Georgia October 29 /Update 4. https://crisis24.garda.com/alerts/2020/10/us-tropical-storm-zeta-causes-disruptions-in-georgia-october-29-update-4Google Scholar
- Zhi Da, Joseph Engelberg, and Pengjie Gao. 2011. In Search of Attention. The Journal of Finance 66, 5 (Oct. 2011), 1461--1499. Google ScholarCross Ref
- Kyle Davidson. 2021. Storms Leave 600,000+ Michiganders without Power; Flash Flood Warning in Wayne County. https://www.freep.com/story/news/local/michigan/2021/08/11/storms-cause-power-outages-michigan/8100403002/Google Scholar
- Downdetector. 2022. About Us | We Detect When Technology Fails. https://downdetector.com/Google Scholar
- Clare Duffy. 2021. Comcast Xfinity Internet Outage Hits Customers across the US. https://www.cnn.com/2021/11/09/tech/comcast-xfinity-outage/index.htmlGoogle Scholar
- Jennifer Elias. 2020. Google 'overwhelmingly' Dominates Search Market, Antitrust Committee States. https://www.cnbc.com/2020/10/06/google-overwhelmingly-dominates-search-market-house-committee-finds.htmlGoogle Scholar
- Gunther Eysenbach. 2006. Infodemiology: Tracking Flu-Related Searches on the Web for Syndromic Surveillance. AMIA Annual Symposium Proceedings 2006 (2006), 244--248. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1839505/Google Scholar
- Mandi Cai Ferman, Erin Douglas and Mitchell. 2022. How Texas' Power Grid Failed in 2021 --- and Who's Responsible for Preventing a Repeat. https://www.texastribune.org/2022/02/15/texas-power-grid-winter-storm-2021/Google Scholar
- Kevin Fogarty. 2020. Outages Spike in Late April as COVID-19 Trends Strain Internet. https://www.spglobal.com/marketintelligence/en/news-insights/latest-news-headlines/outages-spike-in-late-april-as-covid-19-trends-strain-internet-58412923Google Scholar
- Jeremy Ginsberg, Matthew H. Mohebbi, Rajan S. Patel, Lynnette Brammer, Mark S. Smolinski, and Larry Brilliant. 2009. Detecting Influenza Epidemics Using Search Engine Query Data. Nature 457, 7232 (Feb. 2009), 1012--1014. Google ScholarCross Ref
- Sharad Goel, Jake M. Hofman, Sébastien Lahaie, David M. Pennock, and Duncan J. Watts. 2010. Predicting Consumer Behavior with Web Search. Proceedings of the National Academy of Sciences 107, 41 (Oct. 2010), 17486--17490. Google ScholarCross Ref
- Jazmin Goodwin. 2020. Major Internet Outage: Dozens of Websites and Apps Were Down. https://www.cnn.com/2020/08/30/tech/internet-outage-cloudflare/index.htmlGoogle Scholar
- John Graham-Cumming. 2020. Cloudflare Outage on July 17, 2020. http://blog.cloudflare.com/cloudflare-outage-on-july-17-2020/Google Scholar
- Alex Hasenstab. 2021. Massive Pacific Northwest Storm Causes Power Outages, Downed Trees. https://www.opb.org/article/2021/10/24/storm-to-cause-winds-up-to-65-mph-along-pacific-northwest-coast/Google Scholar
- John Heidemann, Yuri Pradkin, Ramesh Govindan, Christos Papadopoulos, Genevieve Bartlett, and Joseph Bannister. 2008. Census and Survey of the Visible Internet. In Proceedings of the 8th ACM SIGCOMM Conference on Internet Measurement (IMC '08). Association for Computing Machinery, New York, NY, USA, 169--182. Google ScholarDigital Library
- Zach Hillstrom. 2021. Severed Power Line Causing Water Outages and Issues in Colorado City. https://www.chieftain.com/story/news/2021/07/22/colorado-city-dealing-water-outages-due-severed-power-line/8055846002/Google Scholar
- Anette Hulth, Gustaf Rydevik, and Annika Linde. 2009. Web Queries as a Source for Syndromic Surveillance. PLoS ONE 4, 2 (Feb. 2009), e4378. Google ScholarCross Ref
- Santosh Janardhan. 2021. Update about the October 4th Outage. https://engineering.fb.com/2021/10/04/networking-traffic/outage/Google Scholar
- Andrew Kirmse. 2020, January. Andrew Kirmse's Page - Topographic Prominence. https://www.andrewkirmse.com/prominenceGoogle Scholar
- Christian Köpp. 2013. Controlled Internet Outage Monitoring. Network (Bristol, England) 11 (2013).Google Scholar
- Maxmind. 2022. GeoIP® Databases & Services: Industry Leading IP Intelligence. https://www.maxmind.com/en/geoip2-services-and-databasesGoogle Scholar
- Juno Mayer, Valerie Sahakian, Emilie Hooft, Douglas Toomey, and Ramakrishnan Durairajan. 2021. On the Resilience of Internet Infrastructures in Pacific Northwest to Earthquakes. In Passive and Active Measurement, Oliver Hohlfeld, Andra Lutu, and Dave Levin (Eds.). Springer International Publishing, Cham, 247--265.Google Scholar
- Carlo Medas. 2019. Building a Real-Time Internet Outage Detection System with Flink @ Fing | LinkedIn. https://www.linkedin.com/pulse/building-real-time-internet-outage-detection-system-flink-carlo-medas/Google Scholar
- Shikhar Mehrotra. 2021. What Led to Internet Outage That Took down Some Major Websites on July 22? Check out Why. https://www.republicworld.com/technology-news/other-tech-news/what-led-to-internet-outage-that-took-down-some-major-websites-on-july-22-check-out-why.htmlGoogle Scholar
- Marti Motoyama, Brendan Meeder, Kirill Levchenko, Geoffrey M Voelker, and Stefan Savage. 2010. Measuring Online Service Availability Using Twitter. In 3rd Workshop on Online Social Networks (WOSN 2010).Google Scholar
- Google News Initiative. 2022. Fundamentals Course. https://newsinitiative.withgoogle.com/training/course/fundamentalsGoogle Scholar
- Matthew Odam and Peter Blackstock. 2021. Severe Weather Causing Cellular and Internet Outages in Austin Area. https://www.austin360.com/story/news/2021/02/15/severe-weather-cellular-internet-outages-austin/6752081002/Google Scholar
- Ookla. 2020. How Downdetector Works. https://www.ookla.com/articles/how-downdetector-worksGoogle Scholar
- Ramakrishna Padmanabhan, Aaron Schulman, Dave Levin, and Neil Spring. 2019. Residential Links under the Weather. In Proceedings of the ACM Special Interest Group on Data Communication (SIGCOMM '19). Association for Computing Machinery, New York, NY, USA, 145--158. Google ScholarDigital Library
- Camille Pelat, Clément Turbelin, Avner Bar-Hen, Antoine Flahault, and Alain-Jacques Valleron. 2009. More Diseases Tracked by Using Google Trends. Emerging Infectious Diseases 15, 8 (Aug. 2009), 1327--1328. Google ScholarCross Ref
- Jay Peters. 2020. YouTube Went down around the World, but It's Now Fixed. https://www.theverge.com/2020/11/11/21561764/youtube-down-outage-loading-videosGoogle Scholar
- Jay Peters. 2021. Networks Are Struggling in Texas amid Historic Winter Storms. https://www.theverge.com/2021/2/15/22284309/texas-winter-storms-internet-cell-network-issues-t-mobile-att-spectrumGoogle Scholar
- Philip M. Polgreen, Yiling Chen, David M. Pennock, and Forrest D. Nelson. 2008. Using Internet Searches for Influenza Surveillance. Clinical Infectious Diseases: An Official Publication of the Infectious Diseases Society of America 47, 11 (Dec. 2008), 1443--1448. Google ScholarCross Ref
- Associated Press. 2021. Thousands Hit by Internet Outage on East Coast. https://www.mercurynews.com/2021/01/26/internet-outage-strikes-many-on-east-coastGoogle Scholar
- Lin Quan, John Heidemann, and Yuri Pradkin. 2013. Trinocular: Understanding Internet Reliability through Adaptive Probing. In Proceedings of the ACM SIGCOMM 2013 Conference on SIGCOMM (SIGCOMM '13). Association for Computing Machinery, New York, NY, USA, 255--266. Google ScholarDigital Library
- Wire Reports. 2021. Internet Recovers after Massive Outage; Verizon Acknowledges 'Issue' as Source | WRAL TechWire. https://wraltechwire.com/2021/01/27/internet-recovers-after-massive-outage-verizon-acknowledges-issue-as-source/Google Scholar
- Reuters. 2021. Amazon Cloud Unit Recovers from Brief Outage Affecting Third-Party Services. https://www.reuters.com/markets/commodities/amazon-owned-twitch-down-many-users-2021-12-15/Google Scholar
- Philipp Richter, Ramakrishna Padmanabhan, Neil Spring, Arthur Berger, and David Clark. 2018. Advancing the Art of Internet Edge Outage Detection. In Proceedings of the Internet Measurement Conference 2018 (IMC '18). Association for Computing Machinery, New York, NY, USA, 350--363. Google ScholarDigital Library
- Simon Rogers. 2016. What Is Google Trends Data --- and What Does It Mean? https://medium.com/google-news-lab/what-is-google-trends-data-and-what-does-it-mean-b48f07342ee8Google Scholar
- Aaron Schulman and Neil Spring. 2011. Pingin' in the Rain. In Proceedings of the 2011 ACM SIGCOMM Conference on Internet Measurement Conference (IMC '11). Association for Computing Machinery, New York, NY, USA, 19--28. Google ScholarDigital Library
- Staff reports. 2020. AT&T Outage Sunday Updates: 'Progress Continues' in Restoring Service after Nashville Bombing. https://www.tennessean.com/story/news/crime/2020/12/26/att-outage-internet-911-phones-nasvhille-updates-saturday/4046849001/Google Scholar
- Michael Tanenbaum. 2020-01-23T15:09:11.484449-05:00. Comcast Experienced a Nationwide Internet Outage on Thursday. https://www.phillyvoice.com/comcast-nationwide-internet-outage-2020-january-23-philadelphia/Google Scholar
- Google Trends. 2022. Google Trends. https://trends.google.com/trends/?geo=CHGoogle Scholar
- USC/LANDER Project. 2022. The ANT Lab: Analysis of Network Traffic | ANT Datasets | Internet Outage Data. https://ant.isi.edu/Google Scholar
- Jordan Valinsky and David Goldman. 2021. Massive Internet Outage: Websites and Apps around the World Go Dark. https://www.cnn.com/2021/06/08/tech/internet-outage-fastly/index.htmlGoogle Scholar
- Zhongqing Wang and Yue Zhang. 2017. DDoS Event Forecasting Using Twitter Data. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence Organization, Melbourne, Australia, 4151--4157. Google ScholarCross Ref
Index Terms
- "Is my internet down?": sifting through user-affecting outages with Google trends
Recommendations
Context-Based Anomaly Detection via Spatial Attributed Graphs in Human Monitoring
Neural Information ProcessingAbstractDetecting anomalous regions focusing on interesting parts in surveillance images is a significant task in human monitoring. The most challenging problem is that a large number of anomalies in monitoring environments are context-based, which ...
Rotation-invariant and scale-invariant Gabor features for texture image retrieval
Conventional Gabor representation and its extracted features often yield a fairly poor performance in retrieving the rotated and scaled versions of the texture image under query. To address this issue, existing methods exploit multiple stages of ...
A Statistical-Based Anomaly Detection Method for Connected Cars in Internet of Things Environment
IOV 2015: Proceedings of the Second International Conference on Internet of Vehicles - Safe and Intelligent Mobility - Volume 9502A connected car is the most successful thing in the era of Internet of Things IoT. The connections between vehicles and networks grow and provide more convenience to users. However, vehicles become exposed to malicious attacks from outside. Therefore, a ...
Comments