skip to main content
10.1145/3366424.3383298acmconferencesArticle/Chapter ViewAbstractPublication PagesthewebconfConference Proceedingsconference-collections
research-article

On Twitter Purge: A Retrospective Analysis of Suspended Users

Published: 20 April 2020 Publication History

Abstract

Abuse and spam in Twitter have long been a pressing issue, and in response, Twitter regularly purges (i.e., suspends in mass) accounts that violate Twitter Rules. However, there is no available information about the characteristics and activities of these regularly purged users. We have developed a novel and comprehensive measurement mechanism to identify millions of purged Twitter users and collect their tweets. We have identified 2.4M purged users and collected 1M tweets made by them over eight months. Using our dataset, we perform a retrospective analysis to characterize their account properties and behavioral activities. We analyze their tweet content to identify their role and abuse strategy over-time.
Our analysis shows that the abuse on Twitter is pervasive globally and not confined in mere spamming. Alarmingly, more than 60% of the purged users survived on Twitter for more than two years. We observe that politics is a major theme among the purged users irrespective of language and location, and these politically motivated users spread controversial content consistently over time. However, the spammers reorient their agenda across time to participate in multiple marketing campaigns. We also discover interaction and associated communities among purged users. Our analysis sheds new light on the evolving nature of abuse in Twitter that can help researchers understanding the characteristics and behavior of emerging malicious users to develop an effective defense system.

References

[1]
2017. Facebook Election 2106 Update. https://about.fb.com/news/2017/09/information-operations-update/
[2]
2018. How Twitter is fighting spam and malicious automation. https://blog.twitter.com/official/en_us/topics/company/2018/how-twitter-is-fighting-spam-and-malicious-automation.html
[3]
2018. Twitter Election 2016 Update. https://blog.twitter.com/en_us/topics/company/2018/2016-election-update.html
[4]
2018. Twitter is sweeping out accounts like never before.https://www.washingtonpost.com/technology/2018/07/06/twitter-is-sweeping-out-fake-accounts-like-never-before-putting-user-growth-risk/
[5]
2019. Mueller report. https://www.justice.gov/storage/report.pdf
[6]
2020. iHeartRadio Music Awards. https://en.wikipedia.org/wiki/IHeartRadio_Music_Awards
[7]
2020. #Qanon. https://en.wikipedia.org/wiki/QAnon
[8]
2020. Twitter terms of services. https://twitter.com/en/tos
[9]
Noor Abu-El-Rub and Abdullah Mueen. 2019. BotCamp: Bot-driven Interactions in Social Campaigns. In The World Wide Web Conference. 2529–2535.
[10]
Hunt Allcott and Matthew Gentzkow. 2017. Social media and fake news in the 2016 election. Journal of economic perspectives 31, 2 (2017), 211–36.
[11]
Abdullah Almaatouq, Ahmad Alabdulkareem, Mariam Nouh, Erez Shmueli, Mansour Alsaleh, Vivek K Singh, Abdulrahman Alarifi, Anas Alfaris, and Alex Sandy Pentland. 2014. Twitter: who gets caught? observed trends in social micro-blogging spam. In Proceedings of the 2014 ACM conference on Web science. ACM, 33–41.
[12]
Ahmer Arif, Leo Graiden Stewart, and Kate Starbird. 2018. Acting the part: Examining information operations within# BlackLivesMatter discourse. Proceedings of the ACM on Human-Computer Interaction 2, CSCW(2018), 20.
[13]
Adam Badawy, Emilio Ferrara, and Kristina Lerman. 2018. Analyzing the digital traces of political manipulation: The 2016 russian interference twitter campaign. In 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM). IEEE, 258–265.
[14]
Alessandro Bessi and Emilio Ferrara. 2016. Social bots distort the 2016 US Presidential election online discussion. First Monday 21, 11-7 (2016).
[15]
Qiang Cao, Xiaowei Yang, Jieqi Yu, and Christopher Palow. 2014. Uncovering large groups of active malicious accounts in online social networks. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security. 477–488.
[16]
Eshwar Chandrasekharan, Mattia Samory, Shagun Jhaver, Hunter Charvat, Amy Bruckman, Cliff Lampe, Jacob Eisenstein, and Eric Gilbert. 2018. The Internet’s Hidden Rules: An Empirical Study of Reddit Norm Violations at Micro, Meso, and Macro Scales. Proceedings of the ACM on Human-Computer Interaction 2, CSCW(2018), 32.
[17]
Nikan Chavoshi, Hossein Hamooni, and Abdullah Mueen. 2016. DeBot: Twitter Bot Detection via Warped Correlation. In ICDM. 817–822.
[18]
Michael D Conover, Jacob Ratkiewicz, Matthew Francisco, Bruno Gonçalves, Filippo Menczer, and Alessandro Flammini. 2011. Political polarization on twitter. In Fifth international AAAI conference on weblogs and social media.
[19]
Clayton Allen Davis, Onur Varol, Emilio Ferrara, Alessandro Flammini, and Filippo Menczer. 2016. Botornot: A system to evaluate social bots. In Proceedings of the 25th international conference companion on world wide web. 273–274.
[20]
Munmun De Choudhury. 2011. Tie formation on twitter: Homophily and structure of egocentric networks. In 2011 IEEE third international conference on privacy, security, risk and trust and 2011 IEEE third international conference on social computing. IEEE, 465–470.
[21]
Juan Echeverria and Shi Zhou. 2017. Discovery, Retrieval, and Analysis of the’Star Wars’ Botnet in Twitter. In Proceedings of the 2017 IEEE/ACM international conference on advances in social networks analysis and mining 2017. 1–8.
[22]
Emilio Ferrara, Onur Varol, Clayton Davis, Filippo Menczer, and Alessandro Flammini. 2016. The rise of social bots. Commun. ACM 59, 7 (2016), 96–104.
[23]
Jane Im, Eshwar Chandrasekharan, Jackson Sargent, Paige Lighthammer, Taylor Denby, Ankit Bhargava, Libby Hemphill, David Jurgens, and Eric Gilbert. 2019. Still out there: Modeling and Identifying Russian Troll Accounts on Twitter. arXiv preprint arXiv:1901.11162(2019).
[24]
Sarah J Jackson, Moya Bailey, and Brooke Foucault Welles. 2018. # GirlsLikeUs: Trans advocacy and community building online. New Media & Society 20, 5 (2018), 1868–1888.
[25]
Zhiwei Jin, Juan Cao, Han Guo, Yongdong Zhang, Yu Wang, and Jiebo Luo. 2017. Detection and analysis of 2016 us presidential election related rumors on twitter. In International conference on social computing, behavioral-cultural modeling and prediction and behavior representation in modeling and simulation. Springer, 14–24.
[26]
Huyen Le, GR Boynton, Zubair Shafiq, and Padmini Srinivasan. 2019. A Postmortem of Suspended Twitter Accounts in the 2016 US Presidential Election. (2019).
[27]
Po-Ching Lin and Po-Min Huang. 2013. A study of effective features for detecting long-surviving Twitter spam accounts. In 2013 15th International Conference on Advanced Communications Technology (ICACT). IEEE, 841–846.
[28]
Amanda Minnich, Nikan Chavoshi, Danai Koutra, and Abdullah Mueen. 2017. BotWalk: Efficient adaptive exploration of Twitter bot networks. In Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017. 467–474.
[29]
Mark EJ Newman. 2006. Modularity and community structure in networks. Proceedings of the national academy of sciences 103, 23(2006), 8577–8582.
[30]
Jacob Ratkiewicz, Michael D Conover, Mark Meiss, Bruno Gonçalves, Alessandro Flammini, and Filippo Menczer Menczer. 2011. Detecting and tracking political abuse in social media. In Fifth international AAAI conference on weblogs and social media.
[31]
Caitlin M Rivers and Bryan L Lewis. 2014. Ethical research standards in a world of big data. F1000Research 3(2014).
[32]
Kurt Thomas, Chris Grier, Dawn Song, and Vern Paxson. 2011. Suspended accounts in retrospect: an analysis of twitter spam. In Proceedings of the 2011 ACM SIGCOMM conference on Internet measurement conference. ACM, 243–258.
[33]
Onur Varol, Emilio Ferrara, Clayton A Davis, Filippo Menczer, and Alessandro Flammini. 2017. Online human-bot interactions: Detection, estimation, and characterization. In Eleventh international AAAI conference on web and social media.
[34]
Svitlana Volkova and Eric Bell. 2017. Identifying effective signals to predict deleted and suspended accounts on twitter across languages. In Eleventh International AAAI Conference on Web and Social Media.
[35]
Wei Wei, Kenneth Joseph, Huan Liu, and Kathleen M Carley. 2015. The fragility of Twitter social networks against suspended users. In 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM). IEEE, 9–16.
[36]
Chao Yang, Robert Harkreader, Jialong Zhang, Seungwon Shin, and Guofei Gu. 2012. Analyzing spammers’ social networks for fun and profit: a case study of cyber criminal ecosystem on twitter. In Proceedings of the 21st international conference on World Wide Web. 71–80.
[37]
Savvas Zannettou, Tristan Caulfield, William Setzer, Michael Sirivianos, Gianluca Stringhini, and Jeremy Blackburn. 2019. Who Let The Trolls Out?: Towards Understanding State-Sponsored Trolls. In Proceedings of the 10th ACM Conference on Web Science. ACM, 353–362.

Cited By

View all
  • (2025)Navigating Early-Stage Migration: Comparing QAnon Community Dynamics on Twitter, Parler, and DotwinCompanion Proceedings of the 2025 ACM International Conference on Supporting Group Work10.1145/3688828.3699641(54-57)Online publication date: 12-Jan-2025
  • (2024)Social Media as an Agent of Influence: Twitter Bots in Russia - Ukraine WarGüvenlik Stratejileri Dergisi10.17752/guvenlikstrtj.139670520:47(99-122)Online publication date: 26-Apr-2024
  • (2024)Bots, Elections, and Controversies: Twitter Insights from Brazil's Polarised ElectionsProceedings of the ACM Web Conference 202410.1145/3589334.3645651(2651-2659)Online publication date: 13-May-2024
  • Show More Cited By

Index Terms

  1. On Twitter Purge: A Retrospective Analysis of Suspended Users
            Index terms have been assigned to the content through auto-classification.

            Recommendations

            Comments

            Information & Contributors

            Information

            Published In

            cover image ACM Conferences
            WWW '20: Companion Proceedings of the Web Conference 2020
            April 2020
            854 pages
            ISBN:9781450370240
            DOI:10.1145/3366424
            Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

            Sponsors

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            Published: 20 April 2020

            Permissions

            Request permissions for this article.

            Check for updates

            Author Tags

            1. Abuse
            2. Social Networks
            3. Suspension

            Qualifiers

            • Research-article
            • Research
            • Refereed limited

            Conference

            WWW '20
            Sponsor:
            WWW '20: The Web Conference 2020
            April 20 - 24, 2020
            Taipei, Taiwan

            Acceptance Rates

            Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

            Contributors

            Other Metrics

            Bibliometrics & Citations

            Bibliometrics

            Article Metrics

            • Downloads (Last 12 months)21
            • Downloads (Last 6 weeks)4
            Reflects downloads up to 02 Mar 2025

            Other Metrics

            Citations

            Cited By

            View all
            • (2025)Navigating Early-Stage Migration: Comparing QAnon Community Dynamics on Twitter, Parler, and DotwinCompanion Proceedings of the 2025 ACM International Conference on Supporting Group Work10.1145/3688828.3699641(54-57)Online publication date: 12-Jan-2025
            • (2024)Social Media as an Agent of Influence: Twitter Bots in Russia - Ukraine WarGüvenlik Stratejileri Dergisi10.17752/guvenlikstrtj.139670520:47(99-122)Online publication date: 26-Apr-2024
            • (2024)Bots, Elections, and Controversies: Twitter Insights from Brazil's Polarised ElectionsProceedings of the ACM Web Conference 202410.1145/3589334.3645651(2651-2659)Online publication date: 13-May-2024
            • (2024)Multifaceted online coordinated behavior in the 2020 US presidential electionEPJ Data Science10.1140/epjds/s13688-024-00467-013:1Online publication date: 19-Apr-2024
            • (2024)The Geopolitics of Deplatforming: A Study of Suspensions of Politically-Interested Iranian Accounts on TwitterPolitical Communication10.1080/10584609.2024.230650341:3(413-434)Online publication date: 14-Feb-2024
            • (2023)Detecting and identifying the reasons for deleted tweets before they are postedFrontiers in Artificial Intelligence10.3389/frai.2023.12197676Online publication date: 29-Sep-2023
            • (2023)“Spreader of conspiracies”: How major U.S. newspapers differentiate them from FacebookNewspaper Research Journal10.1177/0739532923118763444:3(340-360)Online publication date: 26-Jul-2023
            • (2023)Misbehavior and Account Suspension in an Online Financial Communication PlatformProceedings of the ACM Web Conference 202310.1145/3543507.3583385(2686-2697)Online publication date: 30-Apr-2023
            • (2023)Twitter Newcomers: Uncovering the Behavior and Fate of New Accounts Through Early Detection and MonitoringIEEE Access10.1109/ACCESS.2023.328258011(55223-55232)Online publication date: 2023
            • (2022)BlackLivesMatter 2020: An Analysis of Deleted and Suspended Users in TwitterProceedings of the 14th ACM Web Science Conference 202210.1145/3501247.3531539(290-295)Online publication date: 26-Jun-2022
            • Show More Cited By

            View Options

            Login options

            View options

            PDF

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader

            HTML Format

            View this article in HTML Format.

            HTML Format

            Figures

            Tables

            Media

            Share

            Share

            Share this Publication link

            Share on social media