skip to main content
10.1145/3366424.3385777acmconferencesArticle/Chapter ViewAbstractPublication PagesthewebconfConference Proceedingsconference-collections
research-article

Disinformation from the Inside: Combining Machine Learning and Journalism to Investigate Sockpuppet Campaigns

Published: 20 April 2020 Publication History

Abstract

This paper brings together machine learning and investigative journalism to examine sockpuppets accounts, a historical breed of fake accounts that are non-automated and human-controlled. Due to their flexible and human-centered nature, sockpuppets pose a complication for purely technological approaches to detecting and studying fake accounts. We find that as machine learning-based detection methods of bots slowly grow stronger, adversaries engaging in disinformation are turning to such sockpuppets accounts, and in particular a subset of sockpuppets that we call “infiltrators” — those that aim to integrate into a community in order spread disinformation. This represents a new stage in the evolution of the sockpuppet concept: where bots seek to simulate audiences and drown online social media platforms with a particular point of view, infiltrators seek to persuade and assimilate genuine audiences from within. In addition to these insights into infiltrator sockpuppets, combining machine learning and investigative journalism enables learning something more than detection and important patterns of activity: it can also gain a sense of the motivations and reasoning of adversaries who engage in disinformation.

References

[1]
2019. CIA World Factbook: Kyrgyzstan Country Profile.
[2]
Frank Ahrens. 2006. “‘Puppets’ Emerge as Internet’s Effective, and Deceptive, Salesmen”. Washington Post (2006).
[3]
Nurlan Aliyev. 2019. “Protest Against Chinese Migrants in Kyrgyzstan: Sinophobia or Demands for Social Justice?”. CACI Analyst (2019). https://www.cacianalyst.org/publications/analytical-articles/item/13568-protest-against-chinese-migrants-in-kyrgyzstan-sinophobia-or-demands-for-social-justice?.html
[4]
DR Analytica. 2017. “2016 ICT Sector Overview for Kyrgyzstan”. https://analytica.digital.report/wp-content/uploads/2017/07/Kyrgyzstan-The-2016-ICT-Sector-Overview.pdf
[5]
Radio Azattyk, OCCRP, and Kloop.kg. 2019. “Plunder and Patronage in the Heart of Central Asia”. (2019). https://www.occrp.org/en/plunder-and-patronage/
[6]
Poland Bailey. 2016. Haters: Harassment, Abuse, and Violence Online. University of Nebraska Press.
[7]
Ermek Baisalov. 2019. “How to Cope with Disinformation in Kyrgyzstan?”. CABAR.asia (2019). https://cabar.asia/en/how-to-cope-with-disinformation-in-kyrgyzstan/
[8]
Marco Bastos and Shawn T. Walker. 2018. “Facebook’s data lockdown is a disaster for academic researchers”. The Conversation (2018). http://theconversation.com/facebooks-data-lockdown-is-a-disaster-for-academic-researchers-94533
[9]
BBC. 2017. “Kyrgyzstan Country Media Profile”. (2017). https://www.bbc.com/news/world-asia-16187183
[10]
Reporters Without Borders. 2020. “Absurd lawsuit against media outlets over corruption exposé in Kyrgyzstan”. (2020).
[11]
Andrew Chadwick, Cristian Vaccari, and Ben O’Loughlin. 2018. “Do Tabloids Poison the Well of Social Media? Explaining Democratically-Dysfunctional News Sharing”. News Media & Society(2018).
[12]
Yimin Chen, Niall J. Conroy, and Victoria L. Rubin. 2015. “Misleading Online Content: Recognizing Clickbait as ‘False News’. WMDD ’15: Proceedings of the 2015 ACM on Workshop on Multimodal Deception Detection (2015). https://doi.org/10.1145/2823465.2823467
[13]
Justin Cheng, Michael Bernstein, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec1. 2017. “Anyone Can Become a Troll: Causes of Trolling Behavior in Online Discussions”. CSCW (2017). http://dx.doi.org/10.1145/2998181.2998213
[14]
John R. Douceur. 2002. “The Sybil Attack”. In Peer-to-Peer Systems (Lecture Notes in Computer Science). https://doi.org/10.1007/3-540-45748-8_24
[15]
Brigitte Dufour, Farid Tuhbatullin, and Asia-Pacific Human Rights Information Center. 2012. “Central Asia: Censorship and Control of the Internet and Other New Media”. https://www.hurights.or.jp/archives/focus/section2/2012/03/central-asia-censorship-and-control-of-the-internet-and-other-new-media1.html
[16]
Kamila Eshaliyeva. 2018. “Is anti-Chinese mood growing in Kyrgyzstan?”. openDemocracy.net (2018). https://www.opendemocracy.net/en/odr/anti-chinese-mood-growing-kyrgyzstan/
[17]
Massimo Flore, Alexandra Balahur, Aldo Podavini, and Marco Verile. 2019. “Understanding Citizens’ Vulnerabilities to Disinformation and Data-Driven Propaganda”. JRC Technical Reports(2019).
[18]
Anonymous (for security reasons). 2019. “Kazakhstan Country Profile”. In Nations In Transit. Freedom House.
[19]
Franco Galdini. 2014. “The June 2010 ‘Events’ Four Years On: Past, Present, Future”.
[20]
Zukhra Iakupbaeva. 2017. “Kyrgyzstan: A Haven for Reporters amid Love and Strife”. EurasiaNet.org (2017). https://eurasianet.org/kyrgyzstan-haven-reporters-amid-love-and-strife
[21]
Zukhra Iakupbaeva. 2018. “Minorities in Kyrgyzstan: changed by revolution”. openDemocracy.net (2018). https://www.opendemocracy.net/en/odr/minorities-in-kyrgyzstan/
[22]
Gulnara Ibraeva. 2019. “Kyrgyzstan Country Profile”. In Media Sustainability Index. International Research and Exchanges Board. https://www.irex.org/sites/default/files/pdf/media-sustainability-index-europe-eurasia-2019-kyrgyzstan.pdf
[23]
Garth Jowett and Victoria O’Donnell. 2005. “What Is Propaganda and How Does It Differ From Persuasion?”. In Propaganda and Persuasion. Sage.
[24]
Joshua Kucera. 2014. “U.S. Formally Closes Its Kyrgyzstan Air Base”. EurasiaNet.org (2014). https://eurasianet.org/us-formally-closes-its-kyrgyzstan-air-base
[25]
Bahtiyar Kurambayev, Laura Schwartz-Henderson, and Ken Winneg. 2018. “The Spiral Of Silence on Social Media: Cultures of Self-Censorship Online and Offline in Kyrgyzstan”. Internet Policy Observatory(2018). https://globalnetpolicy.org/spiral-of-silence-kyrgyzstan/
[26]
Peter Leonard. 2019. “Kyrgyzstan: Life in jail looming for ex-president”. (2019).
[27]
Alison MacKenzie1 and Ibrar Bhatt. 2018. “Lies, Bullshit and Fake News: Some Epistemological Concerns”. Postdigital Science and Education(2018). https://doi.org/10.1007/s42438-018-0025-4
[28]
Meerim Maturaimova. 2015. “Memory of Territory as an Ethnic Narrative: Kyrgyz and Uzbek Narratives in Kyrgyzstan”. Master’s thesis. Central European University.
[29]
OSCE-ODIHR. 2018. “Final report on Kyrgyzstan presidential election”. (2018). https://www.osce.org/odihr/elections/kyrgyzstan/374743
[30]
G Pennycook and DG Rand. 2018. “Who falls for fake news? The roles of bullshit receptivity, overclaiming, familiarity, and analytic thinking”. Journal of Personality(2018). https://doi.org/10.1111/jopy.12476
[31]
G Pennycook and DG Rand. 2019. “Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning”. Cognition (2019). https://doi.org/10.1016/j.cognition.2018.06.011
[32]
Katharine Quinn-Judge and Paul Stronski. 2016. “Kyrgyzstan at Twenty-Five: Treading Water”. Carnegie Endowment for Peace. https://carnegieendowment.org/2016/07/21/kyrgyzstan-at-twenty-five-treading-water-pub-64152
[33]
Meet Rajdev and Kyumin Le. 2015. “Fake and Spam Messages: Detecting Misinformation During Natural Disasters on Social Media”. 2015 IEEE / WIC / ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT)(2015). https://doi.org/10.1109/WI-IAT.2015.102
[34]
Christopher Rickleton. 2020. “China’s virus storm over Central Asia: Panic is being sown by trouble-making social media users.”. EurasiaNet.org (2020). https://eurasianet.org/chinas-virus-storm-over-central-asia
[35]
Christopher Rickleton and Nurjamal Djanibekova. 2019. “Internet provides new space for Kyrgyzstan’s north-south divide: Online disinformation is manipulating regional divisions.”. (2019).
[36]
Christopher Schwartz. 2015. Field notes.
[37]
Christopher Schwartz. 2019. Piala interview notes.
[38]
Christopher Schwartz and Alisher Khamidov. 2016. “Kyrgyzstan: Corrupt, Anarchic – and Stable?”. The Diplomat (2016).
[39]
Christopher Schwartz and Rebekah Overdorf. 2019. “Subtle Censorship Via Adversarial Fakeness in Kyrgyzstan”. In 19th Privacy Enhancing Technologies Symposium. https://arxiv.org/abs/1906.08021
[40]
Inga Sikorskaya. [n.d.]. “A brief history of conflict in Kyrgyzstan”. Peace Insight ([n. d.]). https://www.peaceinsight.org/blog/2015/09/a-brief-history-of-conflict-in-kyrgyzstan/
[41]
Sputnik.kg. 2019. russian“Страница в поддержку Кашкара Джунушалиева появилась в Instagram”. (2019). https://ru.sputnik.kg/society/20190817/1045465013/kashkar-dzhunushaliev-instagram-podderzhka.html
[42]
Edson Tandoc, Zheng Wei Lim, and Rich Ling. 2017. Defining ‘Fake News’: A typology of scholarly definitions. Digital Journalism(2017). https://doi.org/10.1080/21670811.2017.1360143
[43]
Committee to Protect Journalists. 2007. “Alisher Saipov Profile”. (2007).

Cited By

View all
  • (2025)Systematic Review of Fake News, Propaganda, and Disinformation: Examining Authors, Content, and Social Impact Through Machine LearningIEEE Access10.1109/ACCESS.2025.353068813(17583-17629)Online publication date: 2025
  • (2022)Datavoidant: An AI System for Addressing Political Data Voids on Social MediaProceedings of the ACM on Human-Computer Interaction10.1145/35556166:CSCW2(1-29)Online publication date: 11-Nov-2022

Index Terms

  1. Disinformation from the Inside: Combining Machine Learning and Journalism to Investigate Sockpuppet Campaigns
            Index terms have been assigned to the content through auto-classification.

            Recommendations

            Comments

            Information & Contributors

            Information

            Published In

            cover image ACM Conferences
            WWW '20: Companion Proceedings of the Web Conference 2020
            April 2020
            854 pages
            ISBN:9781450370240
            DOI:10.1145/3366424
            Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

            Sponsors

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            Published: 20 April 2020

            Permissions

            Request permissions for this article.

            Check for updates

            Author Tags

            1. Disinformation
            2. Social Networks
            3. Sock Puppets

            Qualifiers

            • Research-article
            • Research
            • Refereed limited

            Conference

            WWW '20
            Sponsor:
            WWW '20: The Web Conference 2020
            April 20 - 24, 2020
            Taipei, Taiwan

            Acceptance Rates

            Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

            Contributors

            Other Metrics

            Bibliometrics & Citations

            Bibliometrics

            Article Metrics

            • Downloads (Last 12 months)24
            • Downloads (Last 6 weeks)1
            Reflects downloads up to 16 Feb 2025

            Other Metrics

            Citations

            Cited By

            View all
            • (2025)Systematic Review of Fake News, Propaganda, and Disinformation: Examining Authors, Content, and Social Impact Through Machine LearningIEEE Access10.1109/ACCESS.2025.353068813(17583-17629)Online publication date: 2025
            • (2022)Datavoidant: An AI System for Addressing Political Data Voids on Social MediaProceedings of the ACM on Human-Computer Interaction10.1145/35556166:CSCW2(1-29)Online publication date: 11-Nov-2022

            View Options

            Login options

            View options

            PDF

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader

            HTML Format

            View this article in HTML Format.

            HTML Format

            Figures

            Tables

            Media

            Share

            Share

            Share this Publication link

            Share on social media