Yubo Kou
Skip Abstract Section
Abstract
Platforms face the challenge of managing toxic behaviors such as flaming, hateful remarks, and harassment. To discipline their users, platforms usually adopt a punitive approach that issues punishments ranging from a warning message to content removal to permanent ban (PB). As the severest punishment, PB deprives the user of their privileges on the platform, such as account access and purchased content. But little is known regarding the experiential side of PB within the user community. In this study, we analyzed PB in League of Legends, one of the largest online games today. We argue that what PB does is not precisely to discipline players into well-behaved community members. Rather, PB functions to produce the stereotype of "the most toxic player" in the community and is best seen as a platform rhetoric. We further discuss the need to contextualize toxicity from the restorative lens.
- Angela Adrian. 2010. Mischief and grief: virtual torts or real problems? Int. J. Priv. Law 3, 1/2 (2010), 70. DOI:https://doi.org/10.1504/IJPL.2010.029603Google Scholar
Cross Ref
- Mariam Asad. 2019. Prefigurative design as a method for research justice. Proc. ACM Human-Computer Interact. 3, CSCW (November 2019), 41. DOI:https://doi.org/10.1145/3359302Google ScholarDigital Library
- Hugo Adam Bedau and Erin Kelly. 2019. Punishment. The Stanford Encyclopedia of Philosophy (Winter 2019 Edition). Retrieved from https://plato.stanford.edu/archives/win2019/entries/punishment/Google Scholar
- Lindsay Blackwell, Jill Dimond, Sarita Schoenebeck, and Cliff Lampe. 2017. Classification and Its Consequences for Online Harassment: Design Insights from HeartMob. Proc. ACM Human-Computer Interact. 1, CSCW (2017), 24. DOI:https://doi.org/10.1145/3134659Google ScholarDigital Library
- Geoffrey C. Bowker and Susan Leigh Star. 2000. Sorting Things Out: Classification and Its Consequences. MIT Press.Google ScholarDigital Library
- Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qual. Res. Psychol. 3, 2 (January 2006), 77--101. DOI:https://doi.org/10.1191/1478088706qp063oaGoogle ScholarCross Ref
- Virginia Braun and Victoria Clarke. 2019. To saturate or not to saturate? Questioning data saturation as a useful concept for thematic analysis and sample-size rationales. Qualitative Research in Sport, Exercise and Health. DOI:https://doi.org/10.1080/2159676X.2019.1704846Google ScholarCross Ref
- Colin Campbell. 2014. How Riot Games encourages sportsmanship in League of Legends. polygon. Retrieved from https://www.polygon.com/2014/3/20/5529784/how-riot-games-encourages-sportsmanship-in-league-of-legendsGoogle Scholar
- Edward Castronova. 2005. Synthetic Worlds: The Business And Culture of Online Games. University of Chicago Press, Chicago. Retrieved from http://www.amazon.com/Synthetic-Worlds-Business-Culture-Online/dp/0226096270Google ScholarDigital Library
- Stevie Chancellor, Niloufar Salehi, Shion Guha, Sarita Schoenebeck, Jofish Kaye, Elizabeth Stowell, and Jen King. 2019. The relationships between data, power, and justice in CSCW research. In Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW, 102--105. DOI:https://doi.org/10.1145/3311957.3358609Google ScholarDigital Library
- Eshwar Chandrasekharan, Umashanthi Pavalanathan, Anirudh Srinivasan, Adam Glynn, Jacob Eisenstein, and Eric Gilbert. 2017. You Can't Stay Here: The Efficacy of Reddit's 2015 Ban Examined Through Hate Speech. Proc. ACM Human-Computer Interact. 1, CSCW (2017), 31. DOI:https://doi.org/10.1145/3134666Google ScholarDigital Library
- Justin Cheng, Michael Bernstein, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2017. Anyone Can Become a Troll: Causes of Trolling Behavior in Online Discussions. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing - CSCW '17, 1217--1230. DOI:https://doi.org/10.1145/2998181.2998213Google ScholarDigital Library
- Danielle Keats Citron and Benjamin Wittes. 2018. The Problem Isn't Just Backpage: Revising Section 230 Immunity. Georgetown Law Technology Review. Retrieved April 13, 2021 from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3218521Google Scholar
- Kate Conger and Mike Isaac. 2020. Trump's Twitter Account Permanently Suspended. New York Times. Retrieved from https://www.nytimes.com/2021/01/08/technology/twitter-trump-suspended.htmlGoogle Scholar
- Kate Crawford and Tarleton Gillespie. 2016. What is a flag for? Social media reporting tools and the vocabulary of complaint. New Media Soc. 18, 3 (March 2016), 410--428. DOI:https://doi.org/10.1177/1461444814543163Google ScholarCross Ref
- Kord. Davis and Doug. Patterson. 2012. Ethics of big data. O'Reilly.Google Scholar
- Bryan Dosono and Bryan Semaan. 2019. Moderation practices as emotional labor in sustaining online communities: The case of AAPI identity work on reddit. In Conference on Human Factors in Computing Systems - Proceedings. DOI:https://doi.org/10.1145/3290605.3300372Google ScholarDigital Library
- Jenny Fan and Amy X. Zhang. 2020. Digital Juries: A Civics-Oriented Approach to Platform Governance. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1--14. DOI:https://doi.org/10.1145/3313831.3376293Google ScholarDigital Library
- Chek Yang Foo. 2008. Grief Play Management. VDM Verlag.Google Scholar
- Andrea Forte, Vanesa Larco, and Amy Bruckman. 2009. Decentralization in Wikipedia Governance. J. Manag. Inf. Syst. 26, 1 (July 2009), 49--72. DOI:https://doi.org/10.2753/MIS0742--1222260103Google ScholarDigital Library
- Michel Foucault. 1972. The Archaeology of Knowledge. Tavistock Publications Limited.Google Scholar
- Michel Foucault. 1977. Discipline and Punish: The Birth of the Prison. Vintage Books.Google Scholar
- Michel Foucault. 1988. The History of Sexuality, Vol. 3: The Care of the Self (First Vint ed.). Vintage.Google Scholar
- Christian Fuchs. 2012. The Political Economy of Privacy on Facebook. Telev. New Media 13, 2 (March 2012), 139--159. DOI:https://doi.org/10.1177/1527476411415699Google ScholarCross Ref
- Riot Games. 2020. SAYING FAREWELL TO BOARDS. LeagueofLegends.com. Retrieved from https://na.leagueoflegends.com/en-us/news/community/saying-farewell-to-boards/Google Scholar
- James Paul Gee. 2014. How to Do Discourse Analysis: A Toolkit. Routledge. Retrieved from https://books.google.com/books/about/How_to_Do_Discourse_Analysis.html?id=O4SrAgAAQBAJGoogle Scholar
- Ysabel Gerrard. 2018. Beyond the hashtag: Circumventing content moderation on social media. New Media Soc. 20, 12 (December 2018), 4492--4511. DOI:https://doi.org/10.1177/1461444818776611Google ScholarCross Ref
- Tarleton Gillespie. 2018. Custodians of the internet?: platforms, content moderation, and the hidden decisions that shape social media. Yale University Press.Google Scholar
- Eric Goldman. 2021. Dear President Biden: You should save, not revoke, Section 230. Bull. At. Sci. 77, 1 (2021), 36--37. DOI:https://doi.org/10.1080/00963402.2020.1859863Google ScholarCross Ref
- Colin M. Gray, Yubo Kou, Bryan Battles, Joseph Hoggatt, and Austin L. Toombs. 2018. The Dark (Patterns) Side of UX Design. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18, 1--14. DOI:https://doi.org/10.1145/3173574.3174108Google ScholarDigital Library
- James Grimmelmann. 2015. The Virtues of Moderation. Yale J. Law Technol. 17, (2015). Retrieved September 2, 2019 from https://heinonline.org/HOL/Page?handle=hein.journals/yjolt17&id=42&div=3&collection=journalsGoogle Scholar
- Nicholas-Brie Guarriello. 2019. Never give up, never surrender: Game live streaming, neoliberal work, and personalized media economies. New Media Soc. 21, 8 (August 2019), 1750--1769. DOI:https://doi.org/10.1177/1461444819831653Google ScholarCross Ref
- Renyi Hong. 2013. Game Modding, Prosumerism and Neoliberal Labor Practices. International Journal of Communication 7, 19. Retrieved September 15, 2020 from https://ijoc.org/index.php/ijoc/article/view/1659Google Scholar
- Sal Humphreys. 2008. Ruling the virtual world Governance in massively multiplayer online games. Eur. J. Cult. Stud. 11, 2 (May 2008), 149--171. DOI:https://doi.org/10.1177/1367549407088329Google ScholarCross Ref
- Sal Humphreys and Melissa de Zwart. 2012. Griefing, Massacres, Discrimination, and Art: The Limits of Overlapping Rule Sets in Online Games. UC Irvine Law Rev. 2, 2 (2012).Google Scholar
- Itsumo. 2016. Instant Feedback System FAQ. riotgames.com. Retrieved from https://support-leagueoflegends.riotgames.com/hc/en-us/articles/207489286-Instant-Feedback-System-FAQ-Google Scholar
- Shagun Jhaver, Darren Scott Appling, Eric Gilbert, and Amy Bruckman. 2019. "Did You Suspect the Post Would be Removed?": Understanding User Reactions to Content Removals on Reddit. Proc. ACM Human-Computer Interact. 3, CSCW (November 2019), 1--33. DOI:https://doi.org/10.1145/3359294Google ScholarDigital Library
- Shagun Jhaver, Iris Birman, Eric Gilbert, and Amy Bruckman. 2019. Human-Machine Collaboration for Content Regulation: The Case of Reddit Automoderator. ACM Trans. Comput. Interact. 26, 5 (July 2019), 1--35. DOI:https://doi.org/10.1145/3338243Google ScholarDigital Library
- Shagun Jhaver, Amy Buckman, and Eric Gilbert. 2019. Does Transparency in Moderation Really Matter?: User Behavior After Content Removal Explanations on Reddit. Proc. ACM Human-Computer Interact. 3, CSCW (2019), 27.Google ScholarDigital Library
- Shagun Jhaver, Sucheta Ghoshal, Amy Bruckman, and Eric Gilbert. 2018. Online Harassment and Content Moderation: The Case of Blocklists. ACM Trans. Comput. Interact. 25, 2 (March 2018), 1--33. DOI:https://doi.org/10.1145/3185593Google ScholarDigital Library
- Charles Kiene, Donghee Yvette Wohn, Bryan Dosono, Kenny Shores, Eshwar Chandrasekharan, Shagun Jhaver, Jialun "Aaron" Jiang, Brianna Dym, Joseph Seering, Sarah Gilbert, and Kat Lo. 2019. Volunteer Work: Mapping the Future of Moderation Research. In Conference Companion Publication of the 2019 on Computer Supported Cooperative Work and Social Computing - CSCW '19, 492--497. DOI:https://doi.org/10.1145/3311957.3359443Google ScholarDigital Library
- Jason Koebler and Joseph Cox. 2018. Content Moderator Sues Facebook, Says Job Gave Her PTSD. vice. Retrieved from https://motherboard.vice.com/en_us/article/zm5mw5/facebook-content-moderation-lawsuit-ptsdGoogle Scholar
- Adam J. Kolber. 2009. The Subjective Experience of Punishment. Columbia Law Rev. 109, (2009). Retrieved March 15, 2020 from https://heinonline.org/HOL/Page?handle=hein.journals/clr109&id=186&div=7&collection=journalsGoogle Scholar
- Yubo Kou. 2020. Toxic Behaviors in Team-Based Competitive Gaming: The Case of League of Legends. In Proceedings of the 2020 Annual Symposium on Computer-Human Interaction in Play - CHI PLAY '20, 81--92. DOI:https://doi.org/10.1145/3410404.3414243Google ScholarDigital Library
- Yubo Kou and Xinning Gui. 2018. Entangled with Numbers: Quantified Self and Others in a Team-Based Online Game. Proc. ACM Human-Computer Interact. 2, CSCW (November 2018), 1--25. DOI:https://doi.org/10.1145/3274362Google ScholarDigital Library
- Yubo Kou and Xinning Gui. 2021. Flag and Flaggability in Automated Moderation: the Case of Reporting Toxic Behavior in an Online Game Community. In Proceedings of the ACM SIGCHI Conference on Human factors in computing systems.Google Scholar
- Yubo Kou, Xinning Gui, and Yong Ming Kow. 2016. Ranking Practices and Distinction in League of Legends. In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play - CHI PLAY '16, 4--9. DOI:https://doi.org/10.1145/2967934.2968078Google ScholarDigital Library
- Yubo Kou, Xinning Gui, Shaozeng Zhang, and Bonnie Nardi. 2017. Managing Disruptive Behavior through Non-Hierarchical Governance: Crowdsourcing in League of Legends and Weibo. Proc. ACM Human-Computer Interact. 1, CSCW (2017), 62. DOI:https://doi.org/10.1145/3134697Google ScholarDigital Library
- Yubo Kou and Bonnie Nardi. 2014. Governance in League of Legends: A Hybrid System. In Foundations of Digital Games.Google Scholar
- Haewoon Kwak and Jeremy Blackburn. 2014. Linguistic Analysis of Toxic Behavior in an Online Video Game. In International Conference on Social Informatics.Google Scholar
- Amanda Lazar, Norman Makoto Su, Jefrey Bardzell, and Shaowen Bardzell. 2019. Parting the Red Sea: Sociotechnical systems and lived experiences of menopause. In Conference on Human Factors in Computing Systems - Proceedings, 1--16. DOI:https://doi.org/10.1145/3290605.3300710Google ScholarDigital Library
- Alex Leavitt. 2015. "This is a Throwaway Account": Temporary Technical Identities and Perceptions of Anonymity in a Massive Online Community. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing - CSCW '15, 317--327. DOI:https://doi.org/10.1145/2675133.2675175Google ScholarDigital Library
- Richard Lewis. 2015. A look at the relationship between Riot Games and the League of Legends subreddit. dotesports. Retrieved from https://dotesports.com/league-of-legends/news/riot-games-league-of-legends-subreddit-relationship-1606Google Scholar
- Jeffrey Lin. 2013. The Science Behind Shaping Player Behavior in Online Games. In Game Developers Conference.Google Scholar
- Lyte. 2014. Upgrading the Tribunal. Player Behavior. Retrieved from http://na.leagueoflegends.com/en/news/game-updates/player-behavior/upgrading-tribunalGoogle Scholar
- J. Nathan Matias. 2016. Going Dark: Social Factors in Collective Action Against Platform Operators in the Reddit Blackout. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems - CHI '16, 1138--1151. DOI:https://doi.org/10.1145/2858036.2858391Google ScholarDigital Library
- Chip Morningstar and F. Randall Farmer. 1991. The Lessons of Lucasfilm's Habitat. In Cyberspace: First Steps, Michael Benedikt (ed.). MIT Press, Cambridge, MA, USA, 273--302.Google ScholarDigital Library
- Casey Newton. 2019. The secret lives of Facebook moderators in America. The Verge. Retrieved September 3, 2019 from https://www.theverge.com/2019/2/25/18229714/cognizant-facebook-content-moderator-interviews-trauma-working-conditions-arizonaGoogle Scholar
- Arielle Pardes. 2020. All the Social Media Giants Are Becoming the Same. Wired Magazine. Retrieved from https://www.wired.com/story/social-media-giants-look-the-same-tiktok-twitter-instagram/Google Scholar
- Simon Parkin. 2015. A Video-Game Algorithm to Solve Online Abuse. MIT Technology Review'. Retrieved from https://www.technologyreview.com/s/541151/a-video-game-algorithm-to-solve-online-abuse/Google Scholar
- Christopher A Paul. 2018. The Toxic Meritocracy of Video Games: Why Gaming Culture Is the Worst. University of Minnesota Press.Google Scholar
- Riot Games. 2020. Riot Games® Terms of Service. riotgames.com. Retrieved from https://www.riotgames.com/en/terms-of-serviceGoogle Scholar
- RiotTyphon. 2019. Understanding Permanent Bans. League of Legends Support. Retrieved from https://support-leagueoflegends.riotgames.com/hc/en-us/articles/360038430973-Understanding-Permanent-BansGoogle Scholar
- Sarah T. Roberts. 2016. Commercial Content Moderation: Digital Laborers' Dirty Work. Media Stud. Publ. (January 2016).Google Scholar
- Sarah T. Roberts. 2019. Behind the screen?: content moderation in the shadows of social media. Yale University Press.Google Scholar
- Bonnie Ruberg, Amanda L.L. Cullen, and Kathryn Brewster. 2019. Nothing but a "titty streamer": legitimacy, labor, and the debate over women's breasts in video game live streaming. Crit. Stud. Media Commun. 36, 5 (October 2019), 466--481. DOI:https://doi.org/10.1080/15295036.2019.1658886Google ScholarCross Ref
- Christoph Schneider, Markus Weinmann, and Jan vom Brocke. 2018. Digital nudging: guiding online user choices through interface design. Commun. ACM 61, 7 (June 2018), 67--73. DOI:https://doi.org/10.1145/3213765Google ScholarDigital Library
- Nathan Schneider, Primavera De Filippi, Seth Frey, Joshua Z. Tan, and Amy X. Zhang. 2020. Modular Politics: Toward a Governance Layer for Online Communities. arXiv (May 2020). DOI:https://doi.org/10.1145/3449090Google ScholarDigital Library
- Sarita Schoenebeck, Oliver L Haimson, and Lisa Nakamura. 2020. Drawing from justice theories to support targets of online harassment. New Media Soc. (March 2020), 146144482091312. DOI:https://doi.org/10.1177/1461444820913122Google ScholarCross Ref
- Ori Schwarz. 2019. Facebook Rules: Structures of Governance in Digital Capitalism and the Control of Generalized Social Capital. Theory, Cult. Soc. 36, 4 (July 2019), 117--141. DOI:https://doi.org/10.1177/0263276419826249Google ScholarCross Ref
- Joseph Seering. 2020. Reconsidering Community Self-Moderation: The Role of Research in Supporting Community-Based Models for Online Content Moderation. Proc. ACM Human-Computer Interact. 4, CSCW2 (October 2020), 28. DOI:https://doi.org/10.1145/3415178Google ScholarDigital Library
- Kenneth B. Shores, Yilin He, Kristina L. Swanenburg, Robert Kraut, and John Riedl. 2014. The identification of deviance and its impact on retention in a multiplayer game. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing - CSCW '14, 1356--1365. DOI:https://doi.org/10.1145/2531602.2531724Google ScholarDigital Library
- Peter K Smith, Jess Mahdavi, Manuel Carvalho, Sonja Fisher, Shanette Russell, and Neil Tippett. 2008. Cyberbullying: its nature and impact in secondary school pupils. J. Child Psychol. Psychiatry. 49, 4 (April 2008), 376--385. DOI:https://doi.org/10.1111/j.1469--7610.2007.01846.xGoogle ScholarCross Ref
- Stefan Stieglitz, Florian Brachten, Björn Ross, and Anna-Katharina Jung. 2017. Do Social Bots Dream of Electric Sheep? A Categorisation of Social Media Bot Accounts. (October 2017). Retrieved November 2, 2019 from http://arxiv.org/abs/1710.04044Google Scholar
- Nicolas Suzor and Darryl Woodford. 2013. Evaluating Consent and Legitimacy Amongst Shifting Community Norms: an EVE Online Case Study. J. Virtual Worlds Res. 6, 3 (September 2013). DOI:https://doi.org/10.4101/jvwr.v6i3.6409Google ScholarCross Ref
- T. L Taylor. 2006. Beyond management: Considering participatory design and governance in player culture. First Monday Special Issue 7 (2006).Google Scholar
- Selen Türkay, Jessica Formosa, Sonam Adinolf, Robert Cuthbert, and Roger Altizer. 2020. See No Evil, Hear No Evil, Speak No Evil: How Collegiate Players Define, Experience and Cope with Toxicity. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1--13. DOI:https://doi.org/10.1145/3313831.3376191Google ScholarDigital Library
- Marley W. Watkins and Miriam Pacheco. 2000. Interobserver agreement in behavioral research: Importance and calculation. J. Behav. Educ. 10, 4 (2000), 205--212. DOI:https://doi.org/10.1023/A:1012295615144Google ScholarCross Ref
- Chris Weedon. 1998. Feminism & the Principles of Poststructuralism. In Cultural Theory and Popular Culture: A Reader. Pearson Education, 172--184.Google Scholar
- Michael Wenzel, Tyler G. Okimoto, Norman T. Feather, and Michael J. Platow. 2008. Retributive and restorative justice. Law and Human Behavior 32, 375--389. DOI:https://doi.org/10.1007/s10979-007--9116--6Google ScholarCross Ref
- Donghee Yvette Wohn. 2019. Volunteer moderators in twitch micro communities: How they get involved, the roles they play, and the emotional labor they experience. In Conference on Human Factors in Computing Systems - Proceedings. DOI:https://doi.org/10.1145/3290605.3300390Google ScholarDigital Library
- Bingjie Yu, Katta Spiel, and Leon Watts. Supporting Care as a Layer of Concern: Nurturing Attitudes in Online Community Moderation. Retrieved August 31, 2019 from https://www.metafilter.comGoogle Scholar
- Shoshana Zuboff. The age of surveillance capitalism?: the fight for a human future at the new frontier of power.Google Scholar
- Mark Zuckerberg. 2018. A Blueprint for Content Governance and Enforcement. Retrieved from https://www.facebook.com/notes/mark-zuckerberg/a-blueprint-for-content-governance-and-enforcement/10156443129621634/Google Scholar
Index Terms
- Punishment and Its Discontents: An Analysis of Permanent Ban in an Online Game Community
Recommendations
Flag and Flaggability in Automated Moderation: The Case of Reporting Toxic Behavior in an Online Game Community
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing SystemsOnline platforms rely upon users or automated tools to flag toxic behaviors, the very first step in online moderation. While much recent research has examined online moderation, the role of flag remains poorly understood. This question becomes even more ...
Prosocial behavior in an online game community: an ethnographic study
FDG '17: Proceedings of the 12th International Conference on the Foundations of Digital GamesWe report an ethnographic study of prosocial behavior inconnection to League of Legends, one of the most popular games in the world. In this game community, the game developer, Riot Games, implemented a system that allowed players to volunteer their ...
Playing with Streakiness in Online Games: How Players Perceive and React to Winning and Losing Streaks in League of Legends
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing SystemsStreakiness refers to observed tendency towards consecutive appearances of particular patterns. In video games, streakiness is oftentimes inevitable, where a player keeps winning or losing for a short period. However, the phenomenon remains understudied ...
Comments