skip to main content
research-article
Public Access

CrowdSolve: Managing Tensions in an Expert-Led Crowdsourced Investigation

Published:22 April 2021Publication History
Skip Abstract Section

Abstract

Investigators in fields such as journalism and law enforcement have long sought the public's help with investigations. New technologies have also allowed amateur sleuths to lead their own crowdsourced investigations - that have traditionally only been the purview of expert investigators - with mixed results. Through an ethnographic study of a four-day, co-located event with over 250 attendees, we examine the human infrastructure responsible for enabling the success of an expert-led crowdsourced investigation. We find that the experts enabled attendees to generate useful leads; the attendees formed a community around the event; and the victims' families felt supported. Additionally, the co-located setting, legal structures, and emergent social norms impacted collaborative work practice. We also surface three important tensions to consider in future investigations and provide design recommendations to manage these tensions.

References

  1. 2007. Help Find Jim Gray With Web 2.0. https://techcrunch.com/2007/02/03/help-find-jim-gray-with-web-20/Google ScholarGoogle Scholar
  2. 2015. The evolution of the wanted poster. https://www.sfgate.com/news/article/The-evolution-of-the-wanted-poster-6110088.phpGoogle ScholarGoogle Scholar
  3. 2019. Amnesty Decoders Strike Tracker. https://decoders.amnesty.org/projects/strike-trackerGoogle ScholarGoogle Scholar
  4. 2019. Stop Child Abuse ? Trace an Object. https://www.europol.europa.eu/stopchildabuseGoogle ScholarGoogle Scholar
  5. 2020. Top 100 US Podcasts (Apple Podcasts Top Charts). https://www.podcastinsights.com/top-us-podcasts/Google ScholarGoogle Scholar
  6. Elena Agapie, Jaime Teevan, and Andrés Monroy-Hernández. 2015. Crowdsourcing in the field: A case study using local crowds for event reporting. In Third AAAI Conference on Human Computation and Crowdsourcing.Google ScholarGoogle ScholarCross RefCross Ref
  7. Kenneth M Anderson, Ahmet Arif Aydin, Mario Barrenechea, Adam Cardenas, Mazin Hakeem, and Sahar Jambi. 2015. Design challenges/solutions for environments supporting the analysis of social media data in crisis informatics research. In 2015 48th Hawaii International Conference on System Sciences. IEEE, 163--172.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Salvatore Andolina, Hendrik Schneider, Joel Chan, Khalil Klouche, Giulio Jacucci, and Steven Dow. 2017. Crowdboard: Augmenting In-Person Idea Generation with Real-Time Crowds. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition (C&C '17). ACM, New York, NY, USA, 106--118. https://doi.org/10.1145/3059454.3059477Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. James Baysinger. [n. d.]. Hide and Seek. https://podcasts.apple.com/us/podcast/hide-and-seek/id1452121762Google ScholarGoogle Scholar
  10. Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology, Vol. 3, 2 (2006), 77--101.Google ScholarGoogle Scholar
  11. L. Elisa Celis, Sai Praneeth Reddy, Ishaan Preet Singh, and Shailesh Vaya. 2016. Assignment Techniques for Crowdsourcing Sensitive Tasks. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (CSCW '16). ACM, New York, NY, USA, 836--847. https://doi.org/10.1145/2818048.2835202Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Joel Chan, Steven Dang, and Steven P. Dow. 2016a. Comparing Different Sensemaking Approaches for Large-Scale Ideation. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 2717--2728. https://doi.org/10.1145/2858036.2858178Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Joel Chan, Steven Dang, and Steven P. Dow. 2016b. Improving Crowd Innovation with Expert Facilitation. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (CSCW '16). ACM, 1223--1235. https://doi.org/10.1145/2818048.2820023Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Lennon YC Chang and Andy KH Leung. 2015. An introduction to cyber crowdsourcing (human flesh search) in the Greater China region. In Cybercrime Risks and Responses. Springer, 240--252.Google ScholarGoogle Scholar
  15. John Joon Young Chung, Joseph Jay Williams, and Juho Kim. 2018. Collaborative Crowdsourcing Between Experts and Crowds for Chronological Ordering of Narrative Events. HCI (2018), 621--626.Google ScholarGoogle Scholar
  16. Kelly-Leigh Cooper. 2019. Is our growing obsession with true crime a problem? BBC (2019). https://www.bbc.com/news/world-us-canada-47474996Google ScholarGoogle Scholar
  17. Dharma Dailey and Kate Starbird. 2014. Journalists as Crowdsourcerers: Responding to Crisis by Reporting with a Crowd. Computer Supported Cooperative Work (CSCW), Vol. 23, 4 (01 Dec 2014), 445--481. https://doi.org/10.1007/s10606-014-9208-zGoogle ScholarGoogle ScholarDigital LibraryDigital Library
  18. Victor Daniel. 2017. Amateur Sleuths Aim to Identify Charlottesville Marchers, but Sometimes Misfire. https://www.nytimes.com/2017/08/14/us/charlottesville-doxxing.htmlGoogle ScholarGoogle Scholar
  19. J. Davis, J. Arderiu, H. Lin, Z. Nevins, S. Schuon, O. Gallo, and M.H. Yang. 2010. The HPU. In Computer Vision and Pattern Recognition Workshops (CVPRW). 9--16.Google ScholarGoogle Scholar
  20. Katie Derthick, Patrick Tsao, Travis Kriplean, Alan Borning, Mark Zachry, and David W McDonald. 2011. Collaborative sensemaking during admin permission granting in Wikipedia. In International Conference on Online Communities and Social Computing. Springer, 100--109.Google ScholarGoogle ScholarCross RefCross Ref
  21. Caitlin Dewey. 2015. Crowdsourcing may have solved a 20-year-old cold case. The Washington Post (2015). http://www.washingtonpost.com/lifestyle/style/how-theinternet- may-have-solved-a-20-year-old-cold-case/2015/03/10/02a4a326-acb1-11e4-9c91- e9d2f9fde644_story.htmlGoogle ScholarGoogle Scholar
  22. Michaelanne Dye, David Nemer, Josiah Mangiameli, Amy S Bruckman, and Neha Kumar. 2018. El Paquete Semanal: The Week's Internet in Havana. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1--12.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Brianna Dym, Jed R Brubaker, Casey Fiesler, and Bryan Semaan. 2019. "Coming Out Okay" Community Narratives for LGBTQ Identity Recovery Work. Proceedings of the ACM on Human-Computer Interaction, Vol. 3, CSCW (2019), 1--28.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Ray Evangelista. 2019. Rick and Morty and Toxic Fandom: Factors Contributing to Toxic Communication in the Online Rick and Morty Fan Community. Ph.D. Dissertation. Regent University.Google ScholarGoogle Scholar
  25. Sarah Evans, Katie Davis, Abigail Evans, Julie Ann Campbell, David P Randall, Kodlee Yin, and Cecilia Aragon. 2017. More than peer production: fanfiction communities as sites of distributed mentoring. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. 259--272.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Casey Fiesler. 2007. Everything I need to know I learned from fandom: How existing social norms can help shape the next generation of user-generated content. Vand. J. Ent. & Tech. L., Vol. 10 (2007), 729.Google ScholarGoogle Scholar
  27. Casey Fiesler, Shannon Morrison, and Amy S Bruckman. 2016. An archive of their own: a case study of feminist HCI and values in design. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 2574--2585.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Casey Fiesler, Shannon Morrison, R. Benjamin Shapiro, and Amy S. Bruckman. 2017. Growing Their Own: Legitimate Peripheral Participation for Computational Learning in an Online Fandom Community. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17). Association for Computing Machinery, New York, NY, USA, 1375--1386. https://doi.org/10.1145/2998181.2998210Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Kristie Fisher, Scott Counts, and Aniket Kittur. 2012. Distributed Sensemaking: Improving Sensemaking by Leveraging the Efforts of Previous Users. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, 247--256. https://doi.org/10.1145/2207676.2207711Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Rachel Franks. 2016. True Crime: The Regular Reinvention of a Genre. Journal of Asia-Pacific Pop Culture, Vol. 1, 2 (2016), 239--254.Google ScholarGoogle ScholarCross RefCross Ref
  31. Raghu K Ganti, Fan Ye, and Hui Lei. 2011. Mobile crowdsensing: current state and future challenges. IEEE communications Magazine, Vol. 49, 11 (2011), 32--39.Google ScholarGoogle Scholar
  32. Susan Gasson. 2005. The Dynamics of Sensemaking, Knowledge, and Expertise in Collaborative, Boundary-Spanning Design., Vol. 10, 4 (2005). https://doi.org/10.1111/j.1083--6101.2005.tb00277.xGoogle ScholarGoogle ScholarCross RefCross Ref
  33. James Paul Gee. 2003. What video games have to teach us about learning and literacy. Computers in Entertainment (CIE), Vol. 1, 1 (2003), 20--20.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Sara Gentzler. 2019. Man who allegedly confessed to killing Nancy Moyer now says he doesn't recall confessing. The Olympian (Aug. 2019). https://www.theolympian.com/news/local/crime/article234489522.htmlGoogle ScholarGoogle Scholar
  35. Michael Glassman and Min Ju Kang. 2012. Intelligence in the internet age: The emergence and evolution of Open Source Intelligence (OSINT). Computers in Human Behavior, Vol. 28, 2 (2012), 673--682.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Hadass Goldblatt, Orit Karnieli-Miller, and Melanie Neumann. 2011. Sharing qualitative research findings with participants: Study experiences of methodological and ethical dilemmas. Patient education and counseling, Vol. 82, 3 (2011), 389--395.Google ScholarGoogle Scholar
  37. Ricardo Gomez, Kirsten Foot, Meg Young, Rose Paquet-Kinsley, and Stacey Morrison. 2015. Pulling the plug visually: Images of resistance to ICTs and connectivity. First Monday (2015).Google ScholarGoogle Scholar
  38. Joseph A. Gonzales, Casey Fiesler, and Amy Bruckman. 2015. Towards an Appropriable CSCW Tool Ecology: Lessons from the Greatest International Scavenger Hunt the World Has Ever Seen. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '15). Association for Computing Machinery, New York, NY, USA, 946--957. https://doi.org/10.1145/2675133.2675240Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Nitesh Goyal and Susan R. Fussell. 2016. Effects of Sensemaking Translucence on Distributed Collaborative Analysis. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (CSCW '16). ACM, 288--302. https://doi.org/10.1145/2818048.2820071Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Linda Grant. 2014. Hooked on Serial, the whodunnit that has become a cultural phenomenon. The Guardian (2014). https://www.theguardian.com/commentisfree/2014/nov/14/serial-hae-min-lee-adnan-syed-whodunnit-us-crime-dramaGoogle ScholarGoogle Scholar
  41. Mary L. Gray and Siddharth Suri. 2019. Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass. Houghton Mifflin Harcourt, Boston.Google ScholarGoogle Scholar
  42. Peter W Greenwood and Joan Petersiua. 1975. The Criminal Investigation Process Volume I: Summary and Policy Implications. (1975).Google ScholarGoogle Scholar
  43. Daniel Haas, Jason Ansel, Lydia Gu, and Adam Marcus. 2015. Argonaut: Macrotask Crowdsourcing for Complex Data Processing. Proc. VLDB Endow., Vol. 8, 12 (Aug. 2015), 1642--1653. https://doi.org/10.14778/2824032.2824062Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Nathan Hahn, Joseph Chang, Ji Eun Kim, and Aniket Kittur. 2016. The Knowledge Accelerator: Big Picture Thinking in Small Pieces. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, 2258--2270. http://doi.acm.org/10.1145/2858036.2858364Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Rachel Hall. 2004. Danger and desire: Instrumental realism in the history of the wanted poster. Ph.D. Dissertation. The University of North Carolina at Chapel Hill.Google ScholarGoogle Scholar
  46. Joseph M Hellerstein and David L Tennenhouse. 2011. Searching for Jim Gray: a technical overview. Commun. ACM, Vol. 54, 7 (2011), 77--87.Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Marcos A Hernandez. 2019. True Injustice: Cultures of Violence and Stories of Resistance in the New True Crime. IdeaFest: Interdisciplinary Journal of Creative Works and Research from Humboldt State University, Vol. 3, 1 (2019), 13.Google ScholarGoogle Scholar
  48. Eliot Higgins. 2014. A Beginner's Guide to Geolocation. https://www.bellingcat.com/resources/how-tos/2014/07/09/a-beginners-guide-to-geolocation/Google ScholarGoogle Scholar
  49. Serena Hillman, Jason Procyk, and Carman Neustaedter. 2014. 'alksjdf; Lksfd' tumblr and the fandom user experience. In Proceedings of the 2014 conference on Designing interactive systems. 775--784.Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Laura Huey, Johnny Nhan, and Ryan Broll. 2013 ?Uppity civilians' and "cyber-vigilantes': The role of the general public in policing cyber-crime. Criminology & Criminal Justice, Vol. 13, 1 (2013), 81--97. https://doi.org/10.1177/1748895812448086Google ScholarGoogle ScholarCross RefCross Ref
  51. Kristin Hunt. [n. d.]. 9 True-Crime Documentaries That Changed Their Cases' Verdicts. Thrillist ([n. d.]). https://www.thrillist.com/entertainment/nation/true-crime-documentaries-serial-jinx-making-a-murder-changed-verdictsGoogle ScholarGoogle Scholar
  52. Lilly C. Irani and M. Six Silberman. 2013. Turkopticon: Interrupting Worker Invisibility in Amazon Mechanical Turk. Association for Computing Machinery, New York, NY, USA, 611--620. https://doi.org/10.1145/2470654.2470742Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Aarti Israni, Sheena Erete, and Che L. Smith. 2017. Snitches, Trolls, and Social Norms: Unpacking Perceptions of Social Media Use for Crime Prevention. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17). Association for Computing Machinery, New York, NY, USA, 1193--1209. https://doi.org/10.1145/2998181.2998238Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Dewi Jaimangal-Jones. 2014. Utilising ethnography and participant observation in festival and event research. International Journal of Event and Festival Management (2014).Google ScholarGoogle ScholarCross RefCross Ref
  55. Daniel W Jones and Leeann M Lower. 2018. When Fandom Ends and Harassment Begins. Physical Educator, Vol. 75, 4 (2018), 731--736.Google ScholarGoogle Scholar
  56. Nicolas J LaLone, Jess Kropczynski, and Andrea H Tapia. 2018. The Symbiotic Relationship of Crisis Response Professionals and Enthusiasts as Demonstrated by Reddit's User-Interface Over Time.. In ISCRAM.Google ScholarGoogle Scholar
  57. Edith Law, Conner Dalton, Nick Merrill, Albert Young, and Krzysztof Z Gajos. 2013. Curio: a platform for supporting mixed-expertise crowdsourcing. In First AAAI Conference on Human Computation and Crowdsourcing.Google ScholarGoogle ScholarCross RefCross Ref
  58. Edith Law, Krzysztof Z. Gajos, Andrea Wiggins, Mary L. Gray, and Alex Williams. 2017. Crowdsourcing as a Tool for Research: Implications of Uncertainty. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17). Association for Computing Machinery, New York, NY, USA, 1544--1561. https://doi.org/10.1145/2998181.2998197Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. ArLuther Lee. 2019. Police deny link to mysterious 'Umbrella Man,' who broke windows during riot. Atlanta Journal Constitution (2019). https://www.ajc.com/news/police-deny-link-mysterious-umbrella-man-who-broke-windows-during-riot/3j8cSrRBHjvnvX0PRwIYIP/Google ScholarGoogle Scholar
  60. Charlotte P. Lee, Paul Dourish, and Gloria Mark. 2006. The Human Infrastructure of Cyberinfrastructure. In Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work (CSCW '06). Association for Computing Machinery, New York, NY, USA, 483--492. https://doi.org/10.1145/1180875.1180950Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Lawrence Lessig. 1998. The new Chicago school. The Journal of Legal Studies, Vol. 27, S2 (1998), 661--691.Google ScholarGoogle ScholarCross RefCross Ref
  62. Lawrence Lessig. 1999. Code and Other Laws of Cyberspace. Basic Books, Inc., USA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Sheena (Lewis) Erete and Dan A. Lewis. 2012. Examining Technology That Supports Community Policing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). Association for Computing Machinery, New York, NY, USA, 1371--1380. https://doi.org/10.1145/2207676.2208595Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. Tianyi Li, Kurt Luther, and Chris North. 2018. CrowdIA: Solving Mysteries with Crowdsourced Sensemaking., Vol. 2 (2018), 105:1--105:29. Issue CSCW. https://doi.org/10.1145/3274374Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. Tianyi Li, Chandler J Manns, Chris North, and Kurt Luther. 2019. Dropping the baton? Understanding errors and bottlenecks in a crowdsourced sensemaking pipeline. Proceedings of the ACM on Human-Computer Interaction, Vol. 3, CSCW (2019), 1--26.Google ScholarGoogle Scholar
  66. N.A. Lopez Coombs. 2016. Truth Isn't Stranger than Fanfiction: The Serial Podcast and Citizen Journalism. Master's thesis.Google ScholarGoogle Scholar
  67. Cynthia Lum, Maria Maki Haberfeld, George Fachner, and Charles Lieberman. 2009. Police activities to counter terrorism: What we know and what we need to know. In To protect and to serve. Springer, 101--141.Google ScholarGoogle Scholar
  68. Kurt Luther, Scott Counts, Kristin B. Stecher, Aaron Hoff, and Paul Johns. 2009. Pathfinder: An Online Collaboration Environment for Citizen Scientists. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). Association for Computing Machinery, New York, NY, USA, 239--248. https://doi.org/10.1145/1518701.1518741Google ScholarGoogle ScholarDigital LibraryDigital Library
  69. Kurt Luther, Casey Fiesler, and Amy Bruckman. 2013a. Redistributing Leadership in Online Creative Collaboration. In Proceedings of the 2013 Conference on Computer Supported Cooperative Work. Association for Computing Machinery, New York, NY, USA, 1007--1022. https://doi.org/10.1145/2441776.2441891Google ScholarGoogle ScholarDigital LibraryDigital Library
  70. Kurt Luther, Casey Fiesler, and Amy Bruckman. 2013b. Redistributing Leadership in Online Creative Collaboration. In Proceedings of the 2013 Conference on Computer Supported Cooperative Work (CSCW '13). Association for Computing Machinery, New York, NY, USA, 1007--1022. https://doi.org/10.1145/2441776.2441891Google ScholarGoogle ScholarDigital LibraryDigital Library
  71. Andrew Macvean, Sanjeet Hajarnis, Brandon Headrick, Aziel Ferguson, Chinmay Barve, Devika Karnik, and Mark O. Riedl. 2011. WeQuest: Scalable Alternate Reality Games through End-User Content Authoring. In Proceedings of the 8th International Conference on Advances in Computer Entertainment Technology (ACE '11). Association for Computing Machinery, New York, NY, USA, Article 22, 8 pages. https://doi.org/10.1145/2071423.2071450Google ScholarGoogle ScholarDigital LibraryDigital Library
  72. Alexis C. Madrigal. 2013. Hey Reddit, Enough Boston Bombing Vigilantism. The Atlantic (April 2013). http://www.theatlantic.com/technology/archive/2013/04/hey-reddit-enough-boston-bombing-vigilantism/275062/Google ScholarGoogle Scholar
  73. N. Mahyar and M. Tory. 2014. Supporting Communication and Coordination in Collaborative Sensemaking. IEEE Transactions on Visualization and Computer Graphics, Vol. 20, 12 (2014), 1633--1642.Google ScholarGoogle ScholarCross RefCross Ref
  74. Gary T Marx. 2013. The public as partner? Technology can make us auxiliaries as well as vigilantes. IEEE Security & Privacy, Vol. 11, 5 (2013), 56--61.Google ScholarGoogle ScholarDigital LibraryDigital Library
  75. Jane McGonigal. 2003. This is not a game: Immersive aesthetics and collective play. In Melbourne DAC 2003 Streamingworlds Conference Proceedings. Citeseer.Google ScholarGoogle Scholar
  76. Panagiotis Metaxas and Samantha T Finn. 2017. The infamous# Pizzagate conspiracy theory: Insight from a TwitterTrails investigation. (2017).Google ScholarGoogle Scholar
  77. David T Moore. 2013. Sensemaking: A structure for an Intelligence Revolution. Government Printing Office.Google ScholarGoogle Scholar
  78. Meredith Ringel Morris, Jarrod Lombardo, and Daniel Wigdor. 2010. WeSearch: Supporting Collaborative Search and Sensemaking on a Tabletop Display. In Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work (CSCW '10). Association for Computing Machinery, New York, NY, USA, 401--410. https://doi.org/10.1145/1718918.1718987Google ScholarGoogle ScholarDigital LibraryDigital Library
  79. Gabriel Mugar, Carsten Østerlund, Katie DeVries Hassman, Kevin Crowston, and Corey Brian Jackson. 2014. Planet Hunters and Seafloor Explorers: Legitimate Peripheral Participation through Practice Proxies in Online Citizen Science. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '14). Association for Computing Machinery, New York, NY, USA, 109--119. https://doi.org/10.1145/2531602.2531721Google ScholarGoogle ScholarDigital LibraryDigital Library
  80. Tim Murphy. 2013. The Internet Is Actually Surprisingly Good at Fighting Crime. Mother Jones. Mother Jones (2013). https://www.motherjones.com/politics/2013/04/reddit-internet-solving-fighting-crime/Google ScholarGoogle Scholar
  81. Johnny Nhan, Laura Huey, and Ryan Broll. 2017. Digilantism: An analysis of crowdsourcing and the Boston marathon bombings. The British journal of criminology, Vol. 57, 2 (2017), 341--361.Google ScholarGoogle Scholar
  82. Siobhan O'Flynn and David Seelow. 2017. A Concise History of Alternate Reality Games: From Transmedia Marketing Campaigns to College the Classroom. International Journal on Innovations in Online Education, Vol. 1, 4 (2017).Google ScholarGoogle Scholar
  83. Karolyn Park. 2014. The Objects of Our Affection: Objectification, Sexualization and Commodification of the Korean Pop Culture Industry and Fandom. (2014).Google ScholarGoogle Scholar
  84. Sharoda A Paul and Meredith Ringel Morris. 2009. CoSense: enhancing sensemaking for collaborative web search. In Proceedings of the SIGCHI conference on human factors in computing systems. 1771--1780.Google ScholarGoogle ScholarDigital LibraryDigital Library
  85. Peter Pirolli and Stuart Card. 2005. The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In Proceedings of international conference on intelligence analysis, Vol. 5. McLean, VA, USA, 2--4.Google ScholarGoogle Scholar
  86. Daniel M Russell, Gregorio Convertino, Aniket Kittur, Peter Pirolli, and Elizabeth Anne Watkins. 2018. Sensemaking in a Senseless World: 2018 Workshop Abstract. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. 1--7.Google ScholarGoogle ScholarDigital LibraryDigital Library
  87. Mattia Samory and Tanushree Mitra. 2018. Conspiracies online: User discussions in a conspiracy community following dramatic events. In Twelfth International AAAI Conference on Web and Social Media.Google ScholarGoogle ScholarCross RefCross Ref
  88. Benjamin Saunders, Julius Sim, Tom Kingstone, Shula Baker, Jackie Waterfield, Bernadette Bartlam, Heather Burroughs, and Clare Jinks. 2018. Saturation in qualitative research: exploring its conceptualization and operationalization. Quality & quantity, Vol. 52, 4 (2018), 1893--1907.Google ScholarGoogle Scholar
  89. Dane Schiller. 2015. The evolution of the wanted poster. SFGate (2015). https://www.sfgate.com/news/article/The-evolution-of-the-wanted-poster-6110088.phpGoogle ScholarGoogle Scholar
  90. Susan Simpson. 2017. The unlikely role of true crime podcasts in criminal justice reform. Quartz (2017). https://qz.com/1101889/the-unlikely-role-of-true-crime-podcasts-in-criminal-justice-reform/Google ScholarGoogle Scholar
  91. Kate Starbird, Ahmer Arif, and Tom Wilson. 2019. Disinformation as collaborative work: Surfacing the participatory nature of strategic information operations. Proceedings of the ACM on Human-Computer Interaction, Vol. 3, CSCW (2019), 1--26.Google ScholarGoogle ScholarDigital LibraryDigital Library
  92. Kate Starbird, Jim Maddock, Mania Orand, Peg Achterman, and Robert M Mason. 2014. Rumors, false flags, and digital vigilantes: Misinformation on twitter after the 2013 boston marathon bombing. IConference 2014 Proceedings (2014).Google ScholarGoogle Scholar
  93. Kate Starbird and Leysia Palen. 2012. (How) will the revolution be retweeted? Information diffusion and the 2011 Egyptian uprising. In Proceedings of the acm 2012 conference on computer supported cooperative work. 7--16.Google ScholarGoogle ScholarDigital LibraryDigital Library
  94. Deborah Tatar. 2007. The design tensions framework. Human-Computer Interaction, Vol. 22, 4 (2007), 413--451.Google ScholarGoogle ScholarDigital LibraryDigital Library
  95. Craig Timberg and Drew Harwell. 2019. Amateur pro-Trump 'sleuths' scramble to unmask whistleblower: "Your president has asked for your help'. The Washington Post (2019). https://www.washingtonpost.com/technology/2019/09/28/amateur-pro-trump-sleuths-scramble-unmask-whistleblower-your-president-has-asked-your-help/Google ScholarGoogle Scholar
  96. Ramine Tinati, Max Van Kleek, Elena Simperl, Markus Luczak-Rösch, Robert Simpson, and Nigel Shadbolt. 2015. Designing for Citizen Data Analysis: A Cross-Sectional Case Study of a Multi-Domain Citizen Science Platform. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). Association for Computing Machinery, New York, NY, USA, 4069--4078. https://doi.org/10.1145/2702123.2702420Google ScholarGoogle ScholarDigital LibraryDigital Library
  97. Daniel Trottier. 2014. Police and user-led investigations on social media. JL Inf. & Sci., Vol. 23 (2014), 75.Google ScholarGoogle Scholar
  98. Daniel Trottier. 2017. Digital vigilantism as weaponisation of visibility. Philosophy & Technology, Vol. 30, 1 (2017), 55--72.Google ScholarGoogle ScholarCross RefCross Ref
  99. Sukrit Venkatagiri, Jacob Thebault-Spieker, Rachel Kohler, John Purviance, Rifat Sabbir Mansur, and Kurt Luther. 2019. GroundTruth: Augmenting Expert Image Geolocation with Crowdsourcing and Shared Representations. Proc. ACM Hum.-Comput. Interact., Vol. 3, CSCW, Article 107 (Nov. 2019), 30 pages. https://doi.org/10.1145/3359209Google ScholarGoogle ScholarDigital LibraryDigital Library
  100. Katherine Vogt, Lauren Bradel, Christopher Andrews, Chris North, Alex Endert, and Duke Hutchings. 2011. Co-located Collaborative Sensemaking on a Large High-Resolution Display with Multiple Input Devices. In Human-Computer Interaction -- INTERACT 2011. Springer Berlin Heidelberg, Berlin, Heidelberg, 589--604.Google ScholarGoogle ScholarDigital LibraryDigital Library
  101. Luis von Ahn and Laura Dabbish. 2008. Designing Games with a Purpose. Commun. ACM, Vol. 51, 8 (Aug. 2008), 58--67. https://doi.org/10.1145/1378704.1378719Google ScholarGoogle ScholarDigital LibraryDigital Library
  102. Andrea Wiggins and Yurong He. 2016. Community-Based Data Validation Practices in Citizen Science. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (CSCW '16). Association for Computing Machinery, New York, NY, USA, 1548--1559. https://doi.org/10.1145/2818048.2820063Google ScholarGoogle ScholarDigital LibraryDigital Library
  103. Elizabeth Yardley, Adam George Thomas Lynes, David Wilson, and Emma Kelly. 2018. What's the deal with `websleuthing'? News media representations of amateur detectives in networked spaces. Crime, Media, Culture, Vol. 14, 1 (2018), 81--109. https://doi.org/10.1177/1741659016674045Google ScholarGoogle ScholarCross RefCross Ref
  104. Yunwen Ye and Kouichi Kishida. 2003. Toward an understanding of the motivation Open Source Software developers. In Proceedings of the 25th International Conference on Software Engineering. IEEE Computer Society, Portland, Oregon, 419--429. http://portal.acm.org/citation.cfm?id=776867Google ScholarGoogle ScholarDigital LibraryDigital Library
  105. Haiyi Zhu, Robert E. Kraut, Yi-Chia Wang, and Aniket Kittur. 2011. Identifying shared leadership in Wikipedia. In Proceedings of the 2011 annual conference on Human factors in computing systems (CHI '11). ACM, New York, NY, USA, 3431--3434. https://doi.org/10.1145/1978942.1979453Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. CrowdSolve: Managing Tensions in an Expert-Led Crowdsourced Investigation

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image Proceedings of the ACM on Human-Computer Interaction
        Proceedings of the ACM on Human-Computer Interaction  Volume 5, Issue CSCW1
        CSCW
        April 2021
        5016 pages
        EISSN:2573-0142
        DOI:10.1145/3460939
        Issue’s Table of Contents

        Copyright © 2021 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 22 April 2021
        Published in pacmhci Volume 5, Issue CSCW1

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader