ABSTRACT
The current approach of relying primarily on institutional experts to create knowledge to solve humanity's problems is insufficient to meet the scale, diversity, and novelty of people's needs. Building expertise in people to create knowledge they need provides a promising approach. Despite having contextual insights, people fail to rapidly generate sound plans---like experiments---and correctly implement specific actions---like data acquisition and analysis. The limits to progress in multiple domains---like science and healthcare---can potentially be expanded by building procedural expertise among motivated nonexperts so that they can build on their contextual insights to create valid and generalizable knowledge. In this paper, we report on the design and evaluation of tools that highlight two ways to realize this vision. First, Hevelius is a motor impairment assessment tool for patients to conduct neurological assessments online. A rare disease community has provided fine-granular data and insights from their homes that current in-clinic assessments fail to capture. Second, Gut Instinct is a social computing system that supports procedural knowledge acquisition for experimentation. A fermentation community used Gut Instinct to successfully design and run between-subjects experiments to test their intuitions. These results suggest exploring ways of producing knowledge that are distinct from the dominant model of institutionally-situated experts testing their ideas on subjects in a lab or a clinic. More constructively, these systems demonstrate how knowledgeable and committed people can be aided and amplified by technology in creating scientific knowledge.
- National Academies of Sciences, Engineering, and Medicine. 2018. Learning through citizen science: enhancing opportunities by design. National Academies Press.Google Scholar
- Tim Althoff, Rok Sosiš, Jennifer L. Hicks, Abby C. King, Scott L. Delp, and Jure Leskovec. 2017. Large-scale physical activity data reveal worldwide activity inequality. Nature 547, 7663: 336--339.Google Scholar
- Audubon. 2016. Audubon Science. Using data to realize the best conservation outcomes. Retrieved December 31, 2016 from http://www.audubon.org/conservation/science/christmas -bird-countGoogle Scholar
- Rick Bonney, Caren B. Cooper, Janis Dickinson, Steve Kelling, Tina Phillips, Kenneth V. Rosenberg, and Jennifer Shirk. 2009. Citizen Science: A Developing Tool for Expanding Science Knowledge and Scientific Literacy. Bioscience 59, 11: 977--984. https://doi.org/10.1525/bio.2009.59.11.9Google ScholarCross Ref
- Project BudBurst Boulder Colorado. 2016. Project BudBurst: An online database of plant phenological observations. Retrieved December 31, 2016 from http://budburst.org/Google Scholar
- Justin Cheng and Michael Bernstein. 2014. Catalyst: triggering collective action with thresholds. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing, 1211--1221.Google ScholarDigital Library
- Stephanie N Chilton, Jeremy P Burton, and Gregor Reid. 2015. Inclusion of fermented foods in food guides around the world. Nutrients 7, 1: 390--404.Google ScholarCross Ref
- Seth Cooper, Firas Khatib, Adrien Treuille, and Et Al. 2010. Predicting protein structures with a multiplayer online game. Nature 466, 7307: 756--760.Google Scholar
- Michael J. Coren and Fast Company. 2011. Foldit Gamers Solve Riddle of HIV Enzyme within 3 Weeks. Retrieved December 31, 2016 from https://www.scientificamerican.com/article/foldit-gamers-solve-riddle/Google Scholar
- Lorenzo Coviello, Yunkyu Sohn, Adam D.I. Kramer, Cameron Marlow, Massimo Franceschetti, Nicholas A. Christakis, and James H. Fowler. 2014. Detecting emotional contagion in massive social networks. PLoS ONE.Google Scholar
- Dana Lewis. Real-World Use of Open Source Artificial Pancreas Systems. Retrieved from https://openaps.org/2016/06/11/real-world-use-of-open-source-artificial-pancreas-systems-poster-presented-at-american-diabetes-association-scientific-sessions/Google Scholar
- E Ernst. 2003. Kombucha: a systematic review of the clinical evidence. Complementary Medicine Research 10, 2: 85--87.Google ScholarCross Ref
- f.lux. 2019. f.lux: sleep research. Retrieved from justgetflux.com/research.htmlGoogle Scholar
- Krzysztof Z Gajos, Katharina Reinecke, Mary Donovan, Christopher D Stephen, Albert Y Hung, Jeremy D Schmahmann, and Anoopum S Gupta. 2020. Computer mouse use captures ataxia and parkinsonism, enabling accurate measurement and detection. Movement disorders: official journal of the Movement Disorder Society 35, 2: 354--358. https://doi.org/10.1002/mds.27915Google Scholar
- John D Hansen and Justin Reich. 2015. Democratizing education? Examining access and usage patterns in massive open online courses. Science 350, 6265: 1245--1248.Google Scholar
- Eric von Hippel. 2005. Democratizing innovation: The evolving phenomenon of user innovation. Journal fur Betriebswirtschaft 55, 1: 63--78. https://doi.org/10.1007/s11301-004-0002-8Google ScholarCross Ref
- Maia Jacobs, Galina Gheihman, Krzysztof Z Gajos, and Anoopum S Gupta. 2019. "I Think We Know More than Our Doctors": How Primary Caregivers Manage Care Teams with Limited Disease-Related Expertise. Proc. ACM Hum.-Comput. Interact. 3, CSCW. https://doi.org/10.1145/3359261Google ScholarDigital Library
- Alexander Kawrykow, Gary Roumanis, Alfred Kam, Daniel Kwak, Clarence Leung, Chu Wu, Eleyine Zarour, Luis Sarmenta, Mathieu Blanchette, and Jérôme Waldispühl. 2012. Phylo: A citizen science approach for improving multiple sequence alignment. PLoS ONE 7, 3. https://doi.org/10.1371/journal.pone.0031362Google ScholarCross Ref
- Walter Lasecki, Christopher Miller, Adam Sadilek, Andrew Abumoussa, Donato Borrello, Raja Kushalnagar, and Jeffrey Bigham. 2012. Real-time captioning by groups of non-experts. In Proceedings of the 25th annual ACM symposium on User interface software and technology, 23--34.Google ScholarDigital Library
- Jeehyung Lee, Wipapat Kladwang, Minjae Lee, Daniel Cantu, Martin Azizyan, Hanjoo Kim, Alex Limpaecher, Snehal Gaikwad, Sungroh Yoon, Adrien Treuille, and Rhiju Das. 2014. RNA design rules from a massive open laboratory. Proceedings of the National Academy of Sciences 111, 6: 2122--2127.Google ScholarCross Ref
- Wendy E Mackay, Caroline Appert, Michel Beaudouin-Lafon, Olivier Chapuis, Yangzhou Du, Jean-Daniel Fekete, and Yves Guiard. 2007. Touchstone: exploratory design of experiments. CHI '07 Proceedings of the SIGCHI Conference on Human Factors in Computing System: 1425--1434.Google ScholarDigital Library
- D. W. Martin. 2007. Doing psychology experiments. Cengage Learning.Google Scholar
- Daniel McDonald, Embriette Hyde, Justine W Debelius, James T Morton, Antonio Gonzalez, Gail Ackermann, Alexander A Aksenov, Bahar Behsaz, Caitriona Brennan, and Yingfeng Chen. 2018. American Gut: an Open Platform for Citizen Science Microbiome Research. mSystems 3, 3: e00031-18.Google Scholar
- Matthew Might. Hunting down my son's killer. Retrieved from http://matt.might.net/articles/my-sons-killer/Google Scholar
- Vineet Pandey. 2019. Citizen-led Work using Social Computing and Procedural Guidance. Doctoral Dissertation, UC San Diego. Retrieved from https://escholarship.org/uc/item/2ck2p697Google Scholar
- Vineet Pandey, Justine Debelius, Embriette R Hyde, Tomasz Kosciolek, Rob Knight, and Scott Klemmer. 2018. Docent: transforming personal intuitions to scientific hypotheses through content learning and process training. In Proceedings of the Fifth Annual ACM Conference on Learning at Scale, 9.Google ScholarDigital Library
- Birgit Penzenstadler, Jason Plojo, Marinela Sanchez, Ruben Marin, Lam Tran, and Jayden Khakurel. 2018. The DIY Resilient Smart Garden Kit. In Proceedings of the Workshop on Computing within Limits (LIMITS), Calgary, AB, Canada, 6--7.Google Scholar
- Barath Raghavan and Daniel Pargman. 2016. Refactoring Society: Systems Complexity in an Age of Limits.Google ScholarDigital Library
- Katharina Reinecke, Ann Arbor, and Krzysztof Z Gajos. 2015. LabintheWild: Conducting Large-Scale Online Experiments With Uncompensated Samples. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing.Google ScholarDigital Library
- Katharina Reinecke and Krzysztof Z Gajos. 2014. Quantifying visual preferences around the world. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 11--20.Google ScholarDigital Library
- Bethany Rittle-Johnson and Martha Wagner Alibali. 1999. Conceptual and procedural knowledge of mathematics: Does one lead to the other? Journal of Educational Psychology 91, 1: 175--189.Google ScholarCross Ref
- Elena Rocco. 1998. Trust Breaks down in Electronic Contexts but Can Be Repaired by Some Initial Face-to-Face Contact. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '98), 496--502. https://doi.org/10.1145/274644.274711Google ScholarDigital Library
- Kentaro Toyama. 2015. Geek heresy: rescuing social change from the cult of technology / Kentaro Toyama. PublicAffairs, New York.Google Scholar
- UNESCO. 2016. Nearly 69 million new teachers needed to achieve global education goals, UNESCO reports. Retrieved from https://news.un.org/en/story/2016/10/541902-nearly-69-million-new-teachers-needed-achieve-global-education-goals-unescoGoogle Scholar
- Paul Wicks, Timothy E Vaughan, Michael P Massagli, and James Heywood. 2011. Accelerated clinical discovery using self-reported patient data collected online and a patient-matching algorithm. Nature Biotechnology 29, 5: 411--414.Google ScholarCross Ref
- Wikipedia. Community Insights/2018 Report. 2018. Retrieved from https://meta.wikimedia.org/wiki/Community_Insights/2018_Report#What_progress_has_been_made_in_the_diversity_of_Wikimedia_communitiesGoogle Scholar
- Wikipedia. 2018. Bristol stool scale. Retrieved from en.wikipedia.org/wiki/Bristol_stool_scaleGoogle Scholar
- World Health Organization. 2018. Dangers of poor quality health care revealed 'in all countries': WHO report. Retrieved from https://news.un.org/en/story/2018/07/1013942Google Scholar
- Zooniverse. 2007. Galaxy Zoo. Retrieved December 31, 2016 from galaxyzoo.orgGoogle Scholar
Index Terms
- From novices to co-pilots: Fixing the limits on scientific knowledge production by accessing or building expertise
Recommendations
Linking code readability, structure, and comprehension among novices: it's complicated
ICSE-SEET '19: Proceedings of the 41st International Conference on Software Engineering: Software Engineering Education and TrainingNovices' functionally-correct code is often redundant, verbose, or un-idiomatic. Such code could indicate shallow understanding of the programming language, or unfamiliarity with experts' preferences for code structure. Understanding why novices write ...
Comments