Hyunggu Jung
ABSTRACT
We propose a method of generating value scenarios for design research by leveraging ChatGPT, an AI-powered chatbot based on large language models. Identifying the needs of a vulnerable population, such as North Korean defectors, is challenging for researchers. To address this, we introduce ChatGPT-generated value scenarios, an extension of scenario-based design that supports critical, systemic, long-term thinking in current design practice, technology development, and deployment. Using our proposed method, we created a prompt to generate value scenarios on ChatGPT. Based on our analysis of the generated scenarios, we identified that ChatGPT could generate plausible information about Value Implications. However, it lacks details on Pervasiveness and Systemic Effects. After discussing the limitations and opportunities of ChatGPT in generating value scenarios, we conclude with suggestions for how ChatGPT might be better used to generate value scenarios.
- Abubakar Abid, Maheen Farooqi, and James Zou. 2021. Large language models associate Muslims with violence. Nature Machine Intelligence 3, 6 (2021), 461–463. https://doi.org/10.1038/s42256-021-00359-2Google Scholar
Cross Ref
- Gerardo Adesso. 2022. GPT4: The ultimate brain. https://doi.org/10.22541/au.167052124.41804127/v2Google Scholar
- Fatema AlDhaen. 2022. The Use of Artificial Intelligence in Higher Education – Systematic Review. Springer International Publishing, Cham, 269–285. https://doi.org/10.1007/978-3-031-13351-0_13Google Scholar
- Abdulwahhab O. Alshammari and Hyunggu Jung. 2017. Designing community of practice systems: a value sensitive approach. In 2017 International Conference on Informatics, Health & Technology (ICIHT). IEEE, Riyadh, Saudi Arabia, 1–7. https://doi.org/10.1109/ICIHT.2017.7899005Google Scholar
- Alissa Nicole Antle. 2006. Child-Personas: Fact or Fiction?. In Proceedings of the 6th Conference on Designing Interactive Systems (University Park, PA, USA) (DIS ’06). Association for Computing Machinery, New York, NY, USA, 22–30. https://doi.org/10.1145/1142405.1142411Google ScholarDigital Library
- Ömer Aydın and Enis Karaarslan. 2022. OpenAI ChatGPT generated literature review: Digital twin in healthcare. Emerging Computer Technologies 2 (2022), 22-41 pages. https://doi.org/10.2139/ssrn.4308687Google Scholar
- Lanthao Benedikt, Chaitanya Joshi, Louisa Nolan, Ruben Henstra-Hill, Luke Shaw, and Sharon Hook. 2020. Human-in-the-Loop AI in Government: A Case Study. In Proceedings of the 25th International Conference on Intelligent User Interfaces (Cagliari, Italy) (IUI ’20). Association for Computing Machinery, New York, NY, USA, 488–497. https://doi.org/10.1145/3377325.3377489Google ScholarDigital Library
- Margaret Cargill and Patrick O’Connor. 2021. Writing scientific research articles: Strategy and steps. John Wiley & Sons, Oxford, UK.Google Scholar
- J.M. Carrol. 1999. Five reasons for scenario-based design. In Proceedings of the 32nd Annual Hawaii International Conference on Systems Sciences. 1999. HICSS-32. Abstracts and CD-ROM of Full Papers, Vol. Track3. IEEE, Maui, HI, USA, 123. https://doi.org/10.1109/HICSS.1999.772890Google ScholarCross Ref
- John M. Carroll. 2003. Scenario-Based Design. CRC Press, Florida, 45–70.Google Scholar
- Yanran Chen and Steffen Eger. 2022. Transformers Go for the LOLs: Generating (Humourous) Titles from Scientific Abstracts End-to-End. arxiv:2212.10522 [cs.CL]Google Scholar
- Geoffrey M Currie. 2023. Academic integrity and artificial intelligence: is ChatGPT hype, hero or heresy?https://doi.org/10.1053/j.semnuclmed.2023.04.008Google Scholar
- Gianluca Demartini, Stefano Mizzaro, and Damiano Spina. 2020. Human-in-the-loop Artificial Intelligence for Fighting Online Misinformation: Challenges and Opportunities.IEEE Data Eng. Bull. 43, 3 (2020), 65–74.Google Scholar
- Jianyang Deng and Yijia Lin. 2023. The Benefits and Challenges of ChatGPT: An Overview. Frontiers in Computing and Intelligent Systems 2, 2 (Jan. 2023), 81–83. https://doi.org/10.54097/fcis.v2i2.4465Google Scholar
- Michael Dowling and Brian Lucey. 2023. ChatGPT for (finance) research: The Bananarama conjecture. Finance Research Letters 53 (2023), 103662. https://doi.org/10.1016/j.frl.2023.103662Google ScholarCross Ref
- Batya Friedman, Peter H Kahn, Alan Borning, and Alina Huldtgren. 2013. Value sensitive design and information systems. Early engagement and new technologies: Opening up the laboratory 5 (2013), 55–95. https://doi.org/10.1007/978-94-007-7844-3_4Google Scholar
- Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, Andy Jones, Sam Bowman, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Nelson Elhage, Sheer El-Showk, Stanislav Fort, Zac Hatfield-Dodds, Tom Henighan, Danny Hernandez, Tristan Hume, Josh Jacobson, Scott Johnston, Shauna Kravec, Catherine Olsson, Sam Ringer, Eli Tran-Johnson, Dario Amodei, Tom Brown, Nicholas Joseph, Sam McCandlish, Chris Olah, Jared Kaplan, and Jack Clark. 2022. Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned. arxiv:2209.07858 [cs.CL]Google Scholar
- Catherine A Gao, Frederick M Howard, Nikolay S Markov, Emma C Dyer, Siddhi Ramesh, Yuan Luo, and Alexander T Pearson. 2023. Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers. NPJ Digital Medicine 6, 1 (2023), 75. https://doi.org/10.1038/s41746-023-00819-6Google ScholarCross Ref
- Ben Hutchinson, Vinodkumar Prabhakaran, Emily Denton, Kellie Webster, Yu Zhong, and Stephen Denuyl. 2020. Social Biases in NLP Models as Barriers for Persons with Disabilities. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Online, 5491–5501. https://doi.org/10.18653/v1/2020.acl-main.487Google ScholarCross Ref
- Andrew Ng Isa Fulford. 2023. ChatGPT Prompt Engineering for Developers. DeepLearning.AI. https://www.deeplearning.ai/short-courses/chatgpt-prompt-engineering-for-developers/Accessed: May 3, 2023.Google Scholar
- Hyunggu Jung, Woosuk Seo, and Michelle Cha. 2017. Personas and Scenarios to Design Technologies for North Korean Defectors with Depression. In Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Portland, Oregon, USA) (CSCW ’17 Companion). Association for Computing Machinery, New York, NY, USA, 215–218. https://doi.org/10.1145/3022198.3026308Google ScholarDigital Library
- David Jungwirth and Daniela Haluza. 2023. Artificial Intelligence and Public Health: An Exploratory Study. International Journal of Environmental Research and Public Health 20, 5 (Mar 2023), 4541. https://doi.org/10.3390/ijerph20054541Google ScholarCross Ref
- HA Kolnick, Jennifer E Miller, Olivia Dupree, and Lisa Gualtieri. 2021. Design Thinking to Create a Remote Patient Monitoring Platform for Older Adults’ Homes. https://doi.org/10.5210/ojphi.v13i1.11582Google Scholar
- Wanhae Lee, Minki Chun, Hyeonhak Jeong, and Hyunggu Jung. 2023. Toward Keyword Generation through Large Language Models. In Companion Proceedings of the 28th International Conference on Intelligent User Interfaces (Sydney, NSW, Australia) (IUI ’23 Companion). Association for Computing Machinery, New York, NY, USA, 37–40. https://doi.org/10.1145/3581754.3584126Google ScholarDigital Library
- Juho Leinonen, Arto Hellas, Sami Sarsa, Brent Reeves, Paul Denny, James Prather, and Brett A. Becker. 2023. Using Large Language Models to Enhance Programming Error Messages. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 (Toronto ON, Canada) (SIGCSE 2023). Association for Computing Machinery, New York, NY, USA, 563–569. https://doi.org/10.1145/3545945.3569770Google ScholarDigital Library
- Ana Isabel Martins, Alexandra Queirós, Nelson Pacheco Rocha, Telmo Neves, António Damasceno, and Luisa Arieira. 2018. Personas and Scenarios to Improve the Development of an Electronic Social Record Platform. In Proceedings of the 8th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-Exclusion (Thessaloniki, Greece) (DSAI ’18). Association for Computing Machinery, New York, NY, USA, 344–351. https://doi.org/10.1145/3218585.3218681Google ScholarDigital Library
- Lisa P. Nathan, Predrag V. Klasnja, and Batya Friedman. 2007. Value Scenarios: A Technique for Envisioning Systemic Effects of New Technologies. In CHI ’07 Extended Abstracts on Human Factors in Computing Systems (San Jose, CA, USA) (CHI EA ’07). Association for Computing Machinery, New York, NY, USA, 2585–2590. https://doi.org/10.1145/1240866.1241046Google ScholarDigital Library
- Jin-Won Noh and So Hee Lee. 2020. Trauma history and mental health of North Korean defectors. Current Behavioral Neuroscience Reports 7 (2020), 250–257. https://doi.org/10.1007/s40473-020-00219-0Google ScholarCross Ref
- Ministry of Unification Settlement support for North Korean defectors. 2022. 1. Number of North Korean defectors entering South Korea. https://www.unikorea.go.kr/eng_unikorea/whatwedo/support/. Accessed: July 3, 2023.Google Scholar
- OpenAI. 2022. ChatGPT Prompt Design. OpenAI. https://platform.openai.com/docs/guides/completion/prompt-designAccessed: May 3, 2023.Google Scholar
- OpenAI. 2023. ChatGPT: Optimizing Language Models for Dialogue. OpenAI. https://chat.openai.com/Accessed: May 3, 2023.Google Scholar
- Malik Sallam. 2023. ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. Healthcare 11, 6 (2023), 887 pages. https://doi.org/10.3390/healthcare11060887Google Scholar
- Anna Stone. 2022. Student perceptions of academic integrity: a qualitative study of understanding, consequences, and impact. Journal of Academic Ethics (2022), 19 pages. https://doi.org/10.1007/s10805-022-09461-5Google Scholar
- Raju Vaishya, Anoop Misra, and Abhishek Vaish. 2023. ChatGPT: Is this version good for healthcare and research?Diabetes & Metabolic Syndrome: Clinical Research & Reviews 17, 4 (2023), 102744. https://doi.org/10.1016/j.dsx.2023.102744Google Scholar
- Liansheng Wang, Lianyu Zhou, Wenxian Yang, and Rongshan Yu. 2022. Deepfakes: a new threat to image fabrication in scientific publications?Patterns 3, 5 (2022), 4 pages. https://doi.org/10.1016/j.patter.2022.100509Google Scholar
- Pontus Wärnestål, Petra Svedberg, and Jens Nygren. 2014. Co-Constructing Child Personas for Health-Promoting Services with Vulnerable Children. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI ’14). Association for Computing Machinery, New York, NY, USA, 3767–3776. https://doi.org/10.1145/2556288.2557115Google ScholarDigital Library
- Karsten Wenzlaff and Sebastian Spaeth. 2022. Smarter than Humans? Validating how OpenAI’s ChatGPT model explains Crowdfunding, Alternative Finance and Community Finance.WiSo-HH Working Paper Series 75. University of Hamburg, Faculty of Business, Economics and Social Sciences, WISO Research Laboratory. https://doi.org/10.2139/ssrn.4302443Google Scholar
- Michele A. Williams, Amy Hurst, and Shaun K. Kane. 2013. "Pray before You Step out": Describing Personal and Situational Blind Navigation Behaviors. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility (Bellevue, Washington) (ASSETS ’13). Association for Computing Machinery, New York, NY, USA, Article 28, 8 pages. https://doi.org/10.1145/2513383.2513449Google ScholarDigital Library
- Ziang Xiao, Xingdi Yuan, Q. Vera Liao, Rania Abdelghani, and Pierre-Yves Oudeyer. 2023. Supporting Qualitative Analysis with Large Language Models: Combining Codebook with GPT-3 for Deductive Coding. In Companion Proceedings of the 28th International Conference on Intelligent User Interfaces (Sydney, NSW, Australia) (IUI ’23 Companion). Association for Computing Machinery, New York, NY, USA, 75–78. https://doi.org/10.1145/3581754.3584136Google ScholarDigital Library
- J.D. Zamfirescu-Pereira, Richmond Y. Wong, Bjoern Hartmann, and Qian Yang. 2023. Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 437, 21 pages. https://doi.org/10.1145/3544548.3581388Google ScholarDigital Library
- Xiaoming Zhai. 2022. ChatGPT user experience: Implications for education. https://doi.org/10.2139/ssrn.4312418Google Scholar
- Jianlong Zhou, Heimo Müller, Andreas Holzinger, and Fang Chen. 2023. Ethical ChatGPT: Concerns, Challenges, and Commandments. arxiv:2305.10646 [cs.AI]Google Scholar
Index Terms
- Toward Value Scenario Generation Through Large Language Models
Recommendations
Value scenarios: a technique for envisioning systemic effects of new technologies
CHI EA '07: CHI '07 Extended Abstracts on Human Factors in Computing SystemsIn this paper we argue that there is a scarcity of methods which support critical, systemic, long-term thinking in current design practice, technology development and deployment. To address this need we introduce value scenarios, an extension of ...
Science fiction as a value scenario for historical technology
AbstractThe value scenario is a useful tool in the sheaf of methods within value sensitive design. When envisioning new technology, this tool supports the designer in speculatively considering relevant stakeholders, values expressed or rebuffed by an ...
Alternative/Exceptional Scenario Generation with Differential Scenario
ICSEA '08: Proceedings of the 2008 The Third International Conference on Software Engineering AdvancesA method of generating alternative/exceptional scenarios using differential information is presented. Behaviors of normal scenarios of similar purpose are quite similar, while actors and data in scenarios are different among these scenarios. We derive ...
Comments