Inchan Jung
Youn-kyung Lim
ABSTRACT
Interactions with voice-based conversational agents (VCAs) in non-humanoid forms are becoming increasingly pervasive, and researches on non-humanoid VCAs engaging diverse human traits have been conducted. However, there have never been studies employing living body's physiological states to be expressed solely through the voice of such VCAs yet. As physiological expressions of such VCAs can have potential for manifesting health-related issues in a human-like way, we selected healthcare scenarios as a case for exploring novel user experiences that they can induce. We conducted design workshops for identifying design considerations and design opportunities for the physiologically expressible VCAs in the healthcare service domain. Following these findings, we designed the new concept of physiologically expressible healthcare VCAs and conducted a Wizard-of-Oz user study. Finally, we summarize the unique user experiences on physiologically expressible VCA's healthcare services and user perceptions of its physiological expressions, and discuss design implications for physiologically expressible VCAs.
- Emily Couvillon Alagha and Rachel Renee Helbing. 2019. Evaluating the quality of voice assistants’ responses to consumer health questions about vaccines: An exploratory comparison of Alexa, Google Assistant and Siri. BMJ Heal. Care Informatics 26, 1 (2019). https://doi.org/10.1136/bmjhci-2019-100075Google Scholar
- Christine A. Barry, Fiona A. Stevenson, Nicky Britten, Nick Barber, and Colin P. Bradley. 2001. Giving voice to the lifeworld. More humane, more effective medical care? A qualitative study of doctor-patient communication in general practice. Soc. Sci. Med. 53, 4 (2001). https://doi.org/10.1016/S0277-9536(00)00351-8Google Scholar
- Rami Basatneh, Bijan Najafi, and David G. Armstrong. 2018. Health Sensors, Smart Home Devices, and the Internet of Medical Things: An Opportunity for Dramatic Improvement in Care for the Lower Extremity Complications of Diabetes. J. Diabetes Sci. Technol. 12, 3 (2018). https://doi.org/10.1177/1932296818768618Google ScholarCross Ref
- Siân Bayne and Jen Ross. 2007. The “digital native” and “digital immigrant”: a dangerous opposition. Annu. Conf. Soc. Res. into High. Educ. (2007).Google Scholar
- Eva Brandt and Camilla Grunnet. 2000. Evoking the future: Drama and props in user centered design. Proc. Particip. Des. Conf. PDC 2000 December (2000).Google Scholar
- Michael Braun, Anja Mainz, Ronee Chadowitz, Bastian Pfleging, and Florian Alt. 2019. At your service: Designing voice assistant personalities to improve automotive user interfaces a real world driving study. In Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/3290605.3300270Google ScholarDigital Library
- Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qual. Res. Psychol. 3, 2 (2006). https://doi.org/10.1191/1478088706qp063oaGoogle Scholar
- Walter Brenner, Falk Uebernickel, and Thomas Abrell. 2016. Design thinking as mindset, process, and toolbox: Experiences from research and teaching at the university of St.Gallen. In Design Thinking for Innovation: Research and Practice. https://doi.org/10.1007/978-3-319-26100-3_1Google Scholar
- Colin Burns, Eric Dishman, William Verplank, and Bud Lassiter. 1994. Actors, hairdos & videotape - Informance design: Using performance techniques in multi-disciplinary, observation based design. In Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/259963.260102Google ScholarDigital Library
- Julia Cambre and Chinmay Kulkarni. 2019. One voice fits all? Social implications and research challenges of designing voices for smart devices. Proc. ACM Human-Computer Interact. 3, CSCW (2019). https://doi.org/10.1145/3359325Google ScholarDigital Library
- Lola Cañamero and Matthew Lewis. 2016. Making New “New AI” Friends: Designing a Social Robot for Diabetic Children from an Embodied AI Perspective. Int. J. Soc. Robot. 8, 4 (2016). https://doi.org/10.1007/s12369-016-0364-9Google Scholar
- Leigh Clark, Nadia Pantidi, Orla Cooney, Philip Doyle, Diego Garaialde, Justin Edwards, Brendan Spillane, Emer Gilmartin, Christine Murad, Cosmin Munteanu, Vincent Wade, and Benjamin R. Cowan. 2019. What makes a good conversation? Challenges in designing truly conversational agents. In Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/3290605.3300705Google ScholarDigital Library
- N. Dahlbäck, A. Jönsson, and L. Ahrenberg. 1993. Wizard of Oz studies - why and how. Knowledge-Based Syst. 6, 4 (1993). https://doi.org/10.1016/0950-7051(93)90017-NGoogle ScholarDigital Library
- J. R. Dale, S. M. Williams, and V. Bowyer. 2012. What is the effect of peer support on diabetes outcomes in adults? A systematic review. Diabetic Medicine 29. https://doi.org/10.1111/j.1464-5491.2012.03749.xGoogle Scholar
- Li Deng and Xiao Li. 2013. Machine learning paradigms for speech recognition: An overview. IEEE Trans. Audio, Speech Lang. Process. 21, 5 (2013). https://doi.org/10.1109/TASL.2013.2244083Google ScholarDigital Library
- Nicholas Epley, Adam Waytz, and John T. Cacioppo. 2007. On Seeing Human: A Three-Factor Theory of Anthropomorphism. Psychol. Rev. 114, 4 (2007). https://doi.org/10.1037/0033-295X.114.4.864Google Scholar
- Joseph Firth, John Torous, Jennifer Nicholas, Rebekah Carney, Abhishek Pratap, Simon Rosenbaum, and Jerome Sarris. 2017. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry 16, 3 (2017). https://doi.org/10.1002/wps.20472Google Scholar
- Chris Franzese and Marty Coyne. 2017. The promise of voice: Connecting drug delivery through voice-activated technology. ONdrugDelivery 2017, 81 (2017).Google Scholar
- Martha Mitchell Funnell. 2010. Peer-based behavioural strategies to improve chronic disease self-management and clinical outcomes: evidence, logistics, evaluation considerations and needs for future research. Family practice 27 Suppl 1. https://doi.org/10.1093/fampra/cmp027Google Scholar
- Jane Grimson, William Grimson, and Wilhelm Hasselbring. 2000. The SI challenge in health care. Commun. ACM 43, 6 (2000). https://doi.org/10.1145/336460.336474Google ScholarDigital Library
- Ewa Gustafsson, Sara Thomée, Anna Grimby-Ekman, and Mats Hagberg. 2017. Texting on mobile phones and musculoskeletal disorders in young adults: A five-year cohort study. Appl. Ergon. 58, (2017). https://doi.org/10.1016/j.apergo.2016.06.012Google ScholarCross Ref
- André Happe, Bruno Pouliquen, Anita Burgun, Marc Cuggia, and Pierre Le Beux. 2003. Automatic concept extraction from spoken medical reports. Int. J. Med. Inform. 70, 2–3 (2003). https://doi.org/10.1016/S1386-5056(03)00055-8Google ScholarCross Ref
- Julia Hirschberg and Christopher D. Manning. 2015. Advances in natural language processing. Science 349. https://doi.org/10.1126/science.aaa8685Google Scholar
- Michaela Kiernan, Susan D. Moore, Danielle E. Schoffman, Katherine Lee, Abby C. King, C. Barr Taylor, Nancy E. Kiernan, and Michael G. Perri. 2012. Social support for healthy behaviors: Scale psychometrics and prediction of weight loss among women in a behavioral program. Obesity 20, 4 (2012). https://doi.org/10.1038/oby.2011.293Google Scholar
- Da-jung Kim and Youn-kyung Lim. 2019. Co-performing agent: Design for building user–agent partnership in learning and adaptive services. In Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/3290605.3300714Google ScholarDigital Library
- Hankyung Kim, Dong Yoon Koh, Gaeun Lee, Jung-Mi Park, and Youn-kyung Lim. 2019. Developing a Design Guide for Consistent Manifestation of Conversational Agent Personalities. IASDR Conf. 2019 02-05 Sept, (2019), 1–17. Retrieved from https://iasdr2019.org/uploads/files/Proceedings/te-f-1175-Kim-H.pdfGoogle Scholar
- Sei Jin Ko, Charles M. Judd, and Irene V. Blair. 2006. What the voice reveals: Within- and between-category stereotyping on the basis of voice. Personal. Soc. Psychol. Bull. 32, 6 (2006). https://doi.org/10.1177/0146167206286627Google Scholar
- Ahmet Baki Kocaballi, Juan C. Quiroz, Dana Rezazadegan, Shlomo Berkovsky, Farah Magrabi, Enrico Coiera, and Liliana Laranjo. 2020. Responses of conversational agents to health and lifestyle prompts: Investigation of appropriateness and presentation structures. J. Med. Internet Res. 22, 2 (2020). https://doi.org/10.2196/15823Google ScholarCross Ref
- Todd Kulesza, Simone Stumpf, Margaret Burnett, and Irwin Kwan. 2012. Tell me more? the effects of mental model soundness on personalizing an intelligent agent. In Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/2207676.2207678Google ScholarDigital Library
- Liliana Laranjo, Adam G. Dunn, Huong Ly Tong, Ahmet Baki Kocaballi, Jessica Chen, Rabia Bashir, Didi Surian, Blanca Gallego, Farah Magrabi, Annie Y.S. Lau, and Enrico Coiera. 2018. Conversational agents in healthcare: A systematic review. Journal of the American Medical Informatics Association 25. https://doi.org/10.1093/jamia/ocy072Google ScholarCross Ref
- Kwan Min Lee and Clifford Nass. 2005. Social-psychological origins of feelings of presence: Creating social presence with machine-generated voices. Media Psychology 7. https://doi.org/10.1207/S1532785XMEP0701_2Google Scholar
- Sunok Lee, Sungbae Kim, and Sangsu Lee. 2019. “What does your agent look like?” A drawing study to understand users’ perceived persona of conversational agent. In Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/3290607.3312796Google ScholarDigital Library
- Adam S. Miner, Arnold Milstein, Stephen Schueller, Roshini Hegde, Christina Mangurian, and Eleni Linos. 2016. Smartphone-based conversational agents and responses to questions about mental health, interpersonal violence, and physical health. JAMA Intern. Med. 176, 5 (2016). https://doi.org/10.1001/jamainternmed.2016.0400Google Scholar
- Clifford Nass and Scott Brave. 2005. Wired for Speech: How Voice Activates and Advances the Human Computer Relationship. Comput. Linguist. 32, 3 (2005).Google Scholar
- Clifford Nass and Kwan Min Lee. 2000. Does computer-generated speech manifest personality? An experimental test of similarity-attraction. In Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/332040.332452Google ScholarDigital Library
- Clifford Nass, Clifford Nass, Ulla Foehr, Ulla Foehr, Scott Brave, Scott Brave, Michael Somoza, and Michael Somoza. 2001. The effects of emotion of voice in synthesized and recorded speech. In Emotional and Intelligent II: The Tangled Knot of Social Cognition: Proceedings of the 2001 AAAI Fall Symposium.Google Scholar
- Clifford Nass, Janathan Steuer, and Ellen R. Tauber. 1994. Computer are social actors. In Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/259963.260288Google ScholarDigital Library
- Alicia L. Nobles, Eric C. Leas, Theodore L. Caputi, Shu Hong Zhu, Steffanie A. Strathdee, and John W. Ayers. 2020. Responses to addiction help-seeking from Alexa, Siri, Google Assistant, Cortana, and Bixby intelligent virtual assistants. npj Digit. Med. 3, 1 (2020). https://doi.org/10.1038/s41746-019-0215-9Google Scholar
- Marta Perez Garcia and Sarita Saffon Lopez. 2020. Exploring the uncanny valley theory in the constructs of a virtual assistant personality. In Advances in Intelligent Systems and Computing. https://doi.org/10.1007/978-3-030-29516-5_76Google Scholar
- Giulia Perugia, Daniel Rodriguez-Martin, Marta DIaz Boladeras, Andreu Catala Mallofre, Emilia Barakova, and Matthias Rauterberg. 2017. Electrodermal activity: Explorations in the psychophysiology of engagement with social robots in dementia. In RO-MAN 2017 - 26th IEEE International Symposium on Robot and Human Interactive Communication. https://doi.org/10.1109/ROMAN.2017.8172464Google ScholarDigital Library
- Martin Porcheron, Joel E. Fischer, Stuart Reeves, and Sarah Sharples. 2018. Voice interfaces in everyday life. In Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/3173574.3174214Google ScholarDigital Library
- Atieh Poushneh. 2021. Humanizing voice assistant: The impact of voice assistant personality on consumers’ attitudes and behaviors. J. Retail. Consum. Serv. 58, (2021). https://doi.org/10.1016/j.jretconser.2020.102283Google Scholar
- Jenny Preece. 2000. Online Communities: Designing Usability and Supporting Sociability.Google ScholarDigital Library
- Sarah Masud Preum, Sirajum Munir, Meiyi Ma, Mohammad Samin Yasar, David J. Stone, Ronald Williams, Homa Alemzadeh, and John A. Stankovic. 2021. A Review of Cognitive Assistants for Healthcare: Trends, Prospects, and Future Directions. ACM Computing Surveys 53. https://doi.org/10.1145/3419368Google ScholarDigital Library
- Takanori Shibata and Kazuyoshi Wada. 2011. Robot therapy: A new approach for mental healthcare of the elderly - A mini-review. Gerontology 57. https://doi.org/10.1159/000319015Google Scholar
- Elizabeth Sillence, Claire Hardy, Peter R. Harris, and Pam Briggs. 2014. Modeling patient engagement in peer-to-peer healthcare. In WWW 2014 Companion - Proceedings of the 23rd International Conference on World Wide Web. https://doi.org/10.1145/2567948.2576937Google ScholarDigital Library
- Rosemary Stockdale. 2008. Peer-to-peer online communities for people with chronic diseases: A conceptual framework. J. Syst. Inf. Technol. 10, 1 (2008). https://doi.org/10.1108/13287260810876885Google Scholar
- Anselm L. Strauss. 1987. Qualitative Analysis for Social Scientists. https://doi.org/10.1017/cbo9780511557842Google Scholar
- George Terzopoulos and Maya Satratzemi. 2020. Voice assistants and smart speakers in everyday life and in education. Informatics Educ. 19, 3 (2020). https://doi.org/10.15388/infedu.2020.21Google Scholar
- Jayneel Vora, Sudeep Tanwar, Sudhanshu Tyagi, Neeraj Kumar, and Joel J.P.C. Rodrigues. 2017. Home-based exercise system for patients using IoT enabled smart speaker. In 2017 IEEE 19th International Conference on e-Health Networking, Applications and Services, Healthcom 2017. https://doi.org/10.1109/HealthCom.2017.8210826Google ScholarCross Ref
- Kazuyoshi Wada, Takanori Shibata, Tomoko Saito, Kayoko Sakamoto, and Kazuo Tanie. 2005. Psychological and social effects of one year robot assisted activity on elderly people at a health service facility for the aged. In Proceedings - IEEE International Conference on Robotics and Automation. https://doi.org/10.1109/ROBOT.2005.1570535Google ScholarCross Ref
- Monica L. Wang, Lori Pbert, and Stephenie C. Lemon. 2014. Influence of family, friend and coworker social support and social undermining on weight gain prevention among adults. Obesity 22, 9 (2014). https://doi.org/10.1002/oby.20814Google Scholar
- Rena R. Wing and Robert W. Jeffery. 1999. Benefits of recruiting participants with friends and increasing social support for weight loss and maintenance. J. Consult. Clin. Psychol. 67, 1 (1999). https://doi.org/10.1037/0022-006X.67.1.132Google ScholarCross Ref
- Yumi Yogo, Sachiko Tsutsui, Mitsuko Ando, Akie Hashi, and Naoto Yamada. 2000. Judgments of emotion by nurses and students given double-bind information on a patient's tone of voice and message content. Percept. Mot. Skills 90, 3 PART 1 (2000). https://doi.org/10.2466/pms.2000.90.3.855Google Scholar
Recommendations
Exploring the Use of a Voice-based Conversational Agent to Empower Adolescents with Autism Spectrum Disorder
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing SystemsVoice-based Conversational Agents (VCA) have served as personal assistants that support individuals with special needs. Adolescents with Autism Spectrum Disorder (ASD) may also benefit from VCAs to deal with their everyday needs and challenges, ranging ...
Understanding the Negative Aspects of User Experience in Human-likeness of Voice-based Conversational Agents
DIS '22: Proceedings of the 2022 ACM Designing Interactive Systems ConferenceWith advances in artificial intelligence technology, Voice-based Conversational Agents (VCAs) can now imitate human abilities, sometimes almost indistinguishably from humans. However, concerns have been raised that too much perceived similarity can ...
Exploring Effects of Conversational Fillers on User Perception of Conversational Agents
CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing SystemsThrough technological advancements in various areas of our lives, Conversational Agents progressed in their human-likeness. In the field of HCI, however, the use of conversational fillers (e.g., "um," "uh," etc.) by Conversational Agents have not been ...
Comments