skip to main content
10.1145/3334480.3383075acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
abstract

"I Can Feel Your Empathic Voice": Effects of Nonverbal Vocal Cues in Voice User Interface

Published: 25 April 2020 Publication History

Abstract

Active use of voice assistant highlights the importance of interpersonal relationships with the conversational voice agent. When the agent evaluates a user's task performance, empathic feedback is needed to prevent the user's negative feelings. Voice reflects emotions through its nonverbal voice features that can either strengthen or lessen the feeling of empathy. Our study investigated the effect of nonverbal vocal cues in speech interaction on the user's perception toward the agent. 39 university students participated in the experiment, and MANOVA was tested to analyze their responses regarding intimacy, similarity, connectedness, enjoyment, and ease of use. The study result showed that using nonverbal vocal cues on empathic feedback contributes to establishing an interpersonal relationship with the agent, which gives implications to the fields of human-centered agent design.

References

[1]
Ritu Agarwal and Elena Karahanna. 2000. Time flies when you're having fun: Cognitive absorption and beliefs about information technology usage. MIS quarterly (2000), 665--694.
[2]
Marti J Anderson. 2001. A new method for non-parametric multivariate analysis of variance. Austral ecology 26, 1 (2001), 32--46.
[3]
Russell Beale and Chris Creed. 2009. Affective interaction: How emotional agents affect users. International journal of human-computer studies 67, 9 (2009), 755--776.
[4]
Timothy Bickmore and Daniel Schulman. 2007. Practical approaches to comforting users with relational agents. In CHI'07 extended abstracts on Human factors in computing systems. ACM, 2291--2296.
[5]
Timothy Bickmore, Daniel Schulman, and Langxuan Yin. 2010. Maintaining engagement in long-term interventions with relational agents. Applied Artificial Intelligence 24, 6 (2010), 648--666.
[6]
Timothy W Bickmore and Rosalind W Picard. 2005. Establishing and maintaining long-term human-computer relationships. ACM Transactions on Computer-Human Interaction (TOCHI) 12, 2 (2005), 293--327.
[7]
Mimi Bong. 2004. Academic motivation in self-efficacy, task value, achievement goal orientations, and attributional beliefs. The Journal of Educational Research 97, 6 (2004), 287--298.
[8]
Arifa Bunglowala and Aaquil Bunglowala. 2015. Nonverbal communication: An integral part of teaching learning process. International Journal of Research in Advent Technology (2015), 371--375.
[9]
Judee K Burgoon. 1980. Nonverbal communication research in the 1970s: An overview. Annals of the International Communication Association 4, 1 (1980), 179--197.
[10]
Judee K Burgoon, David B Buller, Jerold L Hale, and Mark A de Turck. 1984. Relational messages associated with nonverbal behaviors. Human Communication Research 10, 3 (1984), 351--378.
[11]
Donn Byrne. 1961. Interpersonal attraction and attitude similarity. The Journal of Abnormal and Social Psychology 62, 3 (1961), 713.
[12]
Dan S Chiaburu and Sophia V Marinova. 2005. What predicts skill transfer? An exploratory study of goal orientation, training self-efficacy and organizational supports. International journal of training and development 9, 2 (2005), 110--123.
[13]
John O Greene and Brant Raney Burleson. 2003. Handbook of communication and social interaction skills. Psychology Press.
[14]
Neha Gupta. 2013. Effective body language in organizations. IUP Journal of Soft Skills 7, 1 (2013), 35.
[15]
Timothy G Hegstrom. 1979. Message impact: What percentage is nonverbal? Western Journal of Communication (includes Communication Reports) 43, 2 (1979), 134--142.
[16]
Joris H Janssen, Wijnand A Ijsselsteijn, and Joyce HDM Westerink. 2014. How affective technologies can influence intimate interactions and improve social connectedness. International Journal of Human-Computer Studies 72, 1 (2014), 33--43.
[17]
Mark L Knapp and John Augustine Daly. 2002. Handbook of interpersonal communication. Sage.
[18]
Michael W Kraus. 2017. Voice-only communication enhances empathic accuracy. American Psychologist 72, 7 (2017), 644.
[19]
Jean-Philippe Laurenceau, Lisa Feldman Barrett, and Paula R Pietromonaco. 1998. Intimacy as an interpersonal process: The importance of self-disclosure, partner disclosure, and perceived partner responsiveness in interpersonal exchanges. Journal of personality and social psychology 74, 5 (1998), 1238.
[20]
James C McCroskey, Virginia P Richmond, and John A Daly. 1975. The development of a measure of perceived homophily in interpersonal communication. Human Communication Research 1, 4 (1975), 323--332.
[21]
Monica McHenry, Patricia A Parker, Walter F Baile, and Renato Lenzi. 2012. Voice analysis during bad news discussion in oncology: reduced pitch, decreased speaking rate, and nonverbal communication of empathy. Supportive Care in Cancer 20, 5 (2012), 1073--1078.
[22]
Clifford Nass and Li Gong. 2000. Speech interfaces from an evolutionary perspective. Commun. ACM 43, 9 (2000), 36--43.
[23]
Hien Nguyen and Judith Masthoff. 2009. Designing empathic computers: the effect of multimodal empathic feedback using animated agent. In Proceedings of the 4th International Conference on Persuasive Technology. ACM, 7.
[24]
Jeff S Philpott. 1983. The relative contribution to meaning of verbal and nonverbal channels of communication: A meta-analysis. Unpublished master's thesis, University of Nebraska, Lincoln (1983).
[25]
Rosalind W Picard and Jonathan Klein. 2002. Computers that recognise and respond to user emotion: theoretical and practical implications. Interacting with computers 14, 2 (2002), 141--169.
[26]
Byron Reeves and Clifford Ivar Nass. 1996. The media equation: How people treat computers, television, and new media like real people and places. Cambridge university press.
[27]
Harry T Reis and Phillip Shaver. 1988. Intimacy as an interpersonal process. Handbook of personal relationships (1988).
[28]
Helen Riess and Gordon Kraft-Todd. 2014. EMPATHY: A tool to enhance nonverbal communication between clinicians and their patients. Academic Medicine 89, 8 (2014), 1108--1112.
[29]
Alfred P Rovai. 2002. Development of an instrument to measure classroom community. The Internet and Higher Education 5, 3 (2002), 197--211.
[30]
Eliot R Smith and Diane M Mackie. 2005. Aggression, hatred, and other emotions. On the nature of prejudice: Fifty years after Allport (2005), 361--376.
[31]
Viswanath Venkatesh, Cheri Speier, and Michael G Morris. 2002. User acceptance enablers in individual decision making about technology: Toward an integrated model. Decision sciences 33, 2 (2002), 297--316.
[32]
Nigel Ward. 2000. The challenge of non-lexical speech sounds. In Sixth International Conference on Spoken Language Processing.
[33]
Nigel Ward. 2006. Non-lexical conversational sounds in American English. Pragmatics & Cognition 14, 1 (2006), 129--182.

Cited By

View all
  • (2024)In Whose Voice?: Examining AI Agent Representation of People in Social Interaction through Generative SpeechProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661555(224-245)Online publication date: 1-Jul-2024
  • (2024)Empathetic Pedagogical Agent: Mitigating Harmful Effects of Negative Feedback Through Self-DisclosureInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2425881(1-18)Online publication date: 15-Nov-2024
  • (2024)Leveraging Social Cues for Patient–Physician Interactions: The Impacts of Empathy, Interactivity and Social Validation in Mobile Medical ConsultationsInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2392966(1-13)Online publication date: 2-Sep-2024
  • Show More Cited By

Index Terms

  1. "I Can Feel Your Empathic Voice": Effects of Nonverbal Vocal Cues in Voice User Interface

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems
    April 2020
    4474 pages
    ISBN:9781450368193
    DOI:10.1145/3334480
    Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 25 April 2020

    Check for updates

    Author Tags

    1. conversational agent
    2. empathic voice
    3. interpersonal agent
    4. nonverbal vocal cues

    Qualifiers

    • Abstract

    Funding Sources

    • Ministry of Education (MOE Korea)
    • National Research Foundation of Korea(NRF)

    Conference

    CHI '20
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

    Upcoming Conference

    CHI 2025
    ACM CHI Conference on Human Factors in Computing Systems
    April 26 - May 1, 2025
    Yokohama , Japan

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)219
    • Downloads (Last 6 weeks)13
    Reflects downloads up to 14 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)In Whose Voice?: Examining AI Agent Representation of People in Social Interaction through Generative SpeechProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661555(224-245)Online publication date: 1-Jul-2024
    • (2024)Empathetic Pedagogical Agent: Mitigating Harmful Effects of Negative Feedback Through Self-DisclosureInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2425881(1-18)Online publication date: 15-Nov-2024
    • (2024)Leveraging Social Cues for Patient–Physician Interactions: The Impacts of Empathy, Interactivity and Social Validation in Mobile Medical ConsultationsInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2392966(1-13)Online publication date: 2-Sep-2024
    • (2024)Voice Assistants - Research LandscapeInformation Systems10.1007/978-3-031-56478-9_2(18-37)Online publication date: 30-Mar-2024
    • (2023)The Use of Voice Assistant for Psychological Assessment Elicits Empathy and Engagement While Maintaining Good Psychometric PropertiesBehavioral Sciences10.3390/bs1307055013:7(550)Online publication date: 2-Jul-2023
    • (2023)Urgency Builds Trust: A Voice Agent's Emotional Expression in an EmergencyCompanion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing10.1145/3584931.3606979(343-347)Online publication date: 14-Oct-2023
    • (2023)Can We Understand Other Drivers? An Investigation on Empathic Triggers through Voice Information Transmission in DrivingAdjunct Proceedings of the 15th International Conference on Automotive User Interfaces and Interactive Vehicular Applications10.1145/3581961.3609901(42-47)Online publication date: 18-Sep-2023
    • (2023)The Bot on Speaking Terms: The Effects of Conversation Architecture on Perceptions of Conversational AgentsProceedings of the 5th International Conference on Conversational User Interfaces10.1145/3571884.3597139(1-16)Online publication date: 19-Jul-2023
    • (2023)Exploring the Use of Colored Ambient Lights to Convey Emotional Cues With Conversational Agents: An Experimental Study2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)10.1109/RO-MAN57019.2023.10309310(99-105)Online publication date: 28-Aug-2023
    • (2023)Waiting Experience: Optimization of Feedback Mechanism of Voice User Interfaces Based on Time PerceptionIEEE Access10.1109/ACCESS.2023.325027811(21241-21251)Online publication date: 2023
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media