Adam Brandt
ABSTRACT
Conversation design at least partly aspires to create Voice User Interfaces which emulate human speech production. And yet, there is no established approach for the development of naturalistic conversational infrastructure for VUIs; conversation designers are advised to work from their common sense understanding of conversation, producing written scripts, based on memory and imagination, which are later converted into speech. This is a shortcoming in conversation design which needs to be addressed. In this provocation paper, we argue that the starting point in the development of any VUI should be the examination of natural spoken conversation, preferably from the same interactional context in which the VUI will be deployed. We provide a short example to illustrate how the current process of conversation scriptwriting can be a barrier to this, and demonstrate how this can be overcome using the social scientific approach of Conversation Analysis (CA).
- Natural Speech | Alexa Design Guide. https://developer.amazon.com/en-US/alexa/alexa-haus/natural-speechGoogle Scholar
- Conversation Design. https://developers.google.com/assistant/ conversation- design/welcomeGoogle Scholar
- Saul Albert and Magnus Hamann. 2021. Putting wake words to bed: We speak wake words with systematically varied prosody, but CUIs don't listen. In Proceedings of the 3rd Conference on Conversational User Interfaces (CUI '21). Association for Computing Machinery, New York, NY, USA, Article 13, 1–5. https://doi.org/10.1145/3469595.3469608Google ScholarDigital Library
- Saul Albert, William Housley, and Elizabeth Stokoe. 2019. In case of emergency, order pizza: an urgent case of action formation and recognition. In Proceedings of the 1st International Conference on Conversational User Interfaces (CUI '19). Association for Computing Machinery, New York, NY, USA, Article 15, 1–2. https://doi.org/10.1145/3342775.3342800Google ScholarDigital Library
- Iuliia Avgustis, Aleksandr Shirokov, and Netta Iivari. 2021. “Please Connect Me to a Specialist”: Scrutinising ‘Recipient Design’ in Interaction with an Artificial Conversational Agent. In Human-Computer Interaction – INTERACT 2021: 18th IFIP TC 13 International Conference, Bari, Italy, August 30 – September 3, 2021, Proceedings, Part IV. Springer-Verlag, Berlin, Heidelberg, 155–176. https://doi.org/10.1007/978-3-030-85610-6_10Google ScholarDigital Library
- Leigh Clark, Nadia Pantidi, Orla Cooney, Philip Doyle, Diego Garaialde, Justin Edwards, Brendan Spillane, Emer Gilmartin, Christine Murad, Cosmin Munteanu, Vincent Wade, and Benjamin R. Cowan. 2019. What Makes a Good Conversation? Challenges in Designing Truly Conversational Agents. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). Association for Computing Machinery, New York, NY, USA, Paper 475, 1–12. https://doi.org/10.1145/3290605.3300705Google ScholarDigital Library
- Benjamin R. Cowan, Leigh Clark, Heloisa Candello and Janice Tsai. 2023. Introduction to this special issue: guiding the conversation: new theory and design perspectives for conversational user interfaces, Human–Computer Interaction. 38, 3-4 (2023), 159-167. DOI: 10.1080/07370024.2022.2161905Google ScholarCross Ref
- Paul Drew and John Heritage. 1992. Talk at Work: Interaction in Institutional Settings. Cambridge University Press.Google Scholar
- Joel E. Fischer, Stuart Reeves, Martin Porcheron, and Rein Ove Sikveland. 2019. Progressivity for voice interface design. In Proceedings of the 1st International Conference on Conversational User Interfaces (CUI '19). Association for Computing Machinery, New York, NY, USA, Article 26, 1–8. https://doi.org/10.1145/3342775.3342788Google ScholarDigital Library
- Spencer Hazel and Adam Brandt. forthcoming. Enhancing the Natural Conversation Experience through Conversation Analysis – a design method. In HCI International 2022–Late Breaking Papers. 25th International Conference on Human-Computer Interaction, HCII 2023, 23-28 July. Springer.Google Scholar
- John Heritage and Steven Clayman. 2010. Talk in Action: Interactions, Identities, and Institutions. Wiley-Blackwell.Google Scholar
- William Housley, Saul Albert, and Elizabeth Stokoe. 2019. Natural Action Processing. In Proceedings of the Halfway to the Future Symposium 2019 (HTTF 2019). Association for Computing Machinery, New York, NY, USA, Article 34, 1–4. https://doi.org/10.1145/3363384.3363478Google ScholarDigital Library
- Hanneke Houtkoop-Steenstra. 1991. Opening sequences in Dutch telephone conversations. In Talk and social structure: Studies in ethnomethodology and conversation analysis, Deirdre Boden and Don H. Zimmerman (Eds). Cambridge: Polity, 232–50.
Google Scholar
- Yelim Kim, Mohi Reza, Joanna McGrenere, and Dongwook Yoon. 2021. Designers Characterize Naturalness in Voice User Interfaces: Their Goals, Practices, and Challenges. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI '21). Association for Computing Machinery, New York, NY, USA, Article 242, 1–13. https://doi.org/10.1145/3411764.3445579Google ScholarDigital Library
- Eric Laurier. 2001. Why People Say Where They are during Mobile Phone Calls. Environment and Planning D: Society and Space, 19, 4 (2001). 485–504. https://doi.org/10.1068/d228tGoogle ScholarCross Ref
- Robert J. Moore, Margaret H. Szymanski, Raphael Arar, Guang-Jie Ren (Eds) 2018. Studies in Conversational UX Design. Springer.Google Scholar
- Robert J. Moore and Raphael Arar. 2019. Conversational UX Design: A Practitioner's Guide to the Natural Conversation Framework. Association for Computing Machinery, New York, NY, USA.Google Scholar
- Robert J. Moore, Sungeun An and Guang-Jie Ren. 2023. The IBM natural conversation framework: a new paradigm for conversational UX design, Human–Computer Interaction. 3, 3-4 (2023). 168-193. DOI: 10.1080/07370024.2022.2081571Google ScholarCross Ref
- Christine Murad and Cosmin Munteanu. 2019. "I don't know what you're talking about, HALexa": the case for voice user interface guidelines. In Proceedings of the 1st International Conference on Conversational User Interfaces (CUI '19). Association for Computing Machinery, New York, NY, USA, Article 9, 1–3. https://doi.org/10.1145/3342775.3342795Google ScholarDigital Library
- Christine Murad and Cosmin Munteanu. 2020. Designing Voice Interfaces: Back to the (Curriculum) Basics. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376522Google ScholarDigital Library
- Christine Murad, Cosmin Munteanu, Benjamin R. Cowan, and Leigh Clark. 2021. Finding a New Voice: Transitioning Designers from GUI to VUI Design. In Proceedings of the 3rd Conference on Conversational User Interfaces (CUI '21). Association for Computing Machinery, New York, NY, USA, Article 22, 1–12. https://doi.org/10.1145/3469595.3469617Google ScholarDigital Library
- Christine Murad, Cosmin Munteanu, Benjamin R. Cowan, Leigh Clark, Martin Porcheron, Heloisa Candello, Stephan Schlögl, Matthew P. Aylett, Jaisie Sin, Robert J. Moore, Grace Hughes, and Andrew Ku. 2021. Let's Talk About CUIs: Putting Conversational User Interface Design Into Practice. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (CHI EA '21). Association for Computing Machinery, New York, NY, USA, Article 98, 1–6. https://doi.org/10.1145/3411763.3441336Google ScholarDigital Library
- Christine Murad, Humaira Tasnim, and Cosmin Munteanu. 2022. “Voice-First Interfaces in a GUI-First Design World”: Barriers and Opportunities to Supporting VUI Designers On-the-Job. In Proceedings of the 4th Conference on Conversational User Interfaces (CUI '22). Association for Computing Machinery, New York, NY, USA, Article 17, 1–10. https://doi.org/10.1145/3543829.3543842Google ScholarDigital Library
- Cathy Pearl, Saul Albert, and Elizabeth Stokoe. 2022. What insights from conversation analysis can, should, and should not be leveraged when collaborating in conversation design? 4th International Conference on Conversational User Interfaces (CUI ’22), Glasgow, UK. 26-28 JulyGoogle Scholar
- Hannah R.M. Pelikan and Mathias Broth. 2016. Why That Nao? How Humans Adapt to a Conventional Humanoid Robot in Taking Turns-at-Talk. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). Association for Computing Machinery, New York, NY, USA, 4921–4932. https://doi.org/10.1145/2858036.2858478Google ScholarDigital Library
- Hannah R. M. Pelikan, Mathias Broth, and Leelo Keevallik. 2020. "Are You Sad, Cozmo?": How Humans Make Sense of a Home Robot's Emotion Displays. In Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (HRI '20). Association for Computing Machinery, New York, NY, USA, 461–470. https://doi.org/10.1145/3319502.3374814Google ScholarDigital Library
- Martin Porcheron, Joel E. Fischer, Stuart Reeves, and Sarah Sharples. 2018. Voice Interfaces in Everyday Life. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). Association for Computing Machinery, New York, NY, USA, Paper 640, 1–12. https://doi.org/10.1145/3173574.3174214Google ScholarDigital Library
- Stuart Reeves, Martin Porcheron, Joel E. Fischer, Heloisa Candello, Donald McMillan, Moira McGregor, Robert J. Moore, Rein Sikveland, Alex S. Taylor, Julia Velkovska, and Moustafa Zouinar. 2018. Voice-based Conversational UX Studies and Design. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA '18). Association for Computing Machinery, New York, NY, USA, Paper W38, 1–8. https://doi.org/10.1145/3170427.3170619Google ScholarDigital Library
- Stuart Reeves, and Martin Porcheron. 2022. Conversational AI: Respecifying participation as regulation. In The SAGE Handbook of Digital Society, William Housley, Adam Edwards, Roser Beneito-Montagut, and Richard Fitzgerald (Eds). SAGE.Google Scholar
- Harvey Sacks. 1984, Notes on methodology. In Structures of Social Action: Studies in Conversation Analysis, John Heritage, and J. Maxwell Atkinson (Eds). Cambridge: Cambridge University Press, 2–27.Google Scholar
- Emanuel A. Schegloff. 1986. The routine as achievement. Human Studies. 9, 2 (1986). 111–151.Google Scholar
- Rein Sikveland, Elizabeth Stokoe, Jon Symonds. 2016. Patient burden during appointment-making telephone calls to GP practices. Patient Education and Counselling. 99, 8 (2016). 1310-1318.Google Scholar
- Elizabeth Stokoe, Saul Albert, Sophie Parslow, and Cathy Pearl. 2021. Conversation design and conversation analysis: Where the moonshots are. Medium. https://elizabeth-stokoe.medium.com/conversation-design-and-conversation-analysis-c2a2836cb042Google Scholar
- Sylvaine Tuncer, Christian Licoppe, Paul Luff, and Christian Heath. 2023. Recipient design in human–robot interaction: the emergent assessment of a robot's competence. AI & Society. https://doi.org/10.1007/s00146-022-01608-7Google ScholarCross Ref
Index Terms
- From Writing Dialogue to Designing Conversation: Considering the potential of Conversation Analysis for Voice User Interfaces
Recommendations
Enhancing the Natural Conversation Experience Through Conversation Analysis – A Design Method
HCI International 2023 – Late Breaking PapersAbstractAs Voice User Interfaces (VUIs) become increasingly embedded in a wide range of activities in our everyday life, it falls to the conversation designer to ensure that the user experience is a satisfactory one. Embedding qualities of natural speech ...
Discourse Analysis in Voice User Interface Research: Examining Current and Future Applications of Conversation Analysis and Interactional Sociolinguistics
CUI '21: Proceedings of the 3rd Conference on Conversational User InterfacesThis study examines how two approaches within discourse analysis, Conversation Analysis (CA) and Interactional Sociolinguistics (IS), have been and can be applied to voice user interface (VUI) research. I review how CA has been adapted as a key tool ...
Effect of Speech Entrainment in Human-Computer Conversation: A Review
Intelligent Human Computer InteractionAbstractThe phenomenon of entrainment in conversation is the process where participants become more similar to each other in terms of different verbal and non-verbal aspects such as acoustic-prosodic, lexical, syntactic, pitch, and speech rate. This ...
Comments