ABSTRACT
Text messaging-based conversational systems, popularly called chatbots, have seen massive growth lately. Recent work on evaluating chatbots has found that there exists a mismatch between the chatbot's state of understanding (also called context) and the user's perception of the chatbot's understanding. Users found it difficult to use chatbots for complex tasks as the users were uncertain of the chatbots' intelligence level and contextual state. In this work, we propose Convey (CONtext View), a window added to the chatbot interface, displaying the conversational context and providing interactions with the context values. We conducted a usability evaluation of Convey with 16 participants. Participants preferred using chatbot with Convey and found it to be easier to use, less mentally demanding, faster, and more intuitive compared to a default chatbot without Convey. The paper concludes with a discussion of the design implications offered by Convey.
Supplemental Material
- 2016. jabong. (2016). Retrieved Dec 1, 2017 from https://www.jabong.com/Google Scholar
- 2017. Chatbots. (2017). Retrieved January 11, 2017 from https://chatbottle.co/Google Scholar
- 2017. IBM Watson Conversation. (2017). Retrieved March 1, 2017 from https://www.ibm.com/watson/services/conversation/Google Scholar
- Kathleen Chaykowski. 2016. More Than 11,000 Bots Are Now On Facebook Messenger. (2016). Retrieved December 28, 2016 from http: //www.forbes.com/sites/kathleenchaykowski/2016/07/01/ more-than-11000-bots-are-now-on-facebook-messenger/Google Scholar
- O' Brien Chris. 2016. Facebook Messenger chief says platform's 34,000 chatbots are finally improving user experience. (2016). Retrieved February 7, 2017 from http://venturebeat.com/2016/11/11/ facebook-messenger-chief-says-platforms-34000/ -chatbots-are-finally-improving-user-experience/Google Scholar
- Alan Dix, Tiziana Catarci, Benjamin Habegger, Yannis loannidis, Azrina Kamaruddin, Akrivi Katifori, Giorgos Lepouras, Antonella Poggi, and Devina Ramduny-Ellis. 2006. Intelligent Context-sensitive Interactions on Desktop and the Web. In Proceedings of the International Workshop in Conjunction with AVI 2006 on Context in Advanced Interfaces (CAI '06). 23--27. Google ScholarDigital Library
- Craig Elimeliah. 2016. Why chatbots are replacing apps. (2016). Retrieved January 20, 2017 from http://venturebeat.com/2016/08/02/ why-chatbots-are-replacing-apps/Google Scholar
- Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Advances in psychology 52 (1988), 139--183.Google Scholar
- Mariam Hassib, Daniel Buschek, Pawel W. Wozniak, and Florian Alt. 2017. HeartChat: Heart Rate Augmented Mobile Chat to Support Empathy and Awareness. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). 2239--2251. Google ScholarDigital Library
- BI Intelligence. 2016. Messaging apps are now bigger than social networks. (2016). Retrieved February 7, 2017 from http://www.businessinsider.com/ the-messaging-app-report-2015--11?IR=TGoogle Scholar
- Joonhwan Lee, Soojin Jun, Jodi Forlizzi, and Scott E. Hudson. 2006. Using Kinetic Typography to Convey Emotion in Text-based Interpersonal Communication. In Proceedings of the 6th Conference on Designing Interactive Systems (DIS '06). 41--49. Google ScholarDigital Library
- Vera Q. Liao, Matthew Davis, Werner Geyer, Michael Muller, and N. Sadat Shami. 2016. What Can You Do?: Studying Social-Agent Orientation and Agent Proactive Interactions with an Agent for Employees. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems (DIS '16). 264--275. Google ScholarDigital Library
- Vera Q. Liao, Muhammed Masud Hussain, Praveen Chandar, Matthew Davis, Marco Crasso, Dakuo Wang, Michael Muller, Sadat N. Shami, and Werner Geyer. 2018. All Work and no Play? Conversations with a Question-and-Answer Chatbot in the Wild. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, 13. Google ScholarDigital Library
- Ewa Luger and Abigail Sellen. 2016. "Like Having a Really Bad PA": The Gulf Between User Expectation and Experience of Conversational Agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 5286--5297. Google ScholarDigital Library
- Susan Robinson, David Traum, Midhun Ittycheriah, and Joe Henderer. 2008. What would you ask a conversational agent? Observations of Human-Agent dialogues in a museum setting. In Language Resources and Evaluation Conference (LREC). Marrakech (Morocco).Google Scholar
- Ronald Rosenfeld, Dan Olsen, and Alex Rudnicky. 2001. Universal Speech Interfaces. Interactions 8, 6 (2001), 34--44. Google ScholarDigital Library
- Indrani M Thies, Nandita Menon, Sneha Magapu, Manisha Subramony, and Jacki O'Neill. 2017. How do you want your chatbot? An exploratory Wizard-of-Oz study with young, urban Indians. In Proceedings of the International Conference on Human-Computer Interaction (HCI) (INTERACT '17). IFIP, 20.Google Scholar
- Joseph Weizenbaum. 1966. ELIZA - A computer program for the study of natural language communication between man and machine. Commun. ACM 9, 1 (1966), 36--45. Google ScholarDigital Library
- Tsung-Hsien Wen, Milica Gasic, Nikola Mrksic, Pei-hao Su, David Vandyke, and Steve J. Young. 2015. Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems. CoRR abs/1508.01745 (2015).Google Scholar
- Tom Yeh, Tsung-Hsiang Chang, Bo Xie, Greg Walsh, Ivan Watkins, Krist Wongsuphasawat, Man Huang, Larry S. Davis, and Benjamin B. Bederson. 2011. Creating Contextual Help for GUIs Using Screenshots. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST '11). 145--154. Google ScholarDigital Library
Index Terms
Convey: Exploring the Use of a Context View for Chatbots
Recommendations
The Chatbot Usability Scale: the Design and Pilot of a Usability Scale for Interaction with AI-Based Conversational Agents
AbstractStandardised tools to assess a user’s satisfaction with the experience of using chatbots and conversational agents are currently unavailable. This work describes four studies, including a systematic literature review, with an overall sample of 141 ...
Grounding and turn-taking in multimodal multiparty conversation
HCI'13: Proceedings of the 15th international conference on Human-Computer Interaction: interaction modalities and techniques - Volume Part IVThis study explores the empirical basis for multimodal conversation control acts. Applying conversation analysis as an exploratory approach, we attempt to illuminate the control functions of paralinguistic behaviors in managing multiparty conversation. ...
Dialog behaviors across culture and group size
UAHCI'11: Proceedings of the 6th international conference on Universal access in human-computer interaction: users diversity - Volume Part IIThis study analyzes joint interaction behaviors of two-person and four-person standing conversations from three different cultures, American, Arab, and Mexican. To determine whether people use joint interaction behaviors differently in multiparty versus ...
Comments