research-article

Convey: Exploring the Use of a Context View for Chatbots

Authors:

Ramachandra Kota,

Pratyush Kumar,

Shwetak N. PatelAuthors Info & Claims

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

Paper No.: 468, Pages 1 - 6

https://doi.org/10.1145/3173574.3174042

Published: 21 April 2018 Publication History

Abstract

Text messaging-based conversational systems, popularly called chatbots, have seen massive growth lately. Recent work on evaluating chatbots has found that there exists a mismatch between the chatbot's state of understanding (also called context) and the user's perception of the chatbot's understanding. Users found it difficult to use chatbots for complex tasks as the users were uncertain of the chatbots' intelligence level and contextual state. In this work, we propose Convey (CONtext View), a window added to the chatbot interface, displaying the conversational context and providing interactions with the context values. We conducted a usability evaluation of Convey with 16 participants. Participants preferred using chatbot with Convey and found it to be easier to use, less mentally demanding, faster, and more intuitive compared to a default chatbot without Convey. The paper concludes with a discussion of the design implications offered by Convey.

Supplementary Material

suppl.mov (pn3826-file3.mp4)

Supplemental video

Download
724.66 KB

suppl.mov (pn3826-file5.mp4)

Supplemental video

Download
1.02 MB

MP4 File (pn3826.mp4)

Download
225.09 MB

References

[1]

2016. jabong. (2016). Retrieved Dec 1, 2017 from https://www.jabong.com/

[2]

2017. Chatbots. (2017). Retrieved January 11, 2017 from https://chatbottle.co/

[3]

2017. IBM Watson Conversation. (2017). Retrieved March 1, 2017 from https://www.ibm.com/watson/services/conversation/

[4]

Kathleen Chaykowski. 2016. More Than 11,000 Bots Are Now On Facebook Messenger. (2016). Retrieved December 28, 2016 from http: //www.forbes.com/sites/kathleenchaykowski/2016/07/01/ more-than-11000-bots-are-now-on-facebook-messenger/

[5]

O' Brien Chris. 2016. Facebook Messenger chief says platform's 34,000 chatbots are finally improving user experience. (2016). Retrieved February 7, 2017 from http://venturebeat.com/2016/11/11/ facebook-messenger-chief-says-platforms-34000/ -chatbots-are-finally-improving-user-experience/

[6]

Alan Dix, Tiziana Catarci, Benjamin Habegger, Yannis loannidis, Azrina Kamaruddin, Akrivi Katifori, Giorgos Lepouras, Antonella Poggi, and Devina Ramduny-Ellis. 2006. Intelligent Context-sensitive Interactions on Desktop and the Web. In Proceedings of the International Workshop in Conjunction with AVI 2006 on Context in Advanced Interfaces (CAI '06). 23--27.

Digital Library

[7]

Craig Elimeliah. 2016. Why chatbots are replacing apps. (2016). Retrieved January 20, 2017 from http://venturebeat.com/2016/08/02/ why-chatbots-are-replacing-apps/

[8]

Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Advances in psychology 52 (1988), 139--183.

[9]

Mariam Hassib, Daniel Buschek, Pawel W. Wozniak, and Florian Alt. 2017. HeartChat: Heart Rate Augmented Mobile Chat to Support Empathy and Awareness. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). 2239--2251.

Digital Library

[10]

BI Intelligence. 2016. Messaging apps are now bigger than social networks. (2016). Retrieved February 7, 2017 from http://www.businessinsider.com/ the-messaging-app-report-2015--11?IR=T

[11]

Joonhwan Lee, Soojin Jun, Jodi Forlizzi, and Scott E. Hudson. 2006. Using Kinetic Typography to Convey Emotion in Text-based Interpersonal Communication. In Proceedings of the 6th Conference on Designing Interactive Systems (DIS '06). 41--49.

Digital Library

[12]

Vera Q. Liao, Matthew Davis, Werner Geyer, Michael Muller, and N. Sadat Shami. 2016. What Can You Do?: Studying Social-Agent Orientation and Agent Proactive Interactions with an Agent for Employees. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems (DIS '16). 264--275.

Digital Library

[13]

Vera Q. Liao, Muhammed Masud Hussain, Praveen Chandar, Matthew Davis, Marco Crasso, Dakuo Wang, Michael Muller, Sadat N. Shami, and Werner Geyer. 2018. All Work and no Play? Conversations with a Question-and-Answer Chatbot in the Wild. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, 13.

Digital Library

[14]

Ewa Luger and Abigail Sellen. 2016. "Like Having a Really Bad PA": The Gulf Between User Expectation and Experience of Conversational Agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 5286--5297.

Digital Library

[15]

Susan Robinson, David Traum, Midhun Ittycheriah, and Joe Henderer. 2008. What would you ask a conversational agent? Observations of Human-Agent dialogues in a museum setting. In Language Resources and Evaluation Conference (LREC). Marrakech (Morocco).

[16]

Ronald Rosenfeld, Dan Olsen, and Alex Rudnicky. 2001. Universal Speech Interfaces. Interactions 8, 6 (2001), 34--44.

Digital Library

[17]

Indrani M Thies, Nandita Menon, Sneha Magapu, Manisha Subramony, and Jacki O'Neill. 2017. How do you want your chatbot? An exploratory Wizard-of-Oz study with young, urban Indians. In Proceedings of the International Conference on Human-Computer Interaction (HCI) (INTERACT '17). IFIP, 20.

[18]

Joseph Weizenbaum. 1966. ELIZA - A computer program for the study of natural language communication between man and machine. Commun. ACM 9, 1 (1966), 36--45.

Digital Library

[19]

Tsung-Hsien Wen, Milica Gasic, Nikola Mrksic, Pei-hao Su, David Vandyke, and Steve J. Young. 2015. Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems. CoRR abs/1508.01745 (2015).

[20]

Tom Yeh, Tsung-Hsiang Chang, Bo Xie, Greg Walsh, Ivan Watkins, Krist Wongsuphasawat, Man Huang, Larry S. Davis, and Benjamin B. Bederson. 2011. Creating Contextual Help for GUIs Using Screenshots. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST '11). 145--154.

Digital Library

Cited By

Chaitra CUpadhyay PChakraborty D(2025)Is ChatGPT Ready for Indian-Language Speakers? Findings From a Preliminary Mixed Methods StudyHuman-Computer Interaction. Design and Research10.1007/978-3-031-80829-6_9(193-214)Online publication date: 14-Feb-2025
https://doi.org/10.1007/978-3-031-80829-6_9
Cheng YWang Y(2025)Leveraging Artificial Intelligence–Powered Chatbots for Nonprofit Organizations: Examining the Antecedents and Outcomes of Chatbot Trust and Social Media EngagementJournal of Philanthropy10.1002/nvsm.7001330:1Online publication date: 23-Feb-2025
https://doi.org/10.1002/nvsm.70013
Abed S(2024)Understanding the Determinants of Using Government AI-Chatbots by Citizens in Saudi ArabiaInternational Journal of Electronic Government Research10.4018/IJEGR.34973320:1(1-20)Online publication date: 7-Aug-2024
https://dl.acm.org/doi/10.4018/IJEGR.349733
Show More Cited By

Index Terms

Convey: Exploring the Use of a Context View for Chatbots
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

The Chatbot Usability Scale: the Design and Pilot of a Usability Scale for Interaction with AI-Based Conversational Agents
Abstract
Standardised tools to assess a user’s satisfaction with the experience of using chatbots and conversational agents are currently unavailable. This work describes four studies, including a systematic literature review, with an overall sample of 141 ...
Grounding and turn-taking in multimodal multiparty conversation
HCI'13: Proceedings of the 15th international conference on Human-Computer Interaction: interaction modalities and techniques - Volume Part IV

This study explores the empirical basis for multimodal conversation control acts. Applying conversation analysis as an exploratory approach, we attempt to illuminate the control functions of paralinguistic behaviors in managing multiparty conversation. ...
ADiEU: Toward Domain-Based Evaluation of Spoken Dialog Systems
Proceedings of the 13th International Conference on Human-Computer Interaction. Part I: New Trends

We propose a new approach toward evaluation of spoken dialog systems. The novelty of our method is based on utilization of domain-specific knowledge combined with the deterministic measurement of dialog system performance on a set of individual tasks ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

April 2018

8489 pages

ISBN:9781450356206

DOI:10.1145/3173574

General Chairs:
Regan Mandryk
University of Saskatchewan, Canada
,
Mark Hancock
University of Waterloo, Canada
,
Program Chairs:
Mark Perry
Brunel University London, UK
,
Anna Cox
University College London, UK

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 April 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '18

Sponsor:

SIGCHI

CHI '18: CHI Conference on Human Factors in Computing Systems

April 21 - 26, 2018

Montreal QC, Canada

Acceptance Rates

CHI '18 Paper Acceptance Rate 666 of 2,590 submissions, 26%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

55
Total Citations
View Citations
1,651
Total Downloads

Downloads (Last 12 months)152
Downloads (Last 6 weeks)19

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chaitra CUpadhyay PChakraborty D(2025)Is ChatGPT Ready for Indian-Language Speakers? Findings From a Preliminary Mixed Methods StudyHuman-Computer Interaction. Design and Research10.1007/978-3-031-80829-6_9(193-214)Online publication date: 14-Feb-2025
https://doi.org/10.1007/978-3-031-80829-6_9
Cheng YWang Y(2025)Leveraging Artificial Intelligence–Powered Chatbots for Nonprofit Organizations: Examining the Antecedents and Outcomes of Chatbot Trust and Social Media EngagementJournal of Philanthropy10.1002/nvsm.7001330:1Online publication date: 23-Feb-2025
https://doi.org/10.1002/nvsm.70013
Abed S(2024)Understanding the Determinants of Using Government AI-Chatbots by Citizens in Saudi ArabiaInternational Journal of Electronic Government Research10.4018/IJEGR.34973320:1(1-20)Online publication date: 7-Aug-2024
https://dl.acm.org/doi/10.4018/IJEGR.349733
Do HBrachman MDugan CJohnson JLauer JRai PPan Q(2024)Grounding with Structure: Exploring Design Variations of Grounded Human-AI Collaboration in a Natural Language InterfaceProceedings of the ACM on Human-Computer Interaction10.1145/36869028:CSCW2(1-27)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686902
Cannanure VNgoon TWolf SJasińska KBrown TOgan A(2024)Understanding the Longitudinal Impact of a Chatbot to Facilitate a Virtual Community of Practice for Teachers in Rural Côte d’IvoireACM Journal on Computing and Sustainable Societies10.1145/36757622:3(1-37)Online publication date: 16-Sep-2024
https://dl.acm.org/doi/10.1145/3675762
Liu YTing ILiberona DHsu M(2024)Personalized User Interface Design For Psychological Counseling Chatbots: A Pilot StudyProceedings of the 2024 11th Multidisciplinary International Social Networks Conference10.1145/3675669.3675683(23-28)Online publication date: 21-Aug-2024
https://dl.acm.org/doi/10.1145/3675669.3675683
Hong JLee YKim DChoi DYoon YLee GLee ZKim J(2024)A Context-Aware Onboarding Agent for Metaverse Powered by Large Language ModelsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661579(1857-1874)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661579
Alghamdi EHalvey MNicol E(2024)System and User Strategies to Repair Conversational Breakdowns of Spoken Dialogue Systems: A Scoping ReviewProceedings of the 6th ACM Conference on Conversational User Interfaces10.1145/3640794.3665558(1-13)Online publication date: 8-Jul-2024
https://dl.acm.org/doi/10.1145/3640794.3665558
Mildner TCooney OMeck ABartl MSavino GDoyle PGaraialde DClark LSloan JWenig NMalaka RNiess J(2024)Listening to the Voices: Describing Ethical Caveats of Conversational User Interfaces According to Experts and Frequent UsersProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642542(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642542
Jo EJeong YPark SEpstein DKim Y(2024)Understanding the Impact of Long-Term Memory on Self-Disclosure with Large Language Model-Driven Chatbots for Public Health InterventionProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642420(1-21)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642420
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten