skip to main content
10.1145/3472306.3478358acmconferencesArticle/Chapter ViewAbstractPublication PagesivaConference Proceedingsconference-collections
research-article

Informing the Design of a News Chatbot

Published: 14 September 2021 Publication History

Abstract

Chatbots use conversational interfaces to simulate human communication and recently have been applied to different domains due to advancing techniques of natural language understanding and generation. In particular, in the domain of digital journalism, chatbots provide a new channel for audiences to engage with news. However, most chatbots operated by news organizations have so far failed to achieve business growth. In this paper, we have conducted a qualitative user study to better understand users' attitudes towards news chatbots. Specifically, 15 participants were asked to interact with several news chatbots implemented on Facebook Messenger by large international news organizations (e.g., ABC News, NBC News, and BBC News), by issuing a set of 22 sample questions covering various search and recommendation goals related to COVID-19. Then the participants expressed their expectations of news chatbots and the advantages/disadvantages perceived in using them. From these findings, we derive several design guidelines on effectiveness, informativeness, efficiency, humanization, and facility, suggesting developments to news chatbots that may better serve users' needs.

References

[1]
Eleni Adamopoulou and Lefteris Moussiades. 2020. An Overview of Chatbot Technology. Vol. 584 IFIP. Springer International Publishing. 373-383 pages. https://doi.org/10.1007/978-3-030-49186-4_31
[2]
Ram G. Athreya, Axel-Cyrille Ngonga Ngomo, and Ricardo Usbeck. 2018. Enhancing Community Interactions with Data-Driven Chatbots-The DBpedia Chatbot. In Companion Proceedings of the The Web Conference 2018 (Lyon, France) (WWW '18). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 143--146. https://doi.org/10.1145/3184558.3186964
[3]
Lyn Barnes. 2017. An inexplicable gap: Journalism and gender in New Zealand. Journalism 18, 6 (2017), 736--753. https://doi.org/10.1177/1464884915620231
[4]
Valerie Belair-Gagnon, Seth C Lewis, and Colin Agur. 2020. Failure to Launch: Competing Institutional Logics, Intrapreneurship, and the Case of Chatbots. Journal of Computer-Mediated Communication 25, 4 (jul 2020), 291--306. https://doi.org/10.1093/jcmc/zmaa008
[5]
Facebook Business. 2018. F8 2018: David Marcus' Keynote. Retrieved January 18, 2021 from https://www.facebook.com/business/news/david-marcus-f8-keynote-2018
[6]
James W. Carey. 2019. The press and the public discourse. Kettering Review (winter) (2019), 9--22.
[7]
Veena Chattaraman, Wi-Suk Kwon, and Juan E. Gilbert. 2012. Virtual agents in retail web sites: Benefits of simulated social interaction for older users. Computers in Human Behavior 28, 6 (nov 2012), 2055--2066. https://doi.org/10.1016/j.chb.2012.06.009
[8]
Deborah Compeau, Christopher A. Higgins, and Sid Huff. 1999. Social Cognitive Theory and Individual Reactions to Computing Technology: A Longitudinal Study. MIS quarterly 23, 2 (1999), 145--158. https://doi.org/10.2307/249749
[9]
Mateusz Dubiel, Alessandra Cervone, and Giuseppe Riccardi. 2019. Inquisitive Mind: A Conversational News Companion. In Proceedings of the 1st International Conference on Conversational User Interfaces (Dublin, Ireland) (CUI '19). Association for Computing Machinery, New York, NY, USA, Article 6, 3 pages. https://doi.org/10.1145/3342775.3342802
[10]
Heather Ford and Jonathon Hutchinson. 2019. Newsbots That Mediate Journalist and Audience Relationships. Digital Journalism 7, 8 (2019), 1013--1031. https://doi.org/10.1080/21670811.2019.1626752
[11]
Giampietro Gobo. 2008. Doing ethnography. SAGE, Los Angeles, Calif.; London.
[12]
Cèdric Gossart. 2014. Can digital technologies threaten democracy by creating information cocoons? In Transforming Politics and Policy in the Digital Age, Jonathan Bishop (Ed.). IGI Global, Chapter 10, 145--154. https://doi.org/10.4018/978-1-4666-6038-0.ch010
[13]
Daniel C. Hallin and Paolo Mancini. 2004. Comparing Media Systems: Three Models of Media and Politics. Cambridge University Press. https://doi.org/10.1017/CBO9780511790867
[14]
Shafquat Hussain, Omid Ameri Sianaki, and Nedal Ababneh. 2019. A Survey on Conversational Agents/Chatbots Classification and Design Techniques. In Advances in Intelligent Systems and Computing. Vol. 927. Springer International Publishing, 946--956. https://doi.org/10.1007/978-3-030-15035-8_93
[15]
Becky Inkster, Shubhankar Sarda, and Vinod Subramanian. 2018. An Empathy-Driven, Conversational Artificial Intelligence Agent (Wysa) for Digital Mental Well-Being: Real-World Data Evaluation Mixed-Methods Study. JMIR mHealth and uHealth 6, 11 (nov 2018), e12106. https://doi.org/10.2196/12106
[16]
Mohit Jain, Ramachandra Kota, Pratyush Kumar, and Shwetak Patel. 2018. Convey: Exploring the use of a context view for chatbots. In Conference on Human Factors in Computing Systems-Proceedings, Vol. 2018-April. Association for Computing Machinery, New York, NY, USA, 1--6. https://doi.org/10.1145/3173574.3174042
[17]
Mohit Jain, Pratyush Kumar, Ramachandra Kota, and Shwetak N. Patel. 2018. Evaluating and Informing the Design of Chatbots. In Proceedings of the 2018 Designing Interactive Systems Conference (Hong Kong, China) (DIS '18). Association for Computing Machinery, New York, NY, USA, 895--906. https://doi.org/10.1145/3196709.3196735
[18]
Al Johri, Sam Eui-Hong, and Mehta Dhrumil. 2016. Domain Specific Newsbots - Live Automated Reporting Systems involving Natural Language Communication. Standord Computation Journalism Symposium (2016), 1--5.
[19]
Bronwyn Jones and Rhianne Jones. 2019. Public Service Chatbots: Automating Conversation with BBC News. Digital Journalism 7, 8 (2019), 1032--1053. https://doi.org/10.1080/21670811.2019.1609371 arXiv:https://doi.org/10.1080/21670811.2019.1609371
[20]
Jie Kang, Kyle Condiff, Shuo Chang, Joseph A. Konstan, Loren Terveen, and F. Maxwell Harper. 2017. Understanding How People Use Natural Language to Ask for Recommendations. In Proceedings of the Eleventh ACM Conference on Recommender Systems (Como, Italy) (RecSys '17). Association for Computing Machinery, New York, NY, USA, 229--237. https://doi.org/10.1145/3109859.3109873
[21]
Masamune Kawasaki, Naomi Yamashita, Yi-Chieh Lee, and Kayoko Nohara. 2020. Assessing Users' Mental Status from Their Journaling Behavior through Chatbots. In Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents (Virtual Event, Scotland, UK) (IVA '20). Association for Computing Machinery, New York, NY, USA, Article 32, 8 pages. https://doi.org/10.1145/3383652.3423870
[22]
Bill Kovach and Tom Rosenstiel. 2007. The Elements of Journalism: What News-people Should Know and the Public Should Expect. Three Rivers Press.
[23]
Tetyana Lokot and Nicholas Diakopoulos. 2016. News Bots: Automating news and information dissemination on Twitter. Digital Journalism 4, 6 (2016), 682--699. https://doi.org/10.1080/21670811.2015.1081822
[24]
Ewa Luger and Abigail Sellen. 2016. "Like having a really bad PA": The gulf between user expectation and experience of conversational agents. Conference on Human Factors in Computing Systems - Proceedings (2016), 5286--5297. https://doi.org/10.1145/2858036.2858288
[25]
Theodora A. Maniou and Andreas Veglis. 2020. Employing a chatbot for news dissemination during crisis: Design, implementation and evaluation. Future Internet 12, 12 (2020), 1--14. https://doi.org/10.3390/FI12070109
[26]
Doreen Marchionni. 2015. Online story commenting: An experimental test of conversational journalism and trust. Journalism Practice 9, 2 (mar 2015), 230--249. https://doi.org/10.1080/17512786.2014.938943
[27]
David Marcus. 2016. Messenger Platform at F8. Retrieved February 25, 2021 from https://messengernews.fb.com/2016/04/12/messenger-platform-at-f8/
[28]
Wari Maroengsit, Thanarath Piyakulpinyo, Korawat Phonyiam, Suporn Pongnumkul, Pimwadee Chaovalit, and Thanaruk Theeramunkong. 2019. A Survey on Evaluation Methods for Chatbots. In Proceedings of the 2019 7th International Conference on Information and Education Technology - ICIET 2019, Vol. 29. ACM Press, New York, New York, USA, 111--119. https://doi.org/10.1145/3323771.3323824
[29]
Indrani Medhi Thies, Nandita Menon, Sneha Magapu, Manisha Subramony, and Jacki O'Neill. 2017. How do you want your chatbot? An exploratory Wizard-of-Oz study with young, Urban Indians. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 10513 LNCS (2017), 441--459. https://doi.org/10.1007/978-3-319-67744-6_28
[30]
Chance Miller. 2019. These Were the Most-Downloaded Apps and Games of the Decade. Retrieved January 18, 2021 from https://9to5mac.com/2019/12/16/apps-and-games-of-the-decade/
[31]
Jacob L. Nelson. 2019. The next media regime: The pursuit of 'audience engagement' in journalism. Journalism (jul 2019), 1--18. https://doi.org/10.1177/1464884919862375
[32]
Stefan Olafsson, Teresa K. O'Leary, and Timothy W. Bickmore. 2020. Motivating Health Behavior Change with Humorous Virtual Agents. In Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents (Virtual Event, Scotland, UK) (IVA '20). Association for Computing Machinery, New York, NY, USA, Article 42, 8 pages. https://doi.org/10.1145/3383652.3423915
[33]
Daniel E. Rose and Danny Levinson. 2004. Understanding User Goals in Web Search. In Proceedings of the 13th International Conference on World Wide Web (New York, NY, USA) (WWW '04). Association for Computing Machinery, New York, NY, USA, 13--19. https://doi.org/10.1145/988672.988675
[34]
Cynthia Ross, Karen; Carter. 2011. Women and news: A long and winding road. Media, Culture Society 33, 8 (2011), 1148--1165. https://doi.org/10.1177/0163443711418272
[35]
Nina Svenningsson and Montathar Faraon. 2019. Artificial Intelligence in Conversational Agents: A Study of Factors Related to Perceived Humanness in Chatbots. Dl.Acm.Org (2019), 151--161. https://doi.org/10.1145/3375959.3375973
[36]
Ekaterina Svikhnushina and Pearl Pu. 2020. Social and Emotional Etiquette of Chatbots: A Qualitative Approach to Understanding User Needs and Expectations. (2020). arXiv:2006.13883 http://arxiv.org/abs/2006.13883
[37]
Kristen Vaccaro, Tanvi Agarwalla, Sunaya Shivakumar, and Ranjitha Kumar. 2018. Designing the Future of Personal Fashion. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1--11. https://doi.org/10.1145/3173574.3174201
[38]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc. https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf
[39]
Anbang Xu, Zhe Liu, Yufan Guo, Vibha Sinha, and Rama Akkiraju. 2017. A New Chatbot for Customer Service on Social Media. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI '17). Association for Computing Machinery, New York, NY, USA, 3506--3510. https://doi.org/10.1145/3025453.3025496
[40]
Jennifer Zamora. 2017. I'm Sorry, Dave, I'm Afraid I Can't Do That. In Proceedings of the 5th International Conference on Human Agent Interaction. ACM, New York, NY, USA, 253--260. https://doi.org/10.1145/3125739.3125766
[41]
Darius Zumstein and Sophie Hundertmark. 2018. Chatbots - An Interactive Technology for Personalized Communication, Transactions and Services. IADIS International Journal on WWW/Internet 15 (2018), 96--109.

Cited By

View all
  • (2024)Practical Approach for Smart and Circular Cities: Chatbots Used in Waste RecyclingApplied Sciences10.3390/app1407306014:7(3060)Online publication date: 5-Apr-2024
  • (2024)Towards Designing a Question-Answering Chatbot for Online News: Understanding Questions and PerspectivesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642007(1-17)Online publication date: 11-May-2024
  • (2023)Conversations with the News: Co-speculation into Conversational Interactions with News ContentProceedings of the 5th International Conference on Conversational User Interfaces10.1145/3571884.3597123(1-11)Online publication date: 19-Jul-2023

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
IVA '21: Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents
September 2021
238 pages
ISBN:9781450386197
DOI:10.1145/3472306
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 September 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. News chatbots
  2. conversational agents
  3. design guidelines
  4. qualitative user study

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

Conference

IVA '21
Sponsor:

Acceptance Rates

Overall Acceptance Rate 53 of 196 submissions, 27%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)66
  • Downloads (Last 6 weeks)6
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Practical Approach for Smart and Circular Cities: Chatbots Used in Waste RecyclingApplied Sciences10.3390/app1407306014:7(3060)Online publication date: 5-Apr-2024
  • (2024)Towards Designing a Question-Answering Chatbot for Online News: Understanding Questions and PerspectivesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642007(1-17)Online publication date: 11-May-2024
  • (2023)Conversations with the News: Co-speculation into Conversational Interactions with News ContentProceedings of the 5th International Conference on Conversational User Interfaces10.1145/3571884.3597123(1-11)Online publication date: 19-Jul-2023

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media