skip to main content
10.1145/3570945.3607361acmconferencesArticle/Chapter ViewAbstractPublication PagesivaConference Proceedingsconference-collections
research-article

Accomodating User Expressivity while Maintaining Safety for a Virtual Alcohol Misuse Counselor

Published: 22 December 2023 Publication History

Abstract

Client-centered counseling, in which individuals are prompted to talk about their behavior, is the standard treatment for Alcohol misuse. However, open-ended conversations with virtual agent counselor raise potential safety concerns if the agent misunderstands and provides erroneous advice. Thus, while generative machine learning models have been successful for language understanding and generation tasks, these approaches may not be effective or safe for counseling. We present a hybrid dialog system that uses a machine-learning model to generate responses to individual client speech combined with a rule-based approach to transition through structured counseling sessions. The dialog system is used to drive a virtual agent alcohol misuse counselor. We evaluated this hybrid system by comparing it to a functionally equivalent system in which the dialog is driven by fully-constraining user utterances via multiple-choice menus among individuals with problematic drinking. Participants who interacted with the agent using the hybrid dialog system reported a higher degree of readiness to change their drinking habits compared to the system using constrained input. Additionally, the outputs of the models used by the system were judged to be safe and appropriate for the task by expert counselors.

References

[1]
Substance Abuse. 2018. Mental Health Services Administration. Key substance use and mental health indicators in the United States: results from the 2017 National Survey on drug use and health (HHS publication no. SMA 18-5068, NSDUH series H-53). Rockville, MD: Center for Behavioral Health Statistics and Quality. Substance Abuse and Mental Health Services Administration 439 (2018), 13.
[2]
Daniel Adiwardana, Minh-Thang Luong, David R So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, et al. 2020. Towards a human-like open-domain chatbot. arXiv preprint arXiv:2001.09977 (2020).
[3]
Aaron T Beck. 1970. Cognitive therapy: Nature and relation to behavior therapy. Behavior therapy 1, 2 (1970), 184--200.
[4]
Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big?. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency. 610--623.
[5]
Timothy Bickmore and Justine Cassell. 2001. Relational agents: a model and implementation of building user trust. In Proceedings of the SIGCHI conference on Human factors in computing systems. 396--403.
[6]
Timothy Bickmore, Ha Trinh, Reza Asadi, and Stefan Olafsson. 2018. Safety first: conversational agents for health care. In Studies in conversational UX design. Springer, 33--57.
[7]
Timothy W Bickmore, Ha Trinh, Stefan Olafsson, Teresa K O'Leary, Reza Asadi, Nathaniel M Rickles, and Ricardo Cruz. 2018. Patient and consumer safety risks when using conversational assistants for medical information: an observational study of Siri, Alexa, and Google Assistant. Journal of medical Internet research 20, 9 (2018), e11510.
[8]
Daniel G Bobrow, Ronald M Kaplan, Martin Kay, Donald A Norman, Henry Thompson, and Terry Winograd. 1977. GUS, a frame-driven dialog system. Artificial intelligence 8, 2 (1977), 155--173.
[9]
Dan Bohus and Alexander Rudnicky. 2003. Ravenclaw: Dialog management using hierarchical task decomposition and an expectation agenda. (2003).
[10]
D Christopher Brooks and Jeffrey Pomerantz. 2017. ECAR Study of Undergraduate Students and Information Technology, 2017. EDUCAUSE (2017).
[11]
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877--1901.
[12]
Kimberly M Caldeira, Sarah J Kasperski, Eva Sharma, Kathryn B Vincent, Kevin E O'Grady, Eric D Wish, and Amelia M Arria. 2009. College students rarely seek help despite serious substance use problems. Journal of substance abuse treatment 37, 4 (2009), 368--378.
[13]
Justine Cassell, Tim Bickmore, Lee Campbell, Hannes Vilhjalmsson, and Hao Yan. 2000. Human conversation as a system framework: Designing embodied conversational agents. Embodied conversational agents (2000), 29--63.
[14]
Justine Cassell, Hannes Högni Vilhjálmsson, and Timothy Bickmore. 2004. Beat: the behavior expression animation toolkit. In Life-Like Characters. Springer, 163--185.
[15]
Herbert H Clark and Susan E Brennan. 1991. Grounding in communication. (1991).
[16]
Jimmy Efird. 2011. Blocked randomization with randomly selected block sizes. International journal of environmental research and public health 8, 1 (2011), 15--20.
[17]
Center for Substance Abuse Treatment et al. 1999. Enhancing motivation for change in substance abuse treatment. Treatment improvement protocol (TIP) series, No. 35. HHS Publication No.(SMA) (1999), 13--4212.
[18]
James Gibson, Dogan Can, Bo Xiao, Zac E Imel, David C Atkins, Panayiotis Georgiou, and Shrikanth Narayanan. 2016. A deep learning approach to modeling empathy in addiction counseling. Commitment 111, 2016 (2016), 21.
[19]
David H Gustafson, Michael G Boyle, Bret R Shaw, Andrew Isham, Fiona McTavish, Stephanie Richards, Christopher Schubert, Michael Levy, and Kim Johnson. 2011. An e-health solution for people with alcohol problems. Alcohol Research & Health 33, 4 (2011), 327.
[20]
Nick Heather, Anna Luce, David Peck, Brenda Dunbar, and Ian James. 1999. Development of a treatment version of the Readiness to Change Questionnaire. Addiction Research 7, 1 (1999), 63--83.
[21]
Clara E Hill and Emilie Y Nakayama. 2000. Client-centered therapy: Where has it been and where is it going? A comment on Hathaway (1948). Journal of Clinical Psychology 56, 7 (2000), 861--875.
[22]
Adam O Horvath and Leslie S Greenberg. 1989. Development and validation of the Working Alliance Inventory. Journal of counseling psychology 36, 2 (1989), 223.
[23]
Wura Jacobs, Ann O Amuta, and Kwon Chan Jeon. 2017. Health information seeking in the digital age: An analysis of health information seeking behavior among US adults. Cogent Social Sciences 3, 1 (2017), 1302785.
[24]
Christopher W Kahler, Amy J Caswell, M Barton Laws, Justin Walthers, Molly Magill, Nadine R Mastroleo, Chanelle J Howe, Timothy Souza, Ira Wilson, Kendall Bryant, et al. 2016. Using topic coding to understand the nature of change language in a motivational intervention to reduce alcohol and sex risk behaviors in emergency department patients. Patient education and counseling 99, 10 (2016), 1595--1602.
[25]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[26]
Cheongjae Lee, Sangkeun Jung, Kyungduk Kim, and Gary Geunbae Lee. 2010. Hybrid approach to robust dialog management using agenda and dialog examples. Computer Speech & Language 24, 4(2010), 609--631.
[27]
Peter Lee, Sebastien Bubeck, and Joseph Petro. 2023. Benefits, limits, and risks of GPT-4 as an AI chatbot for medicine. New England Journal of Medicine 388, 13 (2023), 1233--1239.
[28]
Paul Pu Liang, Chiyu Wu, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2021. Towards understanding and mitigating social biases in language models. In International Conference on Machine Learning. PMLR, 6565--6576.
[29]
Christine Lisetti, Reza Amini, and Ugan Yasavur. 2015. Now all together: overview of virtual health assistants emulating face-to-face health interview experience. KI-Künstliche Intelligenz 29, 2 (2015), 161--172.
[30]
Christine Lisetti, Reza Amini, Ugan Yasavur, and Naphtali Rishe. 2013. I can help you change! an empathic virtual agent delivers behavior change health interventions. ACM Transactions on Management Information Systems (TMIS) 4, 4 (2013), 1--28.
[31]
Gale M Lucas, Jonathan Gratch, Aisha King, and Louis-Philippe Morency. 2014. It's only a computer: Virtual humans increase willingness to disclose. Computers in Human Behavior 37 (2014), 94--100.
[32]
Molly Magill, Lara Ray, Brian Kiluk, Ariel Hoadley, Michael Bernstein, J Scott Tonigan, and Kathleen Carroll. 2019. A meta-analysis of cognitive-behavioral therapy for alcohol or other drug use disorders: Treatment efficacy by contrast condition. Journal of consulting and clinical psychology 87, 12 (2019), 1093.
[33]
David Markland, Richard M Ryan, Vannessa Jayne Tobin, and Stephen Rollnick. 2005. Motivational interviewing and self--determination theory. Journal of social and clinical psychology 24, 6 (2005), 811--831.
[34]
William R Miller and Stephen Rollnick. 2012. Motivational interviewing: Helping people change. Guilford press.
[35]
Adam S Miner, Arnold Milstein, Stephen Schueller, Roshini Hegde, Christina Mangurian, and Eleni Linos. 2016. Smartphone-based conversational agents and responses to questions about mental health, interpersonal violence, and physical health. JAMA internal medicine 176, 5 (2016), 619--625.
[36]
Seth M Noar, Christina N Benac, and Melissa S Harris. 2007. Does tailoring matter? Meta-analytic review of tailored print health behavior change interventions. Psychological bulletin 133, 4 (2007), 673.
[37]
Bahador Nooraei, Charles Rich, and Candace L Sidner. 2014. A real-time architecture for embodied conversational agents: beyond turn-taking. ACHI 14 (2014), 381--388.
[38]
Stefan Olafsson, Byron C Wallace, and Timothy W Bickmore. 2020. Towards a Computational Framework for Automating Substance Use Counseling with Virtual Agents. In AAMAS. 966--974.
[39]
World Health Organization et al. 2001. AUDIT: The alcohol use disorders identification test: Guidelines for use in primary health care. Technical Report. World Health Organization.
[40]
Ashwin Paranjape, Abigail See, Kathleen Kenealy, Haojun Li, Amelia Hardy, Peng Qi, Kaushik Ram Sadagopan, Nguyet Minh Phu, Dilara Soylu, and Christopher D Manning. 2020. Neural generation meets real people: Towards emotionally engaging mixed-initiative conversations. arXiv preprint arXiv:2008.12348 (2020).
[41]
Paola Pedrelli, Brian Borsari, Jennifer E Merrill, Lauren B Fisher, Maren Nyer, Benjamin G Shapero, Amy Farabaugh, Emma R Hayden, M Taylor Levine, Maurizio Fava, et al. 2020. Evaluating the combination of a Brief Motivational Intervention plus Cognitive Behavioral Therapy for Depression and heavy episodic drinking in college students. Psychology of Addictive Behaviors 34, 2 (2020), 308.
[42]
Paola Pedrelli, Brian Borsari, Kathleen M Palm, Elizabeth Dalton, and Maurizio Fava. 2013. Combined MI+ CBT for depressive symptoms and binge drinking among young adults: Two case Studies. Journal of cognitive psychotherapy 27, 3 (2013), 235--257.
[43]
Verónica Pérez-Rosas, Rada Mihalcea, Kenneth Resnicow, Satinder Singh, Lawrence An, Kathy J Goggin, and Delwyn Catley. 2017. Predicting counselor behaviors in motivational interviewing encounters. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. 1128--1137.
[44]
Rafael Zambelli Pinto, Manuela L Ferreira, Vinicius C Oliveira, Marcia R Franco, Roger Adams, Christopher G Maher, and Paulo H Ferreira. 2012. Patient-centred communication is associated with positive therapeutic alliance: a systematic review. Journal of physiotherapy 58, 2 (2012), 77--87.
[45]
Kathryn I Pollak, Stewart C Alexander, James A Tulsky, Pauline Lyna, Cynthia J Coffman, Rowena J Dolor, Pål Gulbrandsen, and Truls Østbye. 2011. Physician empathy and listening: associations with patient satisfaction and autonomy. The Journal of the American Board of Family Medicine 24, 6 (2011), 665--672.
[46]
James O Prochaska. 2020. Transtheoretical model of behavior change. Encyclopedia of behavioral medicine (2020), 2266--2270.
[47]
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8(2019), 9.
[48]
Richard C Rapp, Jiangmin Xu, Carey A Carr, D Tim Lane, Jichuan Wang, and Robert Carlson. 2006. Treatment barriers identified by substance abusers assessed at a centralized intake unit. Journal of substance abuse treatment 30, 3 (2006), 227--235.
[49]
Ken Resnicow and Fiona McMaster. 2012. Motivational Interviewing: moving from why to how with autonomy support. International Journal of Behavioral Nutrition and Physical Activity 9, 1 (2012), 1--9.
[50]
Charles Rich and Candace L Sidner. 1998. COLLAGEN: A collaboration manager for software interface agents. In Computational models of mixed-initiative interaction. Springer, 149--184.
[51]
Stephen Rollnick, Nick Heather, Ruth Gold, and Wayne Hall. 1992. Development of a short 'readiness to change'questionnaire for use in brief, opportunistic interventions among excessive drinkers. British journal of addiction 87, 5 (1992), 743--754.
[52]
Amy Rubin, Nicholas A Livingston, Julianne Brady, Elise Hocking, Timothy Bickmore, Molly Sawdy, Nancy Kressin, Richard Saitz, and Steven Simon. 2022. Computerized relational agent to deliver alcohol brief intervention and referral to treatment in primary care: a randomized clinical trial. Journal of general internal medicine 37, 1 (2022), 70--77.
[53]
John B Saunders, Olaf G Aasland, Thomas F Babor, Juan R De La Fuente, and Marcus Grant. 1993. Development of the alcohol use disorders identification test (AUDIT): WHO collaborative project on early detection of persons with harmful alcohol consumption-II. Addiction 88, 6 (1993), 791--804.
[54]
Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. 2022. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100 (2022).
[55]
Daniel Schulman, Timothy W Bickmore, and Candace L Sidner. 2011. An Intelligent Conversational Agent for Promoting Long-Term Health Behavior Change Using Motivational Interviewing. In AAAI Spring Symposium: AI and Health Communication.
[56]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).
[57]
Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. 2019. Universal adversarial triggers for attacking and analyzing NLP. arXiv preprint arXiv:1908.07125 (2019).
[58]
Lawrence R Wheeless and Janis Grotz. 1977. The measurement of trust and its relationship to self-disclosure. Human Communication Research 3, 3 (1977), 250--257.
[59]
Ramin Yaghoubzadeh and Stefan Kopp. 2016. flexdiam--Flexible dialogue management for incremental interaction with virtual agents (demo paper). In International Conference on Intelligent Virtual Agents. Springer, 505--508.
[60]
Ugan Yasavur, Christine Lisetti, and Naphtali Rishe. 2014. Intelligent virtual agents and spoken dialog systems come together to deliver brief health interventions. Journal on Multimodal User Interfaces 1, 1 (2014), 19.
[61]
S Zhou, T Bickmore, A Rubin, C Yeksigian, R Lippin-Foster, M Heilman, and SR Simon. 2017. A relational agent for alcohol misuse screening and intervention in primary care. In CHI 2017 Workshop on Interactive Systems in Healthcare (WISH).

Cited By

View all
  • (2024)Virtual Agents for Alcohol Use Counseling: Exploring LLM-Powered Motivational InterviewingProceedings of the 24th ACM International Conference on Intelligent Virtual Agents10.1145/3652988.3673932(1-10)Online publication date: 16-Sep-2024
  • (2024)Keeping Users Engaged During Repeated Interviews by a Virtual Agent: Using Large Language Models to Reliably Diversify QuestionsProceedings of the 24th ACM International Conference on Intelligent Virtual Agents10.1145/3652988.3673929(1-10)Online publication date: 16-Sep-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
IVA '23: Proceedings of the 23rd ACM International Conference on Intelligent Virtual Agents
September 2023
376 pages
ISBN:9781450399944
DOI:10.1145/3570945
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 December 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. alcohol use disorder
  2. counseling
  3. dialog system
  4. experiment
  5. natural language processing
  6. neural networks
  7. virtual agents

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

IVA '23
Sponsor:

Acceptance Rates

Overall Acceptance Rate 53 of 196 submissions, 27%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)100
  • Downloads (Last 6 weeks)8
Reflects downloads up to 23 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Virtual Agents for Alcohol Use Counseling: Exploring LLM-Powered Motivational InterviewingProceedings of the 24th ACM International Conference on Intelligent Virtual Agents10.1145/3652988.3673932(1-10)Online publication date: 16-Sep-2024
  • (2024)Keeping Users Engaged During Repeated Interviews by a Virtual Agent: Using Large Language Models to Reliably Diversify QuestionsProceedings of the 24th ACM International Conference on Intelligent Virtual Agents10.1145/3652988.3673929(1-10)Online publication date: 16-Sep-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media