Skip to main content
SpringerLink
Log in

Dialogue management in conversational agents through psychology of persuasion and machine learning

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

To be really effective, conversational agents must integrate well with the characteristics of the humans with whom they interact. This exploratory study focuses on a method for integrating well-assessed methods from the field of social psychology in the design of task-oriented conversational agents in which the dialogue management module is developed through machine learning. In particular, the aim is to achieve agents whose policies could take into account the psychological features of the human interactants to deliver personalized and more effective messages. The paper presents the psychological study performed and outlines the overall theoretical architecture of the software framework proposed. On the psychosocial side, we first assessed the effectiveness of differently framed messages aimed to reducing red meat consumption taking the Theory of Planned Behavior (TPB) as the psychosocial model of reference. Turning to the machine learning field, the resulting Structural Equation Model (SEM) was first translated into a probabilistic predictor using Dynamic Bayesian Network (DBN). In turn, such DBN became the fundamental element of a Partially Observable Markov Decision Processes (POMDP) in a reinforcement learning setting. The possibility to elicit complete interaction policies was then studied by applying Neural Monte Carlo Tree Search (Neural MCTS) methods. The results thus obtained introduce the possibility to develop new multidisciplinary and integrated techniques for the development of automated dialogue managing systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Ajzen I (1991) The theory of planned behavior. Organ Behav Hum Decis Process 50(2):179–211. https://doi.org/10.1016/0749-5978(91)90020-t

    Article  Google Scholar 

  2. Allen J, Ferguson G, Stent A (2001) An architecture for more realistic conversational systems. In: Proceedings of the 6th international conference on intelligent user interfaces. https://doi.org/10.1145/359784.359822. ACM, pp 1–8

  3. Bach-Faig A, Berry EM, Lairon D, Reguant J, Trichopoulou A, Dernini S, Medina FX, Battino M, Belahsen R, Miranda G, Serra-Majem L (2011) Mediterranean diet pyramid today. Sci Cult Updates Public Health Nutr 14(12A):2274–2284. https://doi.org/10.1017/s1368980011002515

    Article  Google Scholar 

  4. Ben GI (2007) Bayesian networks. encyclopedia of statistics in quality and reliability. John Wiley & Sons, New York

    Google Scholar 

  5. Bertolotti M, Carfora V, Catellani P (2020) Regulatory focus and the effect of nutritional messages on health and well-being: the case of red meat intake. Appl Psychol Health Well Being 12(1):212–230

    Article  Google Scholar 

  6. Bertolotti M, Carfora V, Catellani P (2020) Different frames to reduce red meat intake: the moderating role of self-efficacy. Health Commun 35 (4):475–482. https://doi.org/10.1111/aphw.12180

    Article  Google Scholar 

  7. Bianchi F, Aveyard P, Astbury NM, Cook B, Cartwright E, Jebb SA (2019) Replacing meat with alternative plant-based products (RE-MAPs): protocol for a randomised controlled trial of a behavioural intervention to reduce meat consumption. BMJ open 9(5):e027016. https://doi.org/10.1136/bmjopen-2018-027016

    Article  Google Scholar 

  8. Bohm I, Lindblom C, Åbacka G, Bengs C, Hörnell A (2015) “He just has to like ham” – The centrality of meat in home and consumer studies. Appetite 95:101–112. https://doi.org/10.1016/j.appet.2015.06.015

    Article  Google Scholar 

  9. Bosone L, Martinez F (2017) When, how and why is loss-framing more effective than gain-and non-gain-framing in the promotion of detection behaviors?. Int Rev Soc Psychol 1(30):184–192. https://doi.org/10.5334/irsp.15

    Article  Google Scholar 

  10. Carfora V, Bertolotti M, Catellani P (2019) Informational and emotional daily messages to reduce red and processed meat consumption. Appetite 141:104331. https://doi.org/10.1080/10410236.2019.1567444

    Article  Google Scholar 

  11. Carfora V, Caso D, Conner M (2017) Randomised controlled trial of a text messaging intervention for reducing processed meat consumption: the mediating roles of anticipated regret and intention. Appetite 117:152–160. https://doi.org/10.1016/j.appet.2017.06.025

    Article  Google Scholar 

  12. Carfora V, Caso D, Palumbo F, Conner M (2018) Promoting water intake. the persuasiveness of a messaging intervention based on anticipated negative affective reactions and self-monitoring. Appetite 130:236–246. https://doi.org/10.1016/j.appet.2018.08.017

    Article  Google Scholar 

  13. Carfora V, Catellani P, Caso D, Conner M (2019) How to reduce red and processed meat consumption by daily text messages targeting environment or health benefits. J Environ Psychol 65:101319. https://doi.org/10.1016/j.jenvp.2019.101319

    Article  Google Scholar 

  14. Caso D, Carfora V (2017) Messaging intervention for promoting self-monitoring of fruit and vegetable consumption — [Un intervento di messaggistica istantanea per la promozione del monitoraggio del consumo di frutta e verdure]. Psicologia della Salute 1:97–111. https://doi.org/10.3280/PDS2017-001005

    Article  Google Scholar 

  15. Cesario J, Corker KS, Jelinek S (2013) A self-regulatory framework for message framing. J Exp Soc Psychol 49:238–249. https://doi.org/10.1016/j.jesp.2012.10.014

    Article  Google Scholar 

  16. Chaslot G, Bakkes S, Szita I, Spronck P (2008) Monte-Carlo tree search: a new framework for game AI. Bijdragen

  17. Cheng T, Woon DK, Lynes JK (2011) The use of message framing in the promotion of environmentally sustainable behaviors. Soc Mar Q 17 (2):48–62. https://doi.org/10.1080/15245004.2011.570859

    Article  Google Scholar 

  18. Corrin T, Papadopoulos A (2017) Understanding the attitudes and perceptions of vegetarian and plant-based diets to shape future health promotion programs. Appetite 10:40–47. https://doi.org/10.1016/j.appet.2016.11.018

    Article  Google Scholar 

  19. Dagum P, Galper A, Horvitz E (1992) Dynamic Network Models For Forecasting. In: Proceedings of the eighth conference on uncertainty in artificial intelligence

  20. de Carolis B, Mazzotta I (2017) A user-adaptive persuasive system based on ‘a-rational’ theory. Int J Hum-Comput St 108:70–88. https://doi.org/10.1016/j.ijhcs.2017.05.005

    Article  Google Scholar 

  21. Dijkstra A, Rothman A, Pietersma S (2011) The persuasive effects of framing messages on fruit and vegetable consumption according to regulatory focus theory. Psychol Health 26(8):1036–1048. https://doi.org/10.1080/08870446.2010.526715

    Article  Google Scholar 

  22. Eagly AH, Chaiken S (1993) The psychology of attitudes. Harcourt Brace Jovanovich College Publishers. https://doi.org/10.1002/mar.4220120509

  23. Eshel G, Martin PA (2006) Diet, energy, and global warming, Earth Interactions (10)1–17. https://doi.org/10.1175/ei167

  24. Fabiani P, Teichteil-Königsbuch F (2010) Markov decision processes in artificial intelligence. Wiley-ISTE, New York

    MATH  Google Scholar 

  25. Farchi S, De Sario M, Lapucci E, Davoli M, Michelozzi P (2017) Meat consumption reduction in Italian regions: health co-benefits and decreases in GHG emissions. Plos One 12(8):e0182960. https://doi.org/10.1371/journal.pone.0182960

    Article  Google Scholar 

  26. Godinho CA, Alvarez MJ, Lima ML (2016) Emphasizing the losses or the gains: comparing situational and individual moderators of framed messages to promote fruit and vegetable intake. Appetite 96:416–425. https://doi.org/10.1016/j.appet.2015.10.001

    Article  Google Scholar 

  27. Graça J, Calheiros MM, Oliveira A (2015) Attached to meat? (Un)Willingness and intentions to adopt a more plant-based diet. Appetite 95:113–125. https://doi.org/10.1016/j.appet.2015.06.024

    Article  Google Scholar 

  28. Hancock GR, Samuelsen KM (2007) Advances in latent variable mixture models IAP

  29. Iacobucci D (2010) Structural equations modeling: Fit indices, sample size, and advanced topics. J Consum Psychol 20:90–98. https://doi.org/10.1016/j.jcps.2009.09.003

    Article  Google Scholar 

  30. Kaelbling PL, Littman M, Anthony RC (1998) Planning and acting in partially observable stochastic domains. Artif Intell 101:99–134

    Article  MathSciNet  Google Scholar 

  31. Katt S, Oliehoek FA, Amato C (2017) Learning in POMDPs with monte carlo tree search proceedings of the 34th international conference on machine learning - Volume 70, ICML 2017

  32. Kline R (1988) Principles and practice of structural equation modeling. Guilford Press, New York

    MATH  Google Scholar 

  33. Kocsis L, Szepesvári C (2006) Bandit based monte-carlo planning. Springer, Berlin, pp 282–293

    Google Scholar 

  34. Laczniak RN, Muehling DD (1993) Toward a better understanding of the role of advertising message involvement in ad processing. P&M 10(4):301–319. https://doi.org/10.1002/mar.4220100405

    Google Scholar 

  35. Liu B (2018) Learning task-oriented dialog with neural network methods. PhD thesis

  36. Misra R, Balagopal P, Raj S, Patel TG (2018) Red meat consumption (heme iron intake) and risk for diabetes and comorbidities?. Curr Diab Rep 18 (11):100. https://doi.org/10.1007/s11892-018-1071-8

    Article  Google Scholar 

  37. Pearl J (1988) Newblock Probabilistic Reasoning in Intelligent systems: Networks of Plausible Inference. Representation and Reasoning Series (2nd printing ed.) Morgan Kaufmann, San Francisco

    Google Scholar 

  38. Petty RE, Cacioppo JT (1986) The elaboration likelihood model of persuasion. In: Communication and persuasion. https://doi.org/10.1007/978-1-4612-4964-1_1. Springer, New York, pp 1–24

  39. Petty RE, Cacioppo JT (2012) Communication and persuasion: central and peripheral routes to attitude change. Springer Science & Business Media, Berlin

    Google Scholar 

  40. Rothman AJ, Bartels RD, Wlaschin J, Salovey P (2006) The strategic use of gain-and loss-framed messages to promote healthy behavior: how theory can inform practice. J Commun 56(suppl_1):S202–S220. https://doi.org/10.1111/j.1460-2466.2006.00290.x

    Article  Google Scholar 

  41. Silver D, Schrittwieser J, Simonyan K, et al. (2017) Mastering the game of go without human knowledge. Nature 550(7676):354–359. https://doi.org/10.1038/nature24270

    Article  Google Scholar 

  42. Silver David, Veness J (2010) Monte-Carlo planning in large POMDPs in advances in neural information processing systems, 23(NIPS 2010) 2164–2172

  43. Stea S, Pickering GL (2019) Optimizing messaging to reduce red meat consumption. Environ Commun 13(5):33–648. https://doi.org/10.1080/17524032.2017.1412994

    Article  Google Scholar 

  44. Stephenson MT, Holbert RL (2003) A Monte Carlo simulation of observable versus latent variable structural equation modeling techniques. Commun Res 30(3):332–354. https://doi.org/10.1177/0093650203030003004

    Article  Google Scholar 

  45. Sutton R, Barto GA (1998) Reinforcement learning: an introduction. IEEE transactions on neural networks / a publication of the IEEE Neural Networks Council 9:1054

    Article  Google Scholar 

  46. Vainio A, Irz X, Hartikainen H (2018) How effective are messages and their characteristics in changing behavioural intentions to substitute plant-based foods for red meat? the mediating role of prior beliefs. Appetite 125:217–224. https://doi.org/10.1016/j.appet.2018.02.002

    Article  Google Scholar 

  47. Weller KE, Greene GW, Redding CA, et al. (2014) Development and validation of green eating behaviors, stage of change, decisional balance, and self-efficacy scales in college students. J Nutr Educ Behav 46(5):324–333. https://doi.org/10.1016/j.jneb.2014.01.002

    Article  Google Scholar 

  48. Wood W (2000) Attitude change: persuasion and social influence. Annu Review Psychol 51(1):539–570. https://doi.org/10.1146/annurev.psych.51.1.539

    Article  Google Scholar 

  49. Zur I, Klöckner C A (2014) Individual motivations for limiting meat consumption. Br Food J 116(4):629642. https://doi.org/10.1108/BFJ-08-2012-0193

    Article  Google Scholar 

Download references

Author information

Author notes

    Authors and Affiliations

    1. Computer Vision and Multimedia Lab, Università degli Studi di Pavia, Pavia, Italy

      Francesca Di Massimo, Rebecca Rastelli & Marco Piastra

    2. Dipartimento di Psicologia, Università Cattolica di Milano, Milan, Italy

      Valentina Carfora & Patrizia Catellani

    Authors
    1. Valentina Carfora
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Francesca Di Massimo
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Rebecca Rastelli
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Patrizia Catellani
      View author publications

      You can also search for this author in PubMed Google Scholar

    5. Marco Piastra
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Valentina Carfora.

    Additional information

    Publisher’s note

    Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

    Valentina Carfora and Francesca Di Massimo contributed equally to this work.

    Appendix: Persuasive messages

    Appendix: Persuasive messages

    Gain

    Non-loss

    Non-gain

    Loss

    Messages

    Messages

    Messages

    Messages

    If you eat little red meat and cold cuts, you will improve the health of your stomach.

    If you eat little red meat and and cold cuts, you will avoid damaging the health of your stomach.

    If you eat much red meat and and cold cuts, you will miss the chance to improve the health of your stomach.

    If you eat much red meat and and cold cuts, you will damage the health of your stomach.

    If you eat little red meat and cold cuts, you will improve the functioning of your bowel.

    If you eat little red meat and and cold cuts, you will avoid damaging the functioning of your bowel.

    If you eat much red meat and and cold cuts, you will miss the opportu-nity to improve the functioning of your bowel.

    If you eat much red meat and and cold cuts, you will damage the functioning of your bowel.

    If you eat little red meat and cold cuts, you will improve the functionality of your heart.

    If you eat little red meat and and cold cuts , you will avoid damaging the functionality of your heart.

    If you eat much red meat and and cold cuts, you will miss the chance to improve the functioning of your heart.

    If you eat much red meat and and cold cuts, you will damage the functionality of your heart.

    If you eat little red meat and cold cuts, you will improve the proper functioning of your arteries.

    If you eat little red meat and and cold cuts, you will avoid increasing the malfunctioning of your arteries.

    If you eat much red meat and and cold cuts, you will miss the opportunity to improve the proper functioning of your arteries.

    If you eat much red meat and and cold cuts, you will increase the malfunctioning of your arteries.

    If you eat little red meat and cold cuts, you will enhance the functionality of your kidneys.

    If you eat little red meat and and cold cuts, you will avoid straining the functionality of your kidneys.

    If you eat much red meat and and cold cuts, you will miss the chance to enhance the functionality of kidneys.

    If you eat much red meat and and cold cuts, you will strain the functionality of your kidneys.

    If you eat little red meat and cold cuts, you will enhance the health of your lungs.

    If you eat little red meat and and cold cuts, you will avoid damaging the health of your lungs.

    If you eat much red meat and and cold cuts, you will miss the opportunity to enhance the health of your lungs.

    If you eat much red meat and and cold cuts, you will damage the health of your lungs.

    If you eat little red meat and cold cuts, you will enhance the health of your pancreas.

    If you eat little red meat and and cold cuts, you will avoid damaging the health of your pancreas.

    If you eat much red meat and and cold cuts, you will miss the chance to enhance the health of your pancreas.

    If you eat much red meat and and cold cuts, you will damage the health of your pancreas.

    If you eat little red meat and cold cuts, you will improve the chance of having an optimal blood pressure.

    If you eat little red meat and and cold cuts, you will decrease the chance of having hypertension.

    If you eat much red meat and and cold cuts, you will miss the chance of having an optimal blood pressure.

    If you eat much red meat and and cold cuts, you will increase the chance of having hypertension.

    Rights and permissions

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Carfora, V., Di Massimo, F., Rastelli, R. et al. Dialogue management in conversational agents through psychology of persuasion and machine learning. Multimed Tools Appl 79, 35949–35971 (2020). https://doi.org/10.1007/s11042-020-09178-w

    Download citation

    • Received:

    • Revised:

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s11042-020-09178-w

    Keywords

    Search

    Navigation