Skip to main content
SpringerLink
Log in

Exploring the Effect of Visual-Based Subliminal Persuasion in Public Speeches Using Explainable AI Techniques

  • Conference paper
  • First Online:
Artificial Intelligence in HCI (HCII 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14050))

Included in the following conference series:

  • 1293 Accesses

Abstract

When it comes to persuading other people, non-verbal cues play an important role in order to be successful. Mostly, people use these non-verbal cues subconsciously and, from the perspective of the persuadee, are not aware of the subliminal impact of them. To raise awareness of subliminal persuasion, we analyzed videos of different political public speeches. We used the labels of three annotators to train three subjective neural networks capable of predicting their degree of perceived persuasiveness based on the images as input only. We then created visualizations of the predictions for each network/annotator to draw conclusions about what the annotators have most likely focused on. For that, we employed layer-wise relevance propagation (LRP) that highlights the most relevant image sections for each prediction. Our results show that techniques like LRP can help uncover existing subliminal bias.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://dictionary.apa.org/subliminal-persuasion.

References

  1. Adadi, A., Berrada, M.: Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE Access 6, 52138–52160 (2018)

    Article  Google Scholar 

  2. Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., Kim, B.: Sanity checks for saliency maps. In: Advances in Neural Information Processing Systems 31, pp. 9505–9515. Curran Associates, Inc. (2018)

    Google Scholar 

  3. Ahmad, W.N.W., Ali, N.M.: A study on persuasive technologies. The relationship between user emotions, trust and persuasion. Int. J. Interact. Multimedia Artif. Intell. 5(1), 57 (2018). https://doi.org/10.9781/ijimai.2018.02.010. http://www.ijimai.org/journal/node/2156

  4. Alber, M., et al.: iNNvestigate neural networks! J. Mach. Learn. Res. 20(93), 1–8 (2019). http://jmlr.org/papers/v20/18-540.html

  5. Andrist, S., Spannan, E., Mutlu, B.: Rhetorical robots: making robots more effective speakers using linguistic cues of expertise. In: 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 341–348. IEEE (2013)

    Google Scholar 

  6. Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.R., Samek, W.: On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLOS ONE 10(7), e0130140 (2015)

    Google Scholar 

  7. Burgoon, J.K., Birk, T., Pfau, M.: Nonverbal behaviors, persuasion, and credibility. Hum. Commun. Res. 17(1), 140–169 (1990). https://doi.org/10.1111/j.1468-2958.1990.tb00229.x. https://academic.oup.com/hcr/article/17/1/140-169/4575795

  8. Buss, D.M.: Selection, evocation, and manipulation. J. Pers. Soc. Psychol. 53(6), 1214–1221 (1987)

    Article  Google Scholar 

  9. Cao, Z., Hidalgo Martinez, G., Simon, T., Wei, S., Sheikh, Y.A.: OpenPose: realtime multi-person 2d pose estimation using part affinity fields. IEEE Transactions on Pattern Analysis and Machine Intelligence (2019)

    Google Scholar 

  10. Chaiken, S.: Heuristic and systematic information processing within and beyond the persuasion context. Unintended Thought, pp. 212–252 (1989)

    Google Scholar 

  11. Chidambaram, V., Chiang, Y.H., Mutlu, B.: Designing persuasive robots: how robots might persuade people using vocal and nonverbal cues. In: Proceedings of the Seventh Annual ACM/IEEE International Conference On Human-robot Interaction, pp. 293–300 (2012)

    Google Scholar 

  12. DeSteno, D., Petty, R.E., Rucker, D.D., Wegener, D.T., Braverman, J.: Discrete emotions and persuasion: the role of emotion-induced expectancies. J. Pers. Soc. Psychol. 86(1), 43 (2004)

    Article  Google Scholar 

  13. Donadello, I., Dragoni, M., Eccher, C.: Persuasive explanation of reasoning inferences on dietary data. In: Demidova, E., et al. (eds.) Joint Proceedings of the 6th International Workshop on Dataset PROFlLing and Search & the 1st Workshop on Semantic Explainability co-located with the 18th International Semantic Web Conference (ISWC 2019), Auckland, New Zealand, 27 October 2019. CEUR Workshop Proceedings, vol. 2465, pp. 46–61. CEUR-WS.org (2019)

    Google Scholar 

  14. Escalante, H.J., et al.: Design of an explainable machine learning challenge for video interviews. In: 2017 International Joint Conference on Neural Networks, IJCNN 2017, Anchorage, AK, USA, 14–19 May 2017, pp. 3688–3695. IEEE (2017)

    Google Scholar 

  15. Greydanus, S., Koul, A., Dodge, J., Fern, A.: Visualizing and understanding Atari agents. In: Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, pp. 1787–1796 (2018)

    Google Scholar 

  16. Ham, J., Bokhorst, R., Cuijpers, R., van der Pol, D., Cabibihan, J.-J.: Making robots persuasive: the influence of combining persuasive strategies (gazing and gestures) by a storytelling robot on its persuasive power. In: Mutlu, B., Bartneck, C., Ham, J., Evers, V., Kanda, T. (eds.) ICSR 2011. LNCS (LNAI), vol. 7072, pp. 71–83. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-25504-5_8

    Chapter  Google Scholar 

  17. Huber, T., Schiller, D., André, E.: Enhancing explainability of deep reinforcement learning through selective layer-wise relevance propagation. In: Benzmüller, C., Stuckenschmidt, H. (eds.) KI 2019. LNCS (LNAI), vol. 11793, pp. 188–202. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30179-8_16

    Chapter  Google Scholar 

  18. Kaptein, M., Lacroix, J., Saini, P.: Individual differences in persuadability in the health promotion domain. In: Ploug, T., Hasle, P., Oinas-Kukkonen, H. (eds.) PERSUASIVE 2010. LNCS, vol. 6137, pp. 94–105. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-13226-1_11

    Chapter  Google Scholar 

  19. van Kleef, G.: Emotions as agents of social influence. In: The Oxford Handbook of Social Influence. Oxford University Press (2019)

    Google Scholar 

  20. Krapinger, G.: Aristoteles: Rhetorik. Reclam, Übersetzt und herausgegeben von Gernot Krapinger. Stuttgart (1999)

    Google Scholar 

  21. Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., Müller, K.R.: Unmasking clever Hans predictors and assessing what machines really learn. Nat. Commun. 10(1), 1096 (2019)

    Article  Google Scholar 

  22. Légal, J.B., Chappé, J., Coiffard, V., Villard-Forest, A.: Don’t you know that you want to trust me? Subliminal goal priming and persuasion. J. Exper. Soc. Psychol. 48(1), 358–360 (Jan 2012). https://doi.org/10.1016/j.jesp.2011.06.006. https://linkinghub.elsevier.com/retrieve/pii/S0022103111001673

  23. Maricchiolo, F., Gnisci, A., Bonaiuto, M., Ficca, G.: Effects of different types of hand gestures in persuasive speech on receivers’ evaluations. Lang. Cognit. Process. 24(2), 239–266 (2009)

    Article  Google Scholar 

  24. Molnar, C.: Interpretable machine learning. http://lulu.com/ (2019)

  25. Montavon, G., Samek, W., Müller, K.: Methods for interpreting and understanding deep neural networks. Dig. Sig. Process. 73, 1–15 (2018)

    Article  MathSciNet  Google Scholar 

  26. Nojavanasghari, B., Gopinath, D., Koushik, J., Baltrušaitis, T., Morency, L.P.: Deep multimodal fusion for persuasiveness prediction. In: Proceedings of the 18th ACM International Conference on Multimodal Interaction, pp. 284–288 (2016)

    Google Scholar 

  27. O’Keefe, D.J., Jackson, S.: Argument quality and persuasive effects: a review of current approaches. In: Argumentation and values: In: Proceedings of the Ninth Alta Conference on Argumentation, pp. 88–92. Speech Communication Association Annandale (1995)

    Google Scholar 

  28. Petty, R.E., Cacioppo, J.T.: The elaboration likelihood model of persuasion. In: Communication and Persuasion. Springer Series in Social Psychology, pp. 1–24. Springer, NY (1986). https://doi.org/10.1007/978-1-4612-4964-1_1

  29. Poggi, I., Vincze, L.: Gesture, gaze and persuasive strategies in political discourse. In: Kipp, M., Martin, J.-C., Paggio, P., Heylen, D. (eds.) MMCorp 2008. LNCS (LNAI), vol. 5509, pp. 73–92. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04793-0_5

    Chapter  Google Scholar 

  30. Prajod, P., Schiller, D., Huber, T., André, E.: Do deep neural networks forget facial action units?-exploring the effects of transfer learning in health related facial expression recognition. In: Shaban-Nejad, A., Michalowski, M., Bianco, S. (eds.) AI for Disease Surveillance and Pandemic Intelligence. W3PHAI 2021. Studies in Computational Intelligence, vol. 1013, pp. 217–233. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-93080-6_16

  31. Ribeiro, M.T., Singh, S., Guestrin, C.: “why should I trust you?”: Explaining the predictions of any classifier. In: Krishnapuram, B., Shah, M., Smola, A.J., Aggarwal, C.C., Shen, D., Rastogi, R. (eds.) Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August 2016, pp. 1135–1144. ACM (2016)

    Google Scholar 

  32. Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: Visual explanations from deep networks via gradient-based localization, vol. 128, pp. 336–359 (2020). https://doi.org/10.1007/s11263-019-01228-7

  33. Simon, T., Joo, H., Matthews, I., Sheikh, Y.: Hand keypoint detection in single images using multiview bootstrapping. In: Conference on Computer Vision and Pattern Recognition (2017)

    Google Scholar 

  34. Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: Visualising Image Classification Models and Saliency Maps. CoRR abs/1312.6034 (2013)

    Google Scholar 

  35. Sixt, L., Granz, M., Landgraf, T.: When explanations lie: Why modified BP attribution fails. CoRR abs/1912.09818 (2019)

    Google Scholar 

  36. Strapparava, C., Guerini, M., Stock, O.: Predicting persuasiveness in political discourses. In: LREC (2010)

    Google Scholar 

  37. Van Kleef, G.A., van den Berg, H., Heerdink, M.W.: The persuasive power of emotions: effects of emotional expressions on attitude formation and change. J. Appl. Psychol. 100(4), 1124 (2015)

    Article  Google Scholar 

  38. Wang, Y., Lucas, G., Khooshabeh, P., De Melo, C., Gratch, J.: Effects of emotional expressions on persuasion. Soc. Influ. 10(4), 236–249 (2015)

    Article  Google Scholar 

  39. Weber, K., et al.: Towards demystifying subliminal persuasiveness: using XAI-techniques to highlight persuasive markers of public speeches. In: Calvaresi, D., Najjar, A., Winikoff, M., Främling, K. (eds.) EXTRAAMAS 2020. LNCS (LNAI), vol. 12175, pp. 113–128. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-51924-7_7

    Chapter  Google Scholar 

  40. Weitz, K., Hassan, T., Schmid, U., Garbas, J.U.: Deep-learned faces of pain and emotions: Elucidating the differences of facial expressions with the help of explainable AI methods. Tm-Technisches Messen 86(7–8), 404–412 (2019)

    Google Scholar 

  41. Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_53

    Chapter  Google Scholar 

  42. Zhang, Y., Chen, X.: Explainable recommendation: a survey and new perspectives. Found. Trends Inf. Retr. 14(1), 1–101 (2020)

    Article  Google Scholar 

Download references

Acknowledgements

This work has been funded by the Deutsche Forschungsgemeinschaft (DFG) within the project “BEA - Building Engaging Argumentation”, Grant Number 455911629, as part of the Priority Program “Robust Argumentation Machines (RATIO)” (SPP-1999).

Author information

Authors and Affiliations

  1. University of Augsburg, Augsburg, Germany

    Klaus Weber, Lukas Tinnes, Tobias Huber & Elisabeth Andre

Authors
  1. Klaus Weber
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lukas Tinnes
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tobias Huber
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Elisabeth Andre
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Klaus Weber .

Editor information

Editors and Affiliations

  1. Siemens Corporation, Princeton, NJ, USA

    Helmut Degen

  2. Foundation for Research and Technology – Hellas (FORTH), Heraklion, Crete, Greece

    Stavroula Ntoa

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Weber, K., Tinnes, L., Huber, T., Andre, E. (2023). Exploring the Effect of Visual-Based Subliminal Persuasion in Public Speeches Using Explainable AI Techniques. In: Degen, H., Ntoa, S. (eds) Artificial Intelligence in HCI. HCII 2023. Lecture Notes in Computer Science(), vol 14050. Springer, Cham. https://doi.org/10.1007/978-3-031-35891-3_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-35891-3_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-35890-6

  • Online ISBN: 978-3-031-35891-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation