skip to main content
10.1145/3173574.3173598acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article
Public Access

Awe the Audience: How the Narrative Trajectories Affect Audience Perception in Public Speaking

Published: 19 April 2018 Publication History

Abstract

Telling a great story often involves a deliberate alteration of emotions. In this paper, we objectively measure and analyze the narrative trajectories of stories in public speaking and their impact on subjective ratings. We conduct the analysis using the transcripts of over 2000 TED talks and estimate potential audience response using over 5 million spontaneous annotations from the viewers. We use IBM Watson Tone Analyzer to extract sentence-wise emotion, language, and social scores. Our study indicates that it is possible to predict (with AUC as high as 0.88) the subjective ratings of the audience by analyzing the narrative trajectories. Additionally, we find that some trajectories (for example, a flat trajectory of joy) correlate well with some specific ratings (e.g. "Longwinded') assigned by the viewers. Such an association could be useful in forecasting audience responses using objective analysis.

Supplementary Material

ZIP File (pn1141-file4.zip)
suppl.mov (pn1141-file5.mp4)
Supplemental video

References

[1]
H Porter Abbott. 2008. The Cambridge introduction to narrative. Cambridge University Press.
[2]
Christopher M Bishop. 2006. Pattern recognition and Machine Learning. Vol. 128. Springer.
[3]
Mark Bubel, Ruiwen Jiang, Christine H Lee, Wen Shi, and Audrey Tse. 2016. AwareMe: Addressing Fear of Public Speech through Awareness. In CHI Conference Extended Abstracts. ACM, 68--73.
[4]
Peter Bull. 2016. Claps and Claptrap: The Analysis of Speaker-Audience Interaction in Political Speeches. Journal of Social and Political Psychology 4, 1 (2016), 473--492.
[5]
Jilin Chen, Gary Hsieh, Jalal U Mahmud, and Jeffrey Nichols. 2014. Understanding individuals' personal values from social media word use. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing. ACM, 405--414.
[6]
Lei Chen and Chong MIn Lee. 2017. Convolutional Neural Network for Humor Recognition. arXiv preprint arXiv:1702.02584 (2017).
[7]
Mathieu Chollet and Stefan Scherer. 2017. Assessing Public Speaking Ability from Thin Slices of Behavior. In Automatic Face & Gesture Recognition (FG 2017), 2017 12th IEEE International Conference on. IEEE, 310--316.
[8]
Jacob Cohen. 1977. Statistical power analysis for the behavioral sciences (revised ed.). (1977).
[9]
Keith Curtis, Gareth JF Jones, and Nick Campbell. 2015. Effects of good speaking techniques on audience engagement. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. ACM, 35--42.
[10]
Ionut Damian, Chiew Seng Sean Tan, Tobias Baur, Johannes Schöning, Kris Luyten, and Elisabeth André. 2015. Augmenting social interactions: Realtime behavioural feedback using social signal processing techniques. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 565--574.
[11]
Sophia Elizabeth De Morgan. 1882. Memoir of Augustus De Morgan. Longmans, Green, and Company.
[12]
Anda Drasovean and Caroline Tagg. 2015. Evaluative Language and Its Solidarity-Building Role on TED. com: An Appraisal and Corpus Analysis. Language@ Internet 12 (2015).
[13]
Olive Jean Dunn. 1961. Multiple comparisons among means. J. Amer. Statist. Assoc. 56, 293 (1961), 52--64.
[14]
David Easley and Jon Kleinberg. 2010. Networks, Crowds, and Markets: Reasoning about a Highly Connected World. Cambridge University Press, Chapter Power Laws and Rich-Get-Richer Phenomena, 543--560.
[15]
Paul Ekman and Wallace V Friesen. 1971. Constants across cultures in the face and emotion. Journal of personality and social psychology 17, 2 (1971), 124.
[16]
Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu, and others. 1996. A density-based algorithm for discovering clusters in large spatial databases with noise. In Kdd, Vol. 96. 226--231.
[17]
Michelle Fung, Yina Jin, RuJie Zhao, and Mohammed Ehsan Hoque. 2015. ROC speak: semi-automated personalized feedback on nonverbal behavior from recorded videos. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 1167--1178.
[18]
Carmine Gallo (Ed.). 2014. Talk Like TED. Emerald Group Publishing Limited.
[19]
Jianbo Gao, Matthew L Jockers, John Laudun, and Timothy Tangherlini. 2016. A multiscale theory for the dynamical evolution of sentiment in novels. In Behavioral, Economic and Socio-cultural Computing (BESC), 2016 International Conference on. IEEE, 1--4.
[20]
Lewis R Goldberg. 1993. The structure of phenotypic personality traits. American psychologist 48, 1 (1993), 26.
[21]
Liang Gou, Michelle X Zhou, and Huahai Yang. 2014. KnowMe and ShareMe: understanding automatically discovered personality traits from social media and user sharing preferences. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems. ACM, 955--964.
[22]
James A Hanley and Barbara J McNeil. 1982. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143, 1 (1982), 29--36.
[23]
IBM. 2017a. The science behind the service. https: //console.bluemix.net/docs/services/tone-analyzer/ science.html#the-science-behind-the-service. (December 2017).
[24]
IBM. 2017b. Tone Analyzer, Understand emotions and communication style in text. https://www.ibm.com/watson/services/tone-analyzer/. (December 2017).
[25]
Matthew L. Jockers. 2015a. The Rest of the Story. http://www.matthewjockers.net/2015/02/25/ the-rest-of-the-story/
[26]
Matthew L. Jockers. 2015b. Syuzhet: Extract Sentiment and Plot Arcs from Text. https://github.com/mjockers/syuzhet
[27]
Zhe Liu, Anbang Xu, Mengdi Zhang, Jalal Mahmud, and Vibha Sinha. 2017. Fostering User Engagement: Rhetorical Devices for Applause Generation Learnt from TED Talks. arXiv preprint arXiv:1704.02362 (2017).
[28]
Wincenty Lutosławski. 1897. The origin and growth of Plato's logic: with an account of Plato's style and of the chronology of his writings. Longmans, Green and Company.
[29]
John H McDonald. 2009. Handbook of biological statistics. Vol. 2. Sparky House Publishing Baltimore, MD.
[30]
Saif Mohammad. 2011. From once upon a time to happily ever after: Tracking emotions in novels and fairy tales. In Proceedings of the 5th ACL-HLT Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities. Association for Computational Linguistics, 105--114.
[31]
Saif M. Mohammad and Peter D. Turney. 2013. Crowdsourcing a Word-Emotion Association Lexicon. 29, 3 (2013), 436--465.
[32]
Kevin P Murphy. 2012. Machine learning: a probabilistic perspective. MIT press.
[33]
Anh-Tuan Nguyen, Wei Chen, and Matthias Rauterberg. 2012. Online feedback system for public speakers. In E-Learning, E-Management and E-Services (IS3e), 2012 IEEE Symposium on. IEEE, 1--5.
[34]
Sampo V Paunonen and Michael C Ashton. 2001. Big five factors and facets and the prediction of behavior. Journal of personality and social psychology 81, 3 (2001), 524.
[35]
F. Pedregosa and others. 2011. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research 12 (2011), 2825--2830.
[36]
Robert Plutchik. 1980. A general psychoevolutionary theory of emotion. Theories of emotion 1, 3--31 (1980), 4.
[37]
David Martin Powers. 2011. Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. (2011).
[38]
Andrew J Reagan, Lewis Mitchell, Dilan Kiley, Christopher M Danforth, and Peter Sheridan Dodds. 2016. The emotional arcs of stories are dominated by six basic shapes. EPJ Data Science 5, 1 (2016), 31.
[39]
Spyridon Samothrakis and Maria Fasli. 2015. Emotional sentence annotation helps predict fiction genre. PloS one 10, 11 (2015), e0141922.
[40]
Alex J Smola and Bernhard Schölkopf. 2004. A tutorial on support vector regression. Statistics and computing 14, 3 (2004), 199--222.
[41]
Carlo Strapparava, Alessandro Valitutti, and others. 2004. WordNet Affect: an Affective Extension of WordNet. In LREC, Vol. 4. 1083--1086.
[42]
Cassidy R Sugimoto, Mike Thelwall, Vincent Larivière, Andrew Tsou, Philippe Mongeon, and Benoit Macaluso. 2013. Scientists popularizing science: characteristics and impact of TED talk presenters. PloS one 8, 4 (2013), e62403.
[43]
M Iftekhar Tanveer, Emy Lin, and Mohammed Ehsan Hoque. 2015. Rhema: A Real-Time In-Situ Intelligent Interface to Help People with Public Speaking. In 20th International Conference on Intelligent User Interfaces. ACM, 286--295.
[44]
M. Iftekhar Tanveer, Ru Zhao, Kezhen Chen, Zoe Tiet, and Mohammed Ehsan Hoque. 2016. AutoManner: An Automated Interface for Making Public Speakers Aware of Their Mannerisms. In 21st International Conference on Intelligent User Interfaces (IUI '16). ACM, New York, NY, USA, 385--396.
[45]
Yla R Tausczik and James W Pennebaker. 2010. The psychological meaning of words: LIWC and computerized text analysis methods. Journal of Language and Social Psychology 29, 1 (2010), 24--54.
[46]
Robert Tibshirani. 1996. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological) (1996), 267--288.
[47]
Andrew Tsou, Mike Thelwall, Philippe Mongeon, and Cassidy R Sugimoto. 2014. A community of curious souls: an analysis of commenting behavior on TED talks videos. PloS one 9, 4 (2014), e93609.
[48]
Vladimir Vapnik and Alexey Chervonenkis. 1964. A note on one class of perceptrons. Automation and remote control 25, 1 (1964).
[49]
Kurt Vonnegut. 1981. Palm Sunday: An Autobiographical Collage. New York: Dell.
[50]
Yichen Wang and Aditya Pal. 2015. Detecting Emotions in Social Media: A Constrained Optimization Approach. In IJCAI. 996--1002.

Cited By

View all
  • (2023)SpeechMirror: A Multimodal Visual Analytics System for Personalized Reflection of Online Public Speaking EffectivenessIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.3326932(1-11)Online publication date: 2023
  • (2023) GestureLens : Visual Analysis of Gestures in Presentation Videos IEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.316917529:8(3685-3697)Online publication date: 1-Aug-2023
  • (2023)Novel Computational Linguistic Measures, Dialogue System and the Development of SOPHIE: Standardized Online Patient for Healthcare Interaction EducationIEEE Transactions on Affective Computing10.1109/TAFFC.2021.305471714:1(223-235)Online publication date: 1-Jan-2023
  • Show More Cited By

Index Terms

  1. Awe the Audience: How the Narrative Trajectories Affect Audience Perception in Public Speaking

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
      April 2018
      8489 pages
      ISBN:9781450356206
      DOI:10.1145/3173574
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 19 April 2018

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. affective computing
      2. clustering
      3. narrative trajectories
      4. pattern recognition
      5. styles of storytelling
      6. ted talks

      Qualifiers

      • Research-article

      Funding Sources

      • DARPA

      Conference

      CHI '18
      Sponsor:

      Acceptance Rates

      CHI '18 Paper Acceptance Rate 666 of 2,590 submissions, 26%;
      Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

      Upcoming Conference

      CHI 2025
      ACM CHI Conference on Human Factors in Computing Systems
      April 26 - May 1, 2025
      Yokohama , Japan

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)423
      • Downloads (Last 6 weeks)98
      Reflects downloads up to 20 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2023)SpeechMirror: A Multimodal Visual Analytics System for Personalized Reflection of Online Public Speaking EffectivenessIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.3326932(1-11)Online publication date: 2023
      • (2023) GestureLens : Visual Analysis of Gestures in Presentation Videos IEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.316917529:8(3685-3697)Online publication date: 1-Aug-2023
      • (2023)Novel Computational Linguistic Measures, Dialogue System and the Development of SOPHIE: Standardized Online Patient for Healthcare Interaction EducationIEEE Transactions on Affective Computing10.1109/TAFFC.2021.305471714:1(223-235)Online publication date: 1-Jan-2023
      • (2023)Tweeting from Left to Right: Tracing the Dynamics of Emotional Messaging and Public Engagement During the 2018 vs. 2022 US Campaigns2023 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom)10.1109/ISPA-BDCloud-SocialCom-SustainCom59178.2023.00159(961-971)Online publication date: 21-Dec-2023
      • (2023)NLP-based Emotions Analysis of the Marginalized Communities of Bangladesh2023 26th International Conference on Computer and Information Technology (ICCIT)10.1109/ICCIT60459.2023.10441517(1-6)Online publication date: 13-Dec-2023
      • (2022)Health Talk: Understanding Practices of Popular Professional YouTubersProceedings of the 21st International Conference on Mobile and Ubiquitous Multimedia10.1145/3568444.3568453(218-230)Online publication date: 27-Nov-2022
      • (2022)Situating Public Speaking: The Politics and Poetics of the Digital Islamic Sermons in BangladeshProceedings of the ACM on Human-Computer Interaction10.1145/35129116:CSCW1(1-28)Online publication date: 7-Apr-2022
      • (2022)Evaluating the Impact of Emotional Apology on Human-Robot Trust2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)10.1109/RO-MAN53752.2022.9900518(1655-1661)Online publication date: 29-Aug-2022
      • (2022)The Shapes of Stories10.1017/9781009270403Online publication date: 1-Jul-2022
      • (2021)Knowledge- and Data-Driven Models of Multimodal Trajectories of Public Speaking Anxiety in Real and Virtual SettingsProceedings of the 2021 International Conference on Multimodal Interaction10.1145/3462244.3479964(712-716)Online publication date: 18-Oct-2021
      • Show More Cited By

      View Options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Login options

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media