Relational Affect in Dyadic Interactions
Article No.: 526, Pages 1 - 9
Abstract
Relational affect is the affective response (encompassing emotion, expression, feeling) that emerges from an interaction between two people. The case study presented here introduces the concept of relational affect through a human perceptual rating task. Forty-five raters watched short video clips of two people interacting and described their perceived emotion of the individuals and that of the overall interaction. Our qualitative analysis of the rater responses showed that raters used a variety of schemes to reason about emotion, including expressions, context, and perceived appraisal of the event. These reasoning schemes were notably different for perceived individual emotion and relational affect. Our findings show that the vocabulary use for relational affect is distinct from that of individual emotion and relational affect as a phenomenon deepens our understanding of social interactions and moves the field a step closer to realizing the goal of fluid interactions between people and technology.
Supplemental Material
- Download
- 19.78 MB
- Transcript
References
[1]
Saeed Abdullah, Elizabeth L Murnane, Jean MR Costa, and Tanzeem Choudhury. 2015. Collective smile: Measuring societal happiness from geolocated images. In Proceedings of the 18th ACM conference on computer supported cooperative work & social computing. 361–374.
[2]
Areej Alhothali and Jesse Hoey. 2015. Good news or bad news: Using affect control theory to analyze readers’ reaction towards news articles. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 1548–1558.
[3]
Nalini Ambady and Heather M Gray. 2002. On being sad and mistaken: mood effects on the accuracy of thin-slice judgments.Journal of personality and social psychology 83, 4 (2002), 947.
[4]
Roghayeh Barmaki and Charles E Hughes. 2015. Providing real-time feedback for student teachers in a virtual rehearsal environment. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. 531–537.
[5]
Lisa Feldman Barrett, Ralph Adolphs, Stacy Marsella, Aleix M Martinez, and Seth D Pollak. 2019. Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements. Psychological science in the public interest 20, 1 (2019), 1–68.
[6]
Penelope Brown and Stephen C Levinson. 1987. Politeness: Some universals in language usage. Vol. 4. Cambridge university press.
[7]
Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette N Chang, Sungbok Lee, and Shrikanth S Narayanan. 2008. IEMOCAP: Interactive emotional dyadic motion capture database. Language resources and evaluation 42 (2008), 335–359.
[8]
Alan S Cowen and Dacher Keltner. 2020. What the face displays: Mapping 28 emotions conveyed by naturalistic expression.American Psychologist 75, 3 (2020), 349.
[9]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).
[10]
Joseph P Forgas. 1981. The perception of social episodes: Categorical and dimensional representations in two different social milieus. In The psychology of social situations. Elsevier, 80–94.
[11]
David Garcia, Fernando Mendez, Uwe Serdült, and Frank Schweitzer. 2012. Political polarization and popularity in online participatory media: an integrated approach. In Proceedings of the first edition workshop on Politics, elections and data. 3–10.
[12]
Saccá Gary and Moreno-Llorena Jaime. 2016. Towards a strategy for recognition of collective emotions on social networking sites. In Proceedings of the XVII International Conference on Human Computer Interaction. 1–8.
[13]
Amit Goldenberg, David Garcia, Eran Halperin, and James J Gross. 2020. Collective emotions. Current Directions in Psychological Science 29, 2 (2020), 154–160.
[14]
Harry F Gollob and Betty B Rossman. 1973. Judgments of an actor’s “power and ability to influence others”. Journal of Experimental Social Psychology 9, 5 (1973), 391–406.
[15]
Jorge Goncalves, Pratyush Pandab, Denzil Ferreira, Mohammad Ghahramani, Guoying Zhao, and Vassilis Kostakos. 2014. Projective testing of diurnal collective emotion. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 487–497.
[16]
Jonathan Gratch and Stacy Marsella. 2005. Evaluating a computational model of emotion. Autonomous Agents and Multi-Agent Systems 11 (2005), 23–43.
[17]
Nick Haslam and Alan Page Fiske. 1999. Relational models theory: A confirmatory factor analysis. Personal relationships 6, 2 (1999), 241–250.
[18]
Changyang He, Lu He, Tun Lu, and Bo Li. 2021. Beyond Entertainment: Unpacking Danmaku and Comments’ Role of Information Sharing and Sentiment Expression in Online Crisis Videos. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–27.
[19]
David R Heise. 1977. Social action as the control of affect. Behavioral Science 22, 3 (1977), 163–177.
[20]
David R Heise. 2007. Expressive order: Confirming sentiments in social actions. Springer Science & Business Media.
[21]
Tianran Hu, Anbang Xu, Zhe Liu, Quanzeng You, Yufan Guo, Vibha Sinha, Jiebo Luo, and Rama Akkiraju. 2018. Touch your heart: A tone-aware chatbot for customer care on social media. In Proceedings of the 2018 CHI conference on human factors in computing systems. 1–12.
[22]
Harold H Kelley, John G Holmes, Norbert L Kerr, Harry T Reis, Caryl E Rusbult, and Paul AM Van Lange. 2003. An atlas of interpersonal situations. (2003).
[23]
Dacher Keltner, Disa Sauter, Jessica Tracy, and Alan Cowen. 2019. Emotional expression: Advances in basic emotion theory. Journal of nonverbal behavior 43 (2019), 133–160.
[24]
Donald J Kiesler. 1983. The 1982 interpersonal circle: A taxonomy for complementarity in human transactions.Psychological review 90, 3 (1983), 185.
[25]
Ronak Kosti, Jose M Alvarez, Adria Recasens, and Agata Lapedriza. 2019. Context based emotion recognition using emotic dataset. IEEE transactions on pattern analysis and machine intelligence 42, 11 (2019), 2755–2766.
[26]
Adam DI Kramer, Jamie E Guillory, and Jeffrey T Hancock. 2014. Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National academy of Sciences of the United States of America 111, 24 (2014), 8788.
[27]
Bryan Li, Dimitrios Dimitriadis, and Andreas Stolcke. 2019. Acoustic and lexical sentiment analysis for customer service calls. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 5876–5880.
[28]
Bo Li, Duoyong Sun, Zihan Lin, and Chaomin Ou. 2014. Agent-based simulation research on group emotion evolution of public emergency. In 2014 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2014). IEEE, 497–502.
[29]
Larry S Liebovitch, Paul R Peluso, Michael D Norman, Jessica Su, and John M Gottman. 2011. Mathematical model of the dynamics of psychotherapy. Cognitive Neurodynamics 5 (2011), 265–275.
[30]
Mufan Luo, Tiffany W Hsu, Joon Sung Park, and Jeffrey T Hancock. 2020. Emotional amplification during live-streaming: Evidence from comments during and after news events. Proceedings of the ACM on human-computer interaction 4, CSCW1 (2020), 1–19.
[31]
Patricia Yancey Martin and Barry A Turner. 1986. Grounded theory and organizational research. The journal of applied behavioral science 22, 2 (1986), 141–157.
[32]
Oludamilare Matthews, Zhanna Sarsenbayeva, Weiwei Jiang, Joshua Newn, Eduardo Velloso, Sarah Clinch, and Jorge Goncalves. 2018. Inferring the mood of a community from their walking speed: a preliminary study. In Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers. 1144–1149.
[33]
Charles Egerton Osgood, William H May, and Murray S Miron. 1975. Cross-cultural universals of affective meaning. Vol. 1. University of Illinois Press.
[34]
Rosalind W Picard. 1999. Affective computing for hci. In HCI (1). Citeseer, 829–833.
[35]
Aaron L Pincus, Pamela Sadler, Erik Woody, Michael J Roche, Katherine M Thomas, and Aidan GC Wright. 2014. Multimethod assessment of interpersonal dynamics. (2014).
[36]
Soujanya Poria, Navonil Majumder, Rada Mihalcea, and Eduard Hovy. 2019. Emotion recognition in conversation: Research challenges, datasets, and recent advances. IEEE Access 7 (2019), 100943–100953.
[37]
William Treval Powers and William T Powers. 1973. Behavior: The control of perception. (1973).
[38]
Zachary H Pugh, Sanghyun Choo, Joseph C Leshin, Kristen A Lindquist, and Chang S Nam. 2022. Emotion depends on context, culture and their interaction: evidence from effective connectivity. Social cognitive and affective neuroscience 17, 2 (2022), 206–217.
[39]
Stefan Rank. 2010. Docking agent-based simulation of collective emotion to equation-based models and interactive agents. In Proceedings of the 2010 Spring Simulation Multiconference. 1–8.
[40]
John F Rauthmann, David Gallardo-Pujol, Esther M Guillaume, Elysia Todd, Christopher S Nave, Ryne A Sherman, Matthias Ziegler, Ashley Bell Jones, and David C Funder. 2014. The Situational Eight DIAMONDS: a taxonomy of major dimensions of situation characteristics.Journal of Personality and Social Psychology 107, 4 (2014), 677.
[41]
Björn W Schuller. 2018. Speech emotion recognition: Two decades in a nutshell, benchmarks, and ongoing trends. Commun. ACM 61, 5 (2018), 90–99.
[42]
Joseph Seering, Juan Pablo Flores, Saiph Savage, and Jessica Hammer. 2018. The social roles of bots: evaluating impact of bots on discussions in online communities. Proceedings of the ACM on Human-Computer Interaction 2, CSCW (2018), 1–29.
[43]
Taylan Sen, Mohammad Rafayet Ali, Mohammed Ehsan Hoque, Ronald Epstein, and Paul Duberstein. 2017. Modeling doctor-patient communication with affective text analysis. In 2017 seventh international conference on affective computing and intelligent interaction (ACII). IEEE, 170–177.
[44]
Jan Slaby. 2019. Relational affect: perspectives from philosophy and cultural studies. In How to do things with affects. Brill, 59–81.
[45]
Eliot R Smith and Diane M Mackie. 2015. Dynamics of group-based emotions: Insights from intergroup emotions theory. Emotion Review 7, 4 (2015), 349–354.
[46]
Roger Andre Søraa, Gunhild Tøndel, Mark W Kharas, and J Artur Serrano. 2023. What do Older Adults Want from Social Robots? A Qualitative Research Approach to Human-Robot Interaction (HRI) Studies. International Journal of Social Robotics 15, 3 (2023), 411–424.
[47]
Jared B Torre and Matthew D Lieberman. 2018. Putting feelings into words: Affect labeling as implicit emotion regulation. Emotion Review 10, 2 (2018), 116–124.
[48]
Wolfgang Tschacher, Georg M Rees, and Fabian Ramseyer. 2014. Nonverbal synchrony and affect in dyadic interactions. Frontiers in psychology 5 (2014), 1323.
[49]
Dina Utami and Timothy Bickmore. 2019. Collaborative user responses in multiparty interaction with a couples counselor robot. In 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 294–303.
[50]
Erik van Haeringen and Charlotte Gerritsen. 2023. Simulating Panic Amplification in Crowds via Density-Emotion Interaction. In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems. 1895–1902.
[51]
ES Van Haeringen, Charlotte Gerritsen, and KV Hindriks. 2023. Emotion contagion in agent-based simulations of crowds: a systematic review. Autonomous Agents and Multi-Agent Systems 37, 1 (2023), 6.
[52]
Christian Von Scheve. 2018. A social relational account of affect. European Journal of Social Theory 21, 1 (2018), 39–59.
[53]
Minzhong Wu, Lei Wang, and Guodong Li. 2022. Crowd Emotion Recognition Based on Causal Spatiotemporal Structure. In Proceedings of the 8th International Conference on Computing and Artificial Intelligence. 368–374.
[54]
Junxiao Xue, Mingchuang Zhang, and Hui Yin. 2023. A Personality-based Model of Emotional Contagion and Control in Crowd Queuing Simulations. ACM Transactions on Modeling and Computer Simulation 33, 1-2 (2023), 1–23.
[55]
Anna Xygkou, Panote Siriaraya, Alexandra Covaci, Holly Gwen Prigerson, Robert Neimeyer, Chee Siang Ang, and Wan-Jou She. 2023. The" Conversation" about Loss: Understanding How Chatbot Technology was Used in Supporting People in Grief. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–15.
[56]
Galit B Yom-Tov, Shelly Ashtar, Daniel Altman, Michael Natapov, Neta Barkay, Monika Westphal, and Anat Rafaeli. 2018. Customer sentiment in web-based service interactions: Automated analyses and new insights. In Companion Proceedings of the The Web Conference 2018. 1689–1697.
Index Terms
- Relational Affect in Dyadic Interactions
Index terms have been assigned to the content through auto-classification.
Recommendations
Tangible Affect: A Literature Review of Tangible Interactive Systems Addressing Human Core Affect, Emotions and Moods
DIS '24: Proceedings of the 2024 ACM Designing Interactive Systems ConferenceTangible user interfaces (TUIs) have been applied to assess, communicate, or regulate human core affect, emotions, or moods. Previous studies identified TUIs as an innovative way to serve people’s affective needs. This review examines the design and ...
Multimodal Dimensional and Continuous Emotion Recognition in Dyadic Video Interactions
Advances in Multimedia Information Processing – PCM 2018AbstractAutomatic emotion recognition is a challenging task which can make great impact on improving natural human computer interactions. In dyadic human-human interactions, a more complex interaction scenario, a person’s emotion state will be influenced ...
A cognitively based approach to affect sensing from text
IUI '06: Proceedings of the 11th international conference on Intelligent user interfacesStudying the relationship between natural language and affective information as well as assessing the underpinned affective qualities of natural language are becoming crucial for improving human computer interaction. Different approaches have already ...
Comments
Information & Contributors
Information
Published In
May 2024
4761 pages
ISBN:9798400703317
DOI:10.1145/3613905
Copyright © 2024 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 11 May 2024
Check for updates
Author Tags
Qualifiers
- Extended-abstract
- Research
- Refereed limited
Conference
Acceptance Rates
Overall Acceptance Rate 6,164 of 23,696 submissions, 26%
Upcoming Conference
CHI 2025
- Sponsor:
- sigchi
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 214Total Downloads
- Downloads (Last 12 months)214
- Downloads (Last 6 weeks)51
Reflects downloads up to 27 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderFull Text
View this article in Full Text.
Full TextHTML Format
View this article in HTML Format.
HTML Format