Mikke Tavast
ABSTRACT
Computational interaction and user modeling is presently limited in the domain of emotions. We investigate a potential new approach to computational modeling of emotional response behavior, by using modern neural language models to generate synthetic self-report data, and evaluating the human-likeness of the results. More specifically, we generate responses to the PANAS questionnaire with four different variants of the recent GPT-3 model. Based on both data visualizations and multiple quantitative metrics, the human-likeness of the responses increases with model size, with the largest Davinci model variant generating the most human-like data.
- Farid Anvari and Daniel Lakens. 2021. Anchor method to determine SESOI. osf.io/89pcfGoogle Scholar
- Farid Anvari and Daniël Lakens. 2021. Using anchor-based methods to determine the smallest effect size of interest. Journal of Experimental Social Psychology 96 (2021), 104159. https://doi.org/10.1016/j.jesp.2021.104159Google ScholarCross Ref
- Jan Ketil Arnulf, Kai Rune Larsen, Øyvind Lund Martinsen, and Chih How Bong. 2014. Predicting Survey Responses: How and Why Semantics Shape Survey Statistics on Organizational Behaviour. PLOS ONE 9, 9 (09 2014), 1–13. https://doi.org/10.1371/journal.pone.0106361Google ScholarCross Ref
- Denny Borsboom, Gideon J Mellenbergh, and Jaap Van Heerden. 2004. The concept of validity.Psychological review 111, 4 (2004), 1061–1071. https://doi.org/10.1037/0033-295X.111.4.1061Google ScholarCross Ref
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (Eds.). Vol. 33. Curran Associates, Inc., 1877–1901. https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdfGoogle Scholar
- Charlotte Caucheteux, Alexandre Gramfort, and Jean-Rémi King. 2021. GPT-2’s activations predict the degree of semantic comprehension in the human brain. bioRxiv (2021). https://doi.org/10.1101/2021.04.20.440622Google ScholarCross Ref
- Noshaba Cheema, Laura A Frey-Law, Kourosh Naderi, Jaakko Lehtinen, Philipp Slusallek, and Perttu Hämäläinen. 2020. Predicting mid-air interaction movements and fatigue using deep reinforcement learning. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13. https://doi.org/10.1145/3313831.3376701Google ScholarDigital Library
- John R. Crawford and Julie D. Henry. 2004. The Positive and Negative Affect Schedule (PANAS): Construct validity, measurement properties and normative data in a large non-clinical sample. British Journal of Clinical Psychology 43, 3 (2004), 245–265. https://doi.org/10.1348/0144665031752934Google ScholarCross Ref
- Peter Sheridan Dodds, Eric M. Clark, Suma Desu, Morgan R. Frank, Andrew J. Reagan, Jake Ryland Williams, Lewis Mitchell, Kameron Decker Harris, Isabel M. Kloumann, James P. Bagrow, Karine Megerdoomian, Matthew T. McMahon, Brian F. Tivnan, and Christopher M. Danforth. 2015. Human language reveals a universal positivity bias. Proceedings of the National Academy of Sciences 112, 8 (2015), 2389–2394. https://doi.org/10.1073/pnas.1411678112Google ScholarCross Ref
- Ariel Goldstein, Zaid Zada, Eliav Buchnik, Mariano Schain, Amy Price, Bobbi Aubrey, Samuel A. Nastase, Amir Feder, Dotan Emanuel, Alon Cohen, Aren Jansen, Harshvardhan Gazula, Gina Choe, Aditi Rao, Catherine Kim, Colton Casto, Lora Fanda, Werner Doyle, Daniel Friedman, Patricia Dugan, Roi Reichart, Sasha Devore, Adeen Flinker, Liat Hasenfratz, Avinatan Hassidim, Michael Brenner, Yossi Matias, Kenneth A. Norman, Orrin Devinsky, and Uri Hasson. 2021. Thinking ahead: spontaneous prediction in context as a keystone of language in humans and machines. bioRxiv (March 2021), 2020.12.02.403477. https://doi.org/10.1101/2020.12.02.403477Google ScholarCross Ref
- Friedrich M Götz, Rakoen Maertens, and Sander van der Linden. 2021. Let the Algorithm Speak: How to Use Neural Networks for Automatic Item Generation in Psychological Scale Development. https://doi.org/10.31234/osf.io/m6s28Google ScholarCross Ref
- Jussi Jokinen, Aditya Acharya, Mohammad Uzair, Xinhui Jiang, and Antti Oulasvirta. 2021. Touchscreen Typing As Optimal Supervisory Control. In CHI ’21: CHI Conference on Human Factors in Computing Systems, Virtual Event / Yokohama, Japan, May 8-13, 2021, Yoshifumi Kitamura, Aaron Quigley, Katherine Isbister, Takeo Igarashi, Pernille Bjørn, and Steven Mark Drucker (Eds.). ACM, 720:1–720:14. https://doi.org/10.1145/3411764.3445483Google ScholarDigital Library
- Antonio Laverghetta Jr, Animesh Nighojkar, Jamshidbek Mirzakhalov, and John Licato. 2021. Can Transformer Language Models Predict Psychometric Properties?arxiv:2106.06849 [cs.CL]Google Scholar
- Rakoen Maertens, Friedrich M Götz, Claudia R Schneider, Jon Roozenbeek, John R Kerr, Stefan Stieger, III McClanahan, William P, Karly Drabot, and Sander van der Linden. 2021. The Misinformation Susceptibility Test (MIST): A psychometrically validated measure of news veracity discernment. https://doi.org/10.31234/osf.io/gk68hGoogle ScholarCross Ref
- Antti Oulasvirta. 2019. It’s time to rediscover HCI models. Interactions 26, 4 (2019), 52–56.Google ScholarDigital Library
- Timo Partala and Aleksi Kallinen. 2012. Understanding the most satisfying and unsatisfying user experiences: Emotions, psychological needs, and context. Interacting with Computers 24, 1 (2012), 25–34. https://doi.org/10.1016/j.intcom.2011.10.001Google ScholarDigital Library
- Gilles Raiche. 2010. an R package for parallel analysis and non graphical solutions to the Cattell scree test. https://CRAN.R-project.org/package=nFactors R package version 2.3.3.1..Google Scholar
- Martin Schrimpf, Idan Asher Blank, Greta Tuckute, Carina Kauf, Eghbal A. Hosseini, Nancy Kanwisher, Joshua B. Tenenbaum, and Evelina Fedorenko. 2021. The neural architecture of language: Integrative modeling converges on predictive processing. Proceedings of the National Academy of Sciences 118, 45(2021). https://doi.org/10.1073/pnas.2105646118Google ScholarCross Ref
- Alexandre N. Tuch, Rune Trusell, and Kasper Hornbæk. 2013. Analyzing Users’ Narratives to Understand Experience with Interactive Products. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 2079–2088. https://doi.org/10.1145/2470654.2481285Google ScholarDigital Library
- Austin van Loon and Jeremy Freese. 2019. Word embeddings reveal how fundamental sentiments structure natural language. https://doi.org/10.31235/osf.io/r7ewxGoogle ScholarCross Ref
- David Watson, Lee Anna Clark, and Auke Tellegen. 1988. Development and validation of brief measures of positive and negative affect: the PANAS scales.Journal of personality and social psychology 54, 6(1988), 1063–1070.Google Scholar
Index Terms
- Language Models Can Generate Human-Like Self-Reports of Emotion
Index terms have been assigned to the content through auto-classification.
Recommendations
Are emoji valid indicators of in-the-moment mood?
AbstractDespite widespread assumptions that emoji represent emotions, research findings show that we do not process emoji in a way which we would expect for emotional stimuli. As such, we might be better placed to consider them more in line with mood ...
Highlights- The Emoji PANAS scale can reduce linguistic confounds when reporting current affective state
- Emoji are unlikely to be a valid means for reporting mood for those high in emotionality
- Emoji may be better consider mood indicators ...
The language of emotion in short blog texts
CSCW '08: Proceedings of the 2008 ACM conference on Computer supported cooperative workEmotion is central to human interactions, and automatic detection could enhance our experience with technologies. We investigate the linguistic expression of fine-grained emotion in 50 and 200 word samples of real blog texts previously coded by expert ...
The communicative role of non-face emojis
Emojis have evolved from imitations of facial expressions meant to communicate affect into pictures of objects, food, and places that are not directly linked to affect. While emojis that resemble facial expressions are well-researched, emojis that ...
Comments