Evaluating emotional algorithms using psychological scales
Authors: 
Shangfei Wang, Xufa WangAuthors Info & Claims
Shangfei Wang, Xufa WangAuthors Info & ClaimsArticle No.: 2, Pages 1 - 6
Abstract
Evaluating the effectiveness of emotional algorithms is a challenge in the research of affective computing. This paper presents an evaluation method using psychological scales. Affective interfaces and artifacts generated by different emotional algorithms are regarded as the external stimuli, while users' feelings are regarded as subjective evaluations. Psychophysical methods, such as the Scheffe method of paired comparison, are adopted to measure users' feelings and obtain a psychological scale. Thus objects generated by different algorithms can be compared on this psychological scale. The higher the value, the better the algorithm is. Empirical studies on three kinds of emotional algorithms, namely emotional image retrieval, emotional speech synthesis, and emotional music generation, are used to illustrate various uses of our approach and how it may be applied to evaluating affective interaction and user-centered design.
References
[1]
R. W. Picard. Affective Computing. MIT Press, 1997.
[2]
K. Isbister and K. Höök. Evaluating affective interfaces: innovative approaches. In CHI '05: CHI '05 extended abstracts on Human factors in computing systems, pages 2119--2119, New York, NY, USA, 2005. ACM.
[3]
K. Isbister and K. Höök. Evaluating affective interactions. International Journal of Human-Computer Studies, 65(4):273--274, 2007.
[4]
N. S. Shami, J. T. Hancock, C. Peter, M. Muller, and R. Mandryk. Measuring affect in hci: going beyond the individual. In CHI '08: CHI '08 extended abstracts on Human factors in computing systems, pages 3901--3904, New York, NY, USA, 2008. ACM.
[5]
K. Höök. User-centered design and evaluation of affective interfaces. pages 127--160, 2004.
[6]
C. Wiberg. Affective computing vs. usability? insights of using traditional usability evaluation methods. April 2005.
[7]
N. Chateau and M. Mersiol. Amuse: A tool for evaluating affective interfaces. April 2005.
[8]
D. Fallman and J. Waterworth. Dealing with user experience and affective evaluation in hci design: A repertory grid approach. April 2005.
[9]
R. W. Picard and S. B. Daily. Evaluating affective interactions: Alternatives to asking what users feel. April 2005.
[10]
R. L. Mandryk and M. S. Atkins. A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies. International Journal of Human-Computer Studies, 65(4):329--347, 2007.
[11]
R. L. Hazlett and J. Benedek. Measuring emotional valence to understand the user's experience of software. International Journal of Human-Computer Studies, 65(4):306--314, 2007.
[12]
M. Isomursu, M. Tahti, S. Vainamo, and K. Kuutti. Experimental evaluation of five methods for collecting emotions in field settings with mobile applications. International Journal of Human-Computer Studies, 65(4):404--418, 2007.
[13]
P. J. Lang. Behavioral treatment and bio-behavioral assessment: computer applications. Albex, Norwood, NJ, 1980.
[14]
P. M. A. Desmet, C. J. Overbeeke, and S. J. E. T. Tax. Designing products with added emotional value: development and application of an approach for research through design. The Design Journal, 4(1):32--47, 2001.
[15]
M. Isomursu, K. Kuutti, and S. Vainamo. Experience clip: method for user participation and evaluation of mobile concepts. In Proceedings of the Participatory Design Conference, pages 83--92, 2004.
[16]
M. Tahti and L. Arhippainen. A proposal of collecting emotions and experiences. Interactive Experiences in HCI, 2:195--198, 2004.
[17]
L. A. Rantakokko and T. M. Tahti. Mobile feedback application for emotion and user experience collection. In Proceedings of PROW 2004, pages 77--81. Helsinki University Press, 2004.
[18]
M. Haringer and S. Beckhaus. Framework for the measurement of affect in interactive experiences and games. 2008.
[19]
S. F. Wang. Emotion semantics image retrieval: An brief overview. In Proceeding of 1st International Conference on Affective Computing and Intelligent Interaction, pages 490--497. Springer, 2005.
[20]
Zh. L. Yang. Experimental Psychology. Zhejiang Education Publishing Company, 2002.
[21]
A. Friberg. pdm: An expressive sequencer with real-time control of the kth music-performance rules. Comput. Music J., 30(1):37--48, 2006.
[22]
H. Scheffe. An analysis of variance for paired comparisons. Journal of American Statistical Association, 147(1):381--400, 1952.
[23]
S. F. Wang. Research on Kansei Information Processing and its Application in Image Retrieval. Doctor dissertation, Universtiy of Science and Technology of China, Hefei, Anhui, P.R. China, May 2002.
[24]
J. Cahn. Generating expression in synthesized speech. Master thesis, M.I.T., 1989.
[25]
S. L. Lv, S. F. Wang, and X. F. Wang. Emotional speech synthesis by xml file using interactive genetic algorithms. In GEC Summit, pages 907--910. ACM, 2009.
[26]
H. Zhu, S. F. Wang, and Zh. Wang. Emotional music generation using interactive genetic algorithm. In 2008 International Conference onComputer Science and Software Engineering, pages 345--348, 2008.
[27]
A. Friberg, R. Bresin, and J. Sundberg. Overview of the kth rule system for music performance. Advances in Cognitive Psychology, 2(1):145--161, 2006.
[28]
M. Wong. Emotion assessment in evaluation of affective interfaces. master thesis, University of Waterloo, 2006.
Index Terms
- Evaluating emotional algorithms using psychological scales
Recommendations
Psychological responses to simulated displays of mismatched emotional expressions
Embodied agents are often designed with the ability to simulate human emotion. This paper investigates the psychological impact of simulated emotional expressions on computer users with a particular emphasis on how mismatched facial and audio ...
Emotional Bandwidth: Information Theory Analysis of Affective Response Ratings Using a Continuous Slider
INTERACT '09: Proceedings of the 12th IFIP TC 13 International Conference on Human-Computer Interaction: Part IEmotions are an important part of the user experience in human machine interaction. More standardized methods of emotion measurement are required to assist in evaluating and comparing these experiences. This research introduces the concept of Emotional ...
Synthesising and evaluating cross-modal emotional ambiguity in virtual agents
IVA'12: Proceedings of the 12th international conference on Intelligent Virtual AgentsEmotional ambiguity, when more than one emotion appears present at a given time, or several emotions are superimposed, is common in human interaction and effects such as irony can be intentionally created through a mismatch of such emotional signals. ...
Comments
Information & Contributors
Information
Published In
November 2009
47 pages
ISBN:9781605586922
DOI:10.1145/1655260
Copyright © 2009 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 06 November 2009
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
Conference
ICMI-MLMI '09
Sponsor:
ICMI-MLMI '09: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERFACES/WORKSHOP ON MACHINE LEARNING FOR MULTIMODAL INTERFACES
November 6, 2009
Massachusetts, Boston
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 390Total Downloads
- Downloads (Last 12 months)2
- Downloads (Last 6 weeks)1
Reflects downloads up to 01 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in