research-article

Clustering approach to characterize haptic expressions of emotions

Authors:

Victoria Eyharabide,

Jean-Claude Martin,

Mehdi AmmiAuthors Info & Claims

ACM Transactions on Applied Perception (TAP), Volume 10, Issue 4

Article No.: 21, Pages 1 - 20

https://doi.org/10.1145/2536764.2536768

Published: 01 October 2013 Publication History

Abstract

Several studies have investigated the relevance of haptics to physically convey various types of emotion. However, they use basic analysis approaches to identify the relevant features for an effective communication of emotion. This article presents an advanced analysis approach, based on the clustering technique, that enables the extraction of the general features of affective haptic expressions as well as the identification of specific features in order to discriminate between close emotions that are difficult to differentiate. This approach was tested in the context of affective communication through a virtual handshake. It uses a haptic device, which enables the expression of 3D movements. The results of this research were compared to those of the standard Analysis of Variance method in order to highlight the advantages and limitations of each approach.

References

[1]

Ahn, S. J., Bailenson, J., Fox, J., and Jabon, M. 2009. Using automated facial expression analysis for emotion and behavior prediction. Paper Presented at the Annual Meeting of the NCA 95th Annual Convention. Chicago Hilton & Towers, Chicago, IL.

[2]

Bailenson, J. N., Yee, N., Brave, S., Merget, D., and Koslow, D. 2007. Virtual interpersonal touch: Expressing and recognizing emotions through haptic devices. Hum.-Comput. Interact. 22, 3, 325--353.

Digital Library

[3]

Basori, A. H., Daman, D., Sunar, M. S., and Bade, A. 2008. The potential of human haptic emotion as technique for virtual human characters movement to augment interactivity in virtual reality game. Int. J. Virtual Reality 7, 2, 27--32.

[4]

Bhatti, M., Yongjin, W., and Guan, L. 2004. A neural network approach for human emotion recognition in speech. In Proceedings of the IEEE International Symposium on Circuits and System. IEEE, Los Alamitos, CA 181--184.

[5]

Bonnet, D., Ammi, M., and Martin, J.-C. 2011. Improvement of the recognition of facial expressions with haptic feedback. In Proceedings of the IEEE Haptic Audio Visual Environments and Games International Workshop. IEEE, Los Alamitos, CA, 81--87.

[6]

Casale, S., Russo, A., and Serrano, S. 2010. Analysis of robustness of attributes selection applied to speech emotion recognition. In Proceedings of the 18th European Signal Processing Conference (EUSIPCO’10). EURASIP, Aalborg, Denmark, 1174--1178.

[7]

Castellano, G. 2008. Movement expressivity analysis in affective computers: From recognition to expression of emotion. Ph.D. thesis, University of Genoa, Italy.

[8]

Chang, J., Maclean, K., and Yohanan, S. 2010. Gesture recognition in the haptic creature. In A. M. L. Kappers, J. B. F. van Erp, W. M. B. Tiest, and F. C. T. van der Helm, Eds., EuroHaptics. Springer, 385--391.

Digital Library

[9]

Coulson, M. 2004. Attributing emotion to static body postures: Recognition accuracy, confusions, and viewpoint dependence. J. Nonverbal Behav. 28, 2, 117--139.

[10]

Courgeon, M., Clavel, C., and Martin, J.-C. 2009. Appraising emotional events during a real-time interactive game. In Proceedings of the International Workshop on Affective-Aware Virtual Agents and Social Robots. ACM, New York, 1--5.

Digital Library

[11]

Courgeon, M., Martin, J.-C., and Jacquemin, C. 2008. MARC: Un personnage virtuel réactif expressif. Noûs, 12--16.

[12]

Dael, N., Mortillaro, M., and Scherer, K. R. 2011. Emotion expression in body action and posture. Emotion 12, 5, 1085--1101.

[13]

Dellaert, F., Polzin, T., and Waibel, A. 1996. Recognizing emotion in speech. In Proceedings of the 4th International Conference on Spoken Language (ICSLP’96). IEEE, Los Alamitos, CA, 1970--1973.

[14]

Dempster, A. P., Laird, N. M., and Rubin, D. B. 1977. Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Series B Stat. Methodol. 39, 1, 1--38.

[15]

Eisenstein, J., Ghandeharizadeh, S., Huang, L., Shahabi, C., Shanbhag, G., Zimmermann, R., and Angeles, L. 2001. Analysis of clustering techniques to detect hand signs. In Proceedings of the International Symposium on Intelligent Multimedia, Video and Speech Processing (ISIMP’01). IEEE, Los Alamitos, CA, 259--262.

[16]

Ekman, P. 1992. Are there basic emotions&quest; Psychol. Rev. 99, 3, 550--553.

[17]

Ekman, P. and Friesen, W. 1975. Unmasking the Face: A Guide to Recognizing Emotions from Facial Clues. Prentice Hall, Englewood Cliffs, NJ.

[18]

Golan, O., Baron-Cohen, S., and Hill, J. 2006. The Cambridge mindreading face-voice battery: Testing complex emotion recognition in adults with and without Asperger syndrome. J. Autism Dev. Disord. 36, 2, 169--183.

[19]

Hartmann, B., Mancini, M., and Pelachaud, C. 2006. Implementing expressive gesture synthesis for embodied conversational agents. In Proceedings of the 6th International Gesture in Human-Computer Interaction and Simulation Workshop, 188--199.

Digital Library

[20]

Hertenstein, M. J., Keltner, D., App, B., Bulleit, B., and Jaskolka, A. R. 2006. Touch communicates distinct emotions. Emotion 6, 3, 528--533.

[21]

Klema, V. and Laub, A. 1980. The singular value decomposition: Its computation and some applications. IEEE Trans. Autom. Control 25, 2, 164--176.

[22]

Knight, H., Toscano, R., Stiehl, W. D., Chang, A., Wang, Y., and Breazeal, C. 2009. Real-time social touch gesture recognition for sensate robots. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, Los Alamitos, CA, 3715--3720.

Digital Library

[23]

Lemmens, P., Crompvoets, F., Brokken, D., van den Eerenbeemd, J., and de Vries, G.-J. 2009. A body-conforming tactile jacket to enrich movie viewing. In Proceedings of the 3rd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. IEEE, Los Alamitos, CA, 7--12.

Digital Library

[24]

Lu, Y., Cohen, I., Zhou, X. S., and Tian, Q. 2007. Feature selection using principal feature analysis. In Proceedings of the 15th International Conference on Multimedia (Multimedia’07). ACM, New York, 301--304.

Digital Library

[25]

Mehrabian, A. and Ferris, S. R. 1967. Inference of attitudes from nonverbal communication in two channels. Consult. Psychol. 3, 31, 248--252.

[26]

Mower, E. and Narayanan, S. S. 2011. A hierarchical static-dynamic framework for emotion classification. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’11). IEEE, Prague, 2372--2375.

[27]

Nicholson, J., Takahashi, K., and Nakatsu, R. 2000. Emotion recognition in speech using neural networks. Neural Comput. Appl. 9, 4, 290--296.

[28]

Nwe, T., Foo, S. W., and De Silva, L. 2003. Speech emotion recognition using hidden Markov models. Speech Commun. 41, 4, 603--623.

[29]

Olausson, H. K. W., Cole, J., Vallbo, A., McGlone, F., Elam, M., Krämer, H. H., Rylander, K., Wessberg, J., and Bushnell, M. C. 2008. Unmyelinated tactile afferents have opposite effects on insular and somatosensory cortical processing. Neurosci. Lett. 436, 2, 128--132.

[30]

Parkinson, B., Fischer, A. H., and Manstead, A. S. R. 2004. Emotion in Social Relations: Cultural, Group, and Interpersonal Processes. Psychology Press.

[31]

Picard, R. 1997. Affective computing. User Model. User-Adap. 12, 1, 85--89.

Digital Library

[32]

Polzehl, T., Sundaram, S., Ketabdar, H., Wagner, M., and Metze, F. 2009. Emotion classification in children’s speech using fusion of acoustic and linguistic features. In Proceedings of the 10th Annual Conference of the International Speech Communication Association (Interspeech’09). IEEE, Los Alamitos, CA, 340--343.

[33]

Russell, J. A. and Mehrabian, A. 1977. Evidence for a three-factor theory of emotions. J. Res. Pers. 11, 3, 273--294.

[34]

Scherer, K. R. 2000. Emotion. In M. Hewstone and W. Stroebe, Eds., Introduction to Social Psychology: A European Perspective (2nd ed.). Blackwell, Oxford, 151--191.

[35]

Scherer, K. R. 2005. What are emotions&quest; And how can they be measured&quest; Soc. Sci. Inform. 44, 4, 695--729.

[36]

Schuller, B., Rigoll, G., and Lang, M. 2003. Hidden Markov model-based speech emotion recognition. In Proceedings of the International Conference on Multimedia and Expo. IEEE, Los Alamitos, CA, II--1--4.

Digital Library

[37]

Smith, J. and Maclean, K. 2007. Communicating emotion through a haptic link: Design space and methodology. Int. J. Hum.-Comput. St. 65, 4, 376--387.

Digital Library

[38]

Tsetserukou, D. and Neviarouskaya, A. 2010. World’s first wearable humanoid robot that augments our emotions. In Proceedings of the 1st Augmented Human International Conference (AH’10). ACM, New York, 47--53.

Digital Library

[39]

Ververidis, D., Kotropoulos, C., and Pitas, I. 2004. Automatic emotional speech classification. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing. I--593--6.

[40]

Vogt, T. and André, E. 2005. Comparing feature sets for acted and spontaneous speech in view of automatic emotion recognition. In Proceedings of the IEEE International Conference on Multimedia and Expo (IEEE’05). IEEE, Los Alamitos, CA, 474--477.

[41]

Wallbott, H. G. 1998. Bodily expression of emotion. Eur. J. Soc. Psychol. 28, 6, 879--896.

[42]

Witten, I. H. and Frank, E. 2005. Data Mining: Practical Machine Learning Tools and Techniques (2nd ed.). Morgan Kaufmann Series in Data Management Systems. Morgan Kaufmann, San Francisco, CA.

Digital Library

[43]

Yohanan, S. and Maclean, K. E. 2011. Design and assessment of the haptic creature’s affect display. In Proceedings of the 6th International Conference on Human-Robot Interaction (HRI’11). ACM, New York, 473--480.

Digital Library

Cited By

Gaffary YMartin JAmmi M(2020)Haptic Expression and Perception of Spontaneous StressIEEE Transactions on Affective Computing10.1109/TAFFC.2018.283037111:1(138-150)Online publication date: 1-Jan-2020
https://doi.org/10.1109/TAFFC.2018.2830371
Tsalamlal YAmorim MMartin JAmmi M(2018)Modeling Emotional Valence Integration From Voice and TouchFrontiers in Psychology10.3389/fpsyg.2018.019669Online publication date: 12-Oct-2018
https://doi.org/10.3389/fpsyg.2018.01966
Turchet LZanotto DMinto SRoda AAgrawal S(2017)Emotion Rendering in Plantar Vibro-Tactile Simulations of Imagined Walking StylesIEEE Transactions on Affective Computing10.1109/TAFFC.2016.25525158:3(340-354)Online publication date: 1-Jul-2017
https://doi.org/10.1109/TAFFC.2016.2552515
Show More Cited By

Index Terms

Clustering approach to characterize haptic expressions of emotions
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Haptic devices

Recommendations

How to Collect Haptic Expressions of Spontaneous Emotions? Methodological Considerations
ACII '13: Proceedings of the 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction

Eliciting emotions to collect affective expressions is frequently used in studies about facial, spoken or bodily expressions of emotions. However, recent works in affective hap tics focus mostly on expressions of acted emotions. The expression of ...
Virtual proximity and facial expressions of computer agents regulate human emotions and attention
CASA' 2010 Special Issue

Emotion- and attention-related subjective and physiological responses to virtual proximity and facial expressions of embodied computer agents (ECA) were studied. Thirty participants viewed female and male characters with a neutral, unpleasant, or ...
Psychological responses to simulated displays of mismatched emotional expressions

Embodied agents are often designed with the ability to simulate human emotion. This paper investigates the psychological impact of simulated emotional expressions on computer users with a particular emphasis on how mismatched facial and audio ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Applied Perception

ACM Transactions on Applied Perception Volume 10, Issue 4

October 2013

190 pages

ISSN:1544-3558

EISSN:1544-3965

DOI:10.1145/2536764

Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 October 2013

Accepted: 01 May 2013

Revised: 01 February 2013

Received: 01 November 2012

Published in TAP Volume 10, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
511
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Gaffary YMartin JAmmi M(2020)Haptic Expression and Perception of Spontaneous StressIEEE Transactions on Affective Computing10.1109/TAFFC.2018.283037111:1(138-150)Online publication date: 1-Jan-2020
https://doi.org/10.1109/TAFFC.2018.2830371
Tsalamlal YAmorim MMartin JAmmi M(2018)Modeling Emotional Valence Integration From Voice and TouchFrontiers in Psychology10.3389/fpsyg.2018.019669Online publication date: 12-Oct-2018
https://doi.org/10.3389/fpsyg.2018.01966
Turchet LZanotto DMinto SRoda AAgrawal S(2017)Emotion Rendering in Plantar Vibro-Tactile Simulations of Imagined Walking StylesIEEE Transactions on Affective Computing10.1109/TAFFC.2016.25525158:3(340-354)Online publication date: 1-Jul-2017
https://doi.org/10.1109/TAFFC.2016.2552515
Cavazza M(2017)Approach and Dominance as Social Signals for Affective InterfacesSocial Signal Processing10.1017/9781316676202.021(287-303)Online publication date: 13-Jul-2017
https://doi.org/10.1017/9781316676202.021
Ammi MDemulier VCaillou SGaffary YTsalamlal YMartin JTapus AAdams JSmart WMutlu BTakayama L(2015)Haptic Human-Robot Affective Interaction in a Handshaking Social ProtocolProceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction10.1145/2696454.2696485(263-270)Online publication date: 2-Mar-2015
https://dl.acm.org/doi/10.1145/2696454.2696485
Yoren GJean-Claude MMehdi A(2013)How to Collect Haptic Expressions of Spontaneous Emotions? Methodological ConsiderationsProceedings of the 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction10.1109/ACII.2013.144(776-779)Online publication date: 2-Sep-2013
https://dl.acm.org/doi/10.1109/ACII.2013.144

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents