Skip to main content
SpringerLink
Log in

Emotional empathy transition patterns from human brain responses in interactive communication situations

Brain–computer and machine interactive interfacing approach

  • Open Forum
  • Published:
AI & SOCIETY Aims and scope Submit manuscript

Abstract

The paper reports our research aiming at utilization of human interactive communication modeling principles in application to a novel interaction paradigm designed for brain–computer/machine-interfacing (BCI/BMI) technologies as well as for socially aware intelligent environments or communication support systems. Automatic procedures for human affective responses or emotional states estimation are still a hot topic of contemporary research. We propose to utilize human brain and bodily physiological responses for affective/emotional as well as communicative interactivity estimation, which potentially could be used in the future for human–machine/environment interaction design. As a test platform for such an intelligent human–machine communication application, an emotional stimuli paradigm was chosen to evaluate brain responses to various affective stimuli in an emotional empathy mode. Videos with moving faces expressing various emotional displays as well as speech stimuli with similarly emotionally articulated sentences are presented to the subjects in order to further analyze different affective responses. From information processing point of view, several challenges with multimodal signal conditioning and stimuli dynamic response extraction in time frequency domain are addressed. Emotions play an important role in human daily life and human-to-human communication. This is why involvement of affective stimuli principles to human–machine communication or machine-mediated communication with utilization of multichannel neurophysiological and periphery physiological signals monitoring techniques, allowing real-time subjective brain responses evaluation, is discussed. We present our preliminary results and discuss potential applications of brain/body affective responses estimation for future interactive/smart environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

References

  • Adler R, Rodman G (2003) Undestanding human communication. Oxford University Press, Oxford

    Google Scholar 

  • Baron-Cohen S (2004) Mind reading—the interactive guide to emotions. Jessica Kingsley Publishers, London

    Google Scholar 

  • Cichocki A, Washizawa Y, Rutkowski T, Bakardjian H, Phan AH, Choi S, Lee H, Zhao Q, Zhang L, Li Y (2008) Noninvasive BCIs: multiway signal-processing array decompositions. Computer 41(10):34–42

    Article  Google Scholar 

  • Guger C, Daban S, Sellers E, Holzner C, Krausz G, Carabalona R, Gramatica F, Edlinger G (2009) How many people are able to control a P300-based brain–computer interface (BCI)?. Neurosci Lett 462(1):94–98

    Article  Google Scholar 

  • Hyvarinen A, Karhunen J, Oja E (2001) Independent component analysis. Wiley, New York

    Book  Google Scholar 

  • Lécuyer A, Lotte F, Reilly RB, Leeb R, Hirose M, Slater M (2008) Brain–computer interfaces, virtual reality, and videogames. Computer 41(10):66–72

    Article  Google Scholar 

  • McFarland DJ, Wolpaw JR (2008) Brain-computer interface operation of robotic and prosthetic devices. Computer 41(10):52–56

    Article  Google Scholar 

  • Murdoch D, Chow ED (2007) Ellipse: functions for drawing ellipses and ellipse-like confidence regions. R package version 0.3–5 (porting to R by Jesus M. Frias Celayeta)

  • Niedermeyer E, Da Silva FL (eds) (2004) Electroencephalography: basic principles, clinical applications, and related fields. 5th edn. Lippincott Williams & Wilkins, Philadelphia

  • Pfurtscheller G, Müller-Putz GR, Scherer R, Neuper C (2008) Rehabilitation with brain-computer interface systems. Computer 41(10):58–65

    Article  Google Scholar 

  • R Development Core Team (2009) R: a language and environment for statistical computing. R foundation for statistical computing, Vienna, Austria, ISBN 3-900051-07-0

  • Rutkowski T, Mandic D, Barros A (2007) A multimodal approach to communicative interactivity classification. J VLSI Signal Process 49(2):317–328

    Article  Google Scholar 

  • Rutkowski TM, Mandic D (2007) Artificial intelligence for human computing. In: Lecture notes in artificial intelligence, vol 4451, chap Modelling the communication atmosphere—a human centered multimedia approach to evaluate communicative situations. Springer, Berlin, pp 155–169

  • Rutkowski TM, Mandic D (2008) Engineering approaches to conversational informatics, chap Modeling communication atmosphere. Wiley, pp 353–369 (Wiley Series in Agent Technology)

  • Rutkowski TM, Yamakata SSY, Kakusho K, Minoh M (2003) Knowledge-based intelligent information and engineering systems. In: Lecture notes in computer science, vol 2774, chap Toward the human communication efficiency monitoring from captured audio and video media in real environments. Springer, Berlin, pp 1093–1100

  • Rutkowski TM, Cichocki A, Ralescu AL, Mandic DP (2008) Emotional states estimation from multichannel EEG maps. In: Wang R, Gu F, Shen E (eds) Advances in cognitive neurodynamics ICCN 2007, Proceedings of the international conference on cognitive neurodynamics, Neuroscience. Springer, Berlin, pp 695–698

  • Rutkowski TM, Cichocki A, Tanaka T, Mandic DP, Cao J, Ralescu AL (2009) Multichannel spectral pattern separation—an EEG processing application. In: Proceedings of the 2009 IEEE international conference on acoustics, speech, and signal processing (ICASSP2009), IEEE. pp 373–376

  • Rutkowski TM, Mandic DP, Cichocki A, Przybyszewski AW (2010) EMD approach to multichannel EEG data—the amplitude and phase components clustering analysis. J Circuits Syst Comput (JCSC) 19(1):215–229

    Article  Google Scholar 

  • Schlögl A, Brunner C (2008) Biosig: a free and open source software library for BCI research. Computer 41(10):44–50

    Article  Google Scholar 

Download references

Acknowledgments

Authors would like to thank Prof. Michihiko Minoh and Prof. Koh Kakusho of Kyoto University for their support and fruitful discussions in frame of the project “Intelligent Media Technology for Supporting Natural Communication between People”, which was partially supported by the Ministry of Education, Science, Sports and Culture in Japan, Grant-in-Aid for Creative Scientific Research, 13GS0003, where communicative interactivity approach was first developed. Also, we would like to thank for many discussions Prof. Victor V. Kryssanov of Ritsumeikan University in Kyoto at the beginning stages of presented research, which were very valuable to shape the final approach.

Author information

Author notes

  1. Tomasz M. Rutkowski

    Present address: BSI-TOYOTA Collaboration Center, RIKEN Brain Science Institute, 2-1 Hirosawa, Wako-shi, Saitama, 351-0198, Japan

Authors and Affiliations

  1. Advanced Brain Signal Processing Lab, RIKEN Brain Science Institute, 2-1 Hirosawa, Wako-shi, Saitama, 351-0198, Japan

    Tomasz M. Rutkowski & Andrzej Cichocki

  2. Department of Electrical and Electronic Engineering, Imperial College London, Exhibition Road London, London, SW7 2BT, United Kingdom

    Danilo P. Mandic

  3. Department of Intelligence Science and Technology, Graduate School of Informatics, Kyoto University, Yoshida-Honmachi, Sakyo-ku, Kyoto, 606-8501, Japan

    Toyoaki Nishida

Authors
  1. Tomasz M. Rutkowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrzej Cichocki
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Danilo P. Mandic
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Toyoaki Nishida
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tomasz M. Rutkowski.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rutkowski, T.M., Cichocki, A., Mandic, D.P. et al. Emotional empathy transition patterns from human brain responses in interactive communication situations. AI & Soc 26, 301–315 (2011). https://doi.org/10.1007/s00146-010-0310-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00146-010-0310-6

Keywords

Search

Navigation