Skip to main content

Advertisement

SpringerLink
Log in

Real-time EEG-based emotion recognition for music therapy

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

Music could change our emotions, could have an influence on our mood, and finally could affect our health. Music therapy is one of the oldest methods used for treating some diseases. Since music therapy is proved to be the helpful approach, we proposed to combine music therapy process with the real-time EEG-based human emotion recognition algorithm. By this, we could identify the user’s current emotional state, and based on such neurofeedback we could adjust the music therapy to the patient’s needs. The proposed emotion recognition algorithm could recognize in real-time six emotions such as fear, frustrated, sad, happy, pleasant, and satisfied. As the algorithm is based on an Arousal-Valence emotion model, it has a potential to recognize all emotions that could be defined by the 2-dimensional model. The experiments on emotion induction with sound stimuli from International Affective Digitized Sounds (IADS) database and with music stimuli and implemented questionnaire were proposed and realized. In this paper, we proposed a general EEG-enabled music therapy algorithm. It allows us to adapt the therapy to the predefined time of the treatment and adjust the music therapy session to the current emotional state of the user in a way as an experienced music therapist works.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Accardo A, Affinito M, Carrozzi M, Bouquet F (1997) Use of the fractal dimension for the analysis of electroencephalographic time series. Biol Cybern 77(5):339–350

    Article  MATH  Google Scholar 

  2. American electroencephalographic society guidelines for standard electrode position nomenclature (1991) J Clin Neurophysiol 8(2):200–202

    Google Scholar 

  3. American music therapy association. Music therapy and medicine. http://www.musictherapy.org/

  4. Block A, Von Bloh W, Schellnhuber HJ (1990) Efficient box-counting determination of generalized fractal dimensions. Phys Rev A 42(4):1869–1874

    Article  MathSciNet  Google Scholar 

  5. Bolls P, Lang A, Potter R (2001) The effects of message valence and listener arousal on attention, memory, and facial muscular responses to radio advertisements. Commun Res 28(5):627–651

    Article  Google Scholar 

  6. Bos DO (2006) EEG-based emotion recognition [online]. http://hmi.ewi.utwente.nl/verslagen/capita-selecta/CS-Oude_Bos-Danny.pdf

  7. Bradley MM (1994) Measuring emotion: the self-assessment manikin and the semantic differential. J Behav Ther Exp Psychiatry 25(1):49–59

    Article  Google Scholar 

  8. Bradley MM, Lang PJ (2007) The international affective digitized sounds (2nd edn.; IADS-2): affective ratings of sounds and instruction manual. Tech. rep., University of Florida, Gainesville

  9. Chanel G (2009) Emotion assessment for affective-computing based on brain and peripheral signals. Ph.D. thesis, University of Geneva, Geneva

  10. Chang MY, Chen CH, Huang KF (2008) Effects of music therapy on psychological health of women during pregnancy. J Clin Nurs 17(19):2580–2587

    Article  Google Scholar 

  11. Emotiv. http://www.emotiv.com

  12. Evans D (2002) The effectiveness of music as an intervention for hospital patients: a systematic review. J Adv Nurs 37(1):8–18

    Article  Google Scholar 

  13. Guetin S, Portet F, Picot MC, Defez C, Pose C, Blayac JP, Touchon J (2009) Impact of music therapy on anxiety and depression for patients with Alzheimer’s disease and on the burden felt by the main caregiver (feasibility study). Interets Musicother Anxiete, Depress Patients Atteints Mal Alzheimer Charg Ressentie Accompagnant Princ 35(1):57–65

    Google Scholar 

  14. Hamann S, Canli T (2004) Individual differences in emotion processing. Curr Opin Neurobiol 14(2):233–238

    Article  Google Scholar 

  15. Hamel W (2001) The effects of music intervention on anxiety in the patient waiting for cardiac catheterization. Intensive Crit Care Nurs 17(5):279–285

    Article  Google Scholar 

  16. Helmes E, Wiancko D (2006) Effects of music in reducing disruptive behavior in a general hospital. J Am Psychiatr Nurs Assoc 12(1):37–44

    Article  Google Scholar 

  17. Higuchi T (1988) Approach to an irregular time series on the basis of the fractal theory. Phys D, Nonlinear Phenom 31(2):277–283

    Article  MathSciNet  MATH  Google Scholar 

  18. IDM-Project (2008) Emotion-based personalized digital media experience in co-spaces http://www3.ntu.edu.sg/home/eosourina/CHCILab/projects.html

  19. Lin YP, Wang CH, Jung TP, Wu TL, Jeng SK, Duann JR, Chen JH (2010) EEG-based emotion recognition in music listening. IEEE Trans Biomed Eng 57(7):1798–1806

    Article  Google Scholar 

  20. Liu Y, Sourina O, Nguyen MK (2010) Real-time EEG-based human emotion recognition and visualization. In: 2010 international conference on Cyberworlds (CW), pp 262–269

    Chapter  Google Scholar 

  21. Liu Y, Sourina O, Nguyen MK (2011) Real-time EEG-based emotion recognition and its applications. In: Transactions on computational science XII. Lecture notes in computer science, vol 6670. Springer, Berlin, pp 256–277

    Chapter  Google Scholar 

  22. Maragos P, Sun FK (1993) Measuring the fractal dimension of signals: morphological covers and iterative optimization. IEEE Trans Signal Process 41(1):108–121

    Article  MATH  Google Scholar 

  23. Mauss IB, Robinson MD (2009) Measures of emotion: a review. Cogn Emot 23(2):209–237

    Article  Google Scholar 

  24. Mehrabian A (1995) Framework for a comprehensive description and measurement of emotional states. Genet Soc Gen Psychol Monogr 121(3):339–361

    Google Scholar 

  25. PET 2-channel bipolar. http://www.brainquiry.com/NeurofeedbackHardware.html

  26. Petrantonakis PC, Hadjileontiadis LJ (2010) Emotion recognition from EEG using higher order crossings. IEEE Trans Inf Technol Biomed 14(2):186–197

    Article  Google Scholar 

  27. Plutchik R (2003) Emotions and life: perspectives from psychology, biology, and evolution, 1st edn. American Psychological Association, Washington

    Google Scholar 

  28. Roth E, Wisser S (2004) Music therapy: the rhythm of recovery. Case Manag 15(3):52–56

    Article  Google Scholar 

  29. Russell JA (1979) Affective space is bipolar. J Pers Soc Psychol 37(3):345–356

    Article  Google Scholar 

  30. Russell JA (1980) A circumplex model of affect. J Pers Soc Psychol 39(6):1161–1178

    Article  Google Scholar 

  31. Sammler D, Grigutsch M, Fritz T, Koelsch S (2007) Music and emotion: electrophysiological correlates of the processing of pleasant and unpleasant music. Psychophysiology 44(2):293–304

    Article  Google Scholar 

  32. Sanei S, Chambers J (2007) EEG signal processing. Wiley, Chichester

    Google Scholar 

  33. Schaaff K (2008) EEG-based emotion recognition. Ph.D. thesis, Universitat Karlsruhe (TH)

  34. Schaaff K, Schultz T (2009) Towards emotion recognition from electroencephalographic signals. In: 3rd international conference on affective computing and intelligent interaction and workshops, 2009. ACII 2009, pp 1–6. doi:10.1109/ACII.2009.5349316

    Chapter  Google Scholar 

  35. Sourina O, Liu Y (2011) A fractal-based algorithm of emotion recognition from EEG using arousal-valence model. In: BIOSIGNALS 2011—proceedings of the international conference on bio-inspired systems and signal processing, pp 209–214

    Google Scholar 

  36. Sourina O, Kulish VV, Sourin A (2008) Novel tools for quantification of brain responses to music stimuli. In: Proc of 13th international conference on biomedical engineering ICBME 2008, pp 411–414

    Google Scholar 

  37. Sourina O, Sourin A, Kulish V (2009) EEG data driven animation and its application. In: Computer vision/computer graphics collaboration techniques. Lecture notes in computer science, vol 5496. Springer, Berlin, pp 380–388

    Chapter  Google Scholar 

  38. Sourina O, Liu Y, Wang Q, Nguyen MK (2011) EEG-based personalized digital experience. In: Universal access in human-computer interaction. Users diversity. Lecture notes in computer science, vol 6766. Springer, Berlin, pp 591–599

    Chapter  Google Scholar 

  39. Sourina O, Wang Q, Liu Y, Nguyen MK (2011) A real-time fractal-based brain state recognition from EEG and its applications. In: BIOSIGNALS 2011—proceedings of the international conference on bio-inspired systems and signal processing, pp 82–90

    Google Scholar 

  40. Sourina O, Wang Q, Nguyen MK (2011) EEG-based “serious” games and monitoring tools for pain management. In: Studies in health technology and informatics, vol 163, pp 606–610

    Google Scholar 

  41. Stevens K (1990) Patients’ perceptions of music during surgery. J Adv Nurs 15(9):1045–1051

    Article  Google Scholar 

  42. Takahashi K (2004) Remarks on emotion recognition from multi-modal bio-potential signals. In: 2004 IEEE international conference on industrial technology, 2004. IEEE ICIT’04, vol 3, pp 1138–1143

    Chapter  Google Scholar 

  43. Tyson F (1981) Psychiatric music therapy: origins and development. Creative arts rehabilitation center

  44. Wang Q, Sourina O, Nguyen MK (2010) EEG-based “serious” games design for medical applications. In: 2010 international conference on cyberworlds (CW), pp 270–276

    Chapter  Google Scholar 

  45. Wang Q, Sourina O, Nguyen MK (2011) Fractal dimension based neurofeedback in serious games. Vis Comput 27:299–309

    Article  Google Scholar 

  46. Winter M, Paskin S, Baker T (1994) Music reduces stress and anxiety of patients in the surgical holding area. J Post Anesth Nurs 9(6):340–343

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Nanyang Technological University, Singapore, Singapore

    Olga Sourina, Yisi Liu & Minh Khoa Nguyen

Authors
  1. Olga Sourina
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yisi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Minh Khoa Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yisi Liu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sourina, O., Liu, Y. & Nguyen, M.K. Real-time EEG-based emotion recognition for music therapy. J Multimodal User Interfaces 5, 27–35 (2012). https://doi.org/10.1007/s12193-011-0080-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-011-0080-6

Keywords

Search

Navigation