Skip to main content
SpringerLink
Log in
Book cover

Conference of Information Technologies in Biomedicine

ITiB 2016: Information Technologies in Medicine pp 27–39Cite as

Emotion Recognition Based on Physiological Sensor Data Using Codebook Approach

  • Conference paper
  • First Online:

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 472))

Abstract

This paper addresses emotion recognition based on physiological signal sequences (e.g., blood pressure, galvanic skin response and respiration) that can be obtained using state-of-the-art wearable sensors. We formulate this as a machine learning problem to distinguish sequences labelled with a certain emotion from the other sequences. In particular, we explore how to extract a feature that effectively characterises a sequence and yields accurate emotion recognition. With respect to this, existing methods rely on hand-crafted features that are manually defined based on prior knowledge about physiological signals. However, in addition to intensive labour, it is difficult to manually design features which can represent the details of a sequence. To overcome this, we propose a codebook approach where a sequence is represented with a feature describing the distribution of characteristic subsequences, called codewords. These are statistically justified because they are obtained by clustering a large number of subsequences. In addition, the details of the sequence can be maintained by considering the distribution of hundreds of codewords. Experimental results validate the effectiveness of our codebook-based emotion recognition method.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)

    Article  Google Scholar 

  2. Chang, C.C., Lin, C.J.: Libsvm: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 27:1–27:27 (2011)

    Google Scholar 

  3. Chatfield, K., Lempitsky, V., Vedaldi, A., Zisserman, A.: The devil is in the details: an evaluation of recent feature encoding methods. In: Proceedings of the 22th British Machine Vision Conference (BMVC 2011), pp. 76.1–76.12 (2011)

    Google Scholar 

  4. Chen, Y., Keogh, E., Hu, B., Begum, N., Bagnall, A., Mueen, A., Batista, G.: The UCR time series classification archive (2015). www.cs.ucr.edu/~eamonn/time_series_data/. Accessed 24 Feb 2016

  5. Garbarino, M. et al.: Empatica E3 - A wearable wireless multi-sensor device for real-time computerized biofeedback and data acquisition. In: Proceedings of the 2014 EAI 4th International Conference on Wireless Mobile Communication and Healthcare (Mobihealth 2014), pp. 39–42 (2014)

    Google Scholar 

  6. Han, J., Kamber, M., Pei, J.: Data Mining: Concepts and Techniques, 3rd edn. Morgan Kaufmann (2011)

    Google Scholar 

  7. Healey, J.A.: Wearable and Automotive Systems for Affect Recognition from Physiology. Ph.D. thesis, Massachusetts Institute of Technology (2000)

    Google Scholar 

  8. Hong, J.H., Ramos, J., Dey, A.K.: Understanding physiological responses to stressors during physical activity. In: Proceedings of the 14th ACM International Conference on Ubiquitous Computing (UbiComp 2012), pp. 270–279 (2012)

    Google Scholar 

  9. Jiang, Y.G., Yang, J., Ngo, C.W., Hauptmann, A.G.: Representations of keypoint-based semantic concept detection: a comprehensive study. IEEE Trans. Multimed. 12(1), 42–53 (2010)

    Article  Google Scholar 

  10. Kim, J., Andre, E.: Emotion recognition based on physiological changes in music listening. IEEE Tran. Pattern Anal. Mach. Intell. 30(12), 2067–2083 (2008)

    Article  Google Scholar 

  11. Koelstra, S., Mühl, C., Soleymani, M., Lee, J.S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., Patras, I.: DEAP: a database for emotion analysis using physiological signals. IEEE Trans. Affect. Comput. 3(1), 18–31 (2012)

    Article  Google Scholar 

  12. Metz, R.: This fitness wristband wants to play doctor. MIT Technol. Rev. (2014). https://www.technologyreview.com/s/524376/this-fitness-wristband-wants-to-play-doctor/. Accessed 24 Feb 2016

  13. Microsoft Corporation: Microsoft Band Official Site. https://www.microsoft.com/microsoft-band/en-us. Accessed 24 Feb 2016

  14. Pentland, A., Lazer, D., Brewer, D., Heibeck, T.: Using reality mining to improve public health and medicine. In: Bushko, R.G. (ed.) Strategy for the Future of Health, pp. 93–102. IOS press (2009)

    Google Scholar 

  15. Picard, R.W., Vyzas, E., Healey, J.: Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans. Pattern Anal. Mach. Intell. 23(10), 1175–1191 (2001)

    Article  Google Scholar 

  16. Plarre, K. et al.: Continuous inference of psychological stress from sensory measurements collected in the natural environment. In: Proceedings of the 10th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN 2011), pp. 97–108 (2011)

    Google Scholar 

  17. Rashidi, P., Mihailidis, A.: A survey on ambient-assisted living tools for older adults. IEEE J. Biomed. Health Inf. 17(3), 579–590 (2013)

    Article  Google Scholar 

  18. Shirahama, K., Grzegorzek, M.: Towards large-scale multimedia retrieval enriched by knowledge about human interpretation: retrospective survey. Multimed. Tools Appl. 75(1), 297–331 (2016)

    Article  Google Scholar 

  19. Shirahama, K., Uehara, K.: Kobe university and Muroran institute of technology at TRECVID 2012 semantic indexing task. In: Proceedings of the TREC Video Retrieval Evaluation Workshop (TRECVID 2012), pp. 239–247 (2012)

    Google Scholar 

  20. Snoek, C.G.M., Worring, M., Smeulders, A.W.M.: Early versus late fusion in semantic video analysis. In: Proceedings of the 13th Annual ACM International Conference on Multimedia (MM 2005), pp. 399–402 (2005)

    Google Scholar 

  21. Soleymani, M., Lichtenauer, J., Pun, T., Pantic, M.: A multimodal database for affect recognition and implicit tagging. IEEE Trans. Affect. Comput. 3(1), 42–55 (2012)

    Article  Google Scholar 

  22. van de Sande, K.E., Gevers, T., Snoek, C.G.: Evaluating color descriptors for object and scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1582–1596 (2010)

    Article  Google Scholar 

  23. Van Gemert, J.C., Veenman, C.J., Smeulders, A.W.M., Geusebroek, J.M.: Visual word ambiguity. IEEE Trans. Pattern Anal. Mach. Intell. 32(7), 1271–1283 (2010)

    Article  Google Scholar 

  24. Vapnik, V.N.: Statistical Learning Theory. Wiley-Interscience (1998)

    Google Scholar 

Download references

Acknowledgments

Research and development activities leading to this article have been supported by the German Federal Ministry of Education and Research within the project Cognitive Village: Adaptively Learning Technical Support System for Elderly (Grant Number: 16SV7223K).

Author information

Authors and Affiliations

  1. Pattern Recognition Group, University of Siegen, Hoelderlinstr. 3, 57076, Siegen, Germany

    Kimiaki Shirahama & Marcin Grzegorzek

Authors
  1. Kimiaki Shirahama
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marcin Grzegorzek
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kimiaki Shirahama .

Editor information

Editors and Affiliations

  1. Fac of Biomed Engg, ul. Roosevelta 40, Silesian University of Technology, Gliwice, Poland

    Ewa Piętka

  2. Fac of Biomed Engg, ul. Roosevelta 40, Silesian University of Technology, Gliwice, Poland

    Pawel Badura

  3. Faculty of Biomedical Engineering, Silesian University of Technology, Gliwice, Poland

    Jacek Kawa

  4. Faculty of Biomedical Engineering, Silesian University of Technology, Gliwice, Poland

    Wojciech Wieclawek

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Shirahama, K., Grzegorzek, M. (2016). Emotion Recognition Based on Physiological Sensor Data Using Codebook Approach. In: Piętka, E., Badura, P., Kawa, J., Wieclawek, W. (eds) Information Technologies in Medicine. ITiB 2016. Advances in Intelligent Systems and Computing, vol 472. Springer, Cham. https://doi.org/10.1007/978-3-319-39904-1_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-39904-1_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-39903-4

  • Online ISBN: 978-3-319-39904-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation