Skip to main content
SpringerLink
Log in

HiMotion: a new research resource for the study of behavior, cognition, and emotion

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The HiMotion research project was designed to create a multimodal database and several support tools for the study of human behavior, cognition and emotion, in the context of computer-based tasks designed to elicit cognitive load and specialized affective responses. The database includes both human-computer interaction (HCI) and psychophysiological data, collected through an experimental setup that we devised for synchronized recording of keyboard, mouse, and central/ peripheral nervous system measurements. Currently we provide a battery of five different cognitive tasks, and a video bank for affective elicitation, together with a set of introductory and self-reporting screens. We have conducted two experiments, one involving a population of 27 subjects, which followed the cognitive tasks protocol, and another involving a population of 20 subjects, which followed the video bank visualization protocol. We provide an overview of several studies that have used the HiMotion database to test multiple hypothesis in the behavioral and affective domains, highlighting the usefulness of our contribution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Aidos H, Fred A, Silva H, Carreiras C (2013) Evidence accumulation approach applied to EEG analysis. In: Proceedings of the 2nd international conference on pattern recognition applications and methods (ICPRAM), pp 479–484

  2. Batliner A, Fischer K, Huber R, Spilker J, Nöth E (2003) How to find trouble in communication. Speech Commun 40(1–2):117–143

    Article  MATH  Google Scholar 

  3. Bradley M, Lang P (1999) International affective digitized sounds (IADS): stimuli, instruction manual and affective ratings. Tech. Rep. B-2, The Center for Research in Psychophysiology, University of Florida, Gainesville, FL

  4. Butcher JN, Mineka S, Hooley JM (2009) Abnormal psychology, 14th edn. Pearson

  5. Canento F, Fred A, Silva H, Gamboa H, Lourenço A (2011) Multimodal biosignal sensor data handling for emotion recognition. In: Proceedings of the IEEE sensors conference, pp 28–31

  6. Canento F, Silva H, Fred A (2012) Applicability of multi-modal electrophysiological data acquisition and processing to emotion recognition. In: Proceedings of the 2nd international workshop on computing paradigms for mental health (MindCare)

  7. Carreiras C, Aidos H, Silva H, Fred A (2013) Exploratory EEG analysis using clustering and phase-locking factor. In: Proceedings of the 6th international conference on bio-inspired systems and signal processing (BIOSTEC), pp 79–88

  8. Carvalho S, Leite J, Galdo-Álvarez S, Gonçalves OF (2012) The emotional movie database (EMDB): a self-report and psychophysiological study. Appl Psychophysiol Biofeedback 37(4):279–294

    Article  Google Scholar 

  9. Douglas-Cowie E, Cowie R, Sneddon I, Cox C, Lowry O, Mcrorie M, Martin JC, Devillers L, Abrilian S, Batliner A, Amir N, Karpouzis K (2007) The HUMAINE database: addressing the collection and annotation of naturalistic and induced emotional data. In: Proceedings of the 2nd international conference on Affective Computing and Intelligent Interaction (ACII). Springer

  10. Fridlund AJ, Cacioppo JT (1986) Guidelines for human electromyographic research. Psychophysiology 23(5):567–589

    Article  Google Scholar 

  11. Fulton J (2000) MENSA—The genius test. Carlton Books

  12. Gamboa H (2008) Multi-modal behavioural biometrics based on HCI and electrophysiology. PhD thesis, Universidade Técnica de Lisboa, Instituto Superior Técnico

  13. Gamboa H, Ferreira V (2003) WIDAM—web interaction display and monitoring. In: 5th International Conference on Enterprise Information Systems (ICEIS), pp 21–27

  14. Gamboa H, Fred A (2004) A behavioral biometric system based on human-computer interaction. In: Jain AK, Ratha NK (eds) SPIE 5404—biometric technology for human identification. Orlando, USA

  15. Gamboa H, Fred A, Jain A (2007) Webbiometrics: user verification via web interaction. In: Biometrics symposium, pp 1–6

  16. Goldberger A, Amaral L, Glass L, Hausdorff J, Ivanov P, Marck R, Mietus J, Moody G, Peng C, Stanley H (2000) PhysioBank, physiotoolkit, and physionet: components of a new research resource for complex physiologic signals. Circulation 101(23):215–220

    Article  Google Scholar 

  17. Healey J, Picard R (2005) Detecting stress during real-world driving tasks using physiological sensors. IEEE Trans Intell Transp Syst 6(2):156–166

    Article  Google Scholar 

  18. Hedman E, Quigley K, Picard R (2012) Using ranked SCRs in ambulatory measurements: a new approach to distinguishing real-world salient events. In: Proceedings of the 52nd annual meeting of society for psychophysiological research

  19. Herwig U, Satrapi P, Schonfeldt-Lecuona C (2003) Using the international 10–20 EEG system for positioning of transcranial magnetic stimulation. Brain Topogr 16(2):95–99

    Article  Google Scholar 

  20. Hewig J, Hagemann D, Seifert J, Gollwitzer M, Naumann E, Bartussek D (2005) A revised film set for the induction of basic emotions. Cogn Emot 19(7):1095–1109

    Article  Google Scholar 

  21. Kaplan RM, Saccuzzo DR (2001) Psychological testing: principles, applications, and issues, 5th edn. Wadsworth

  22. Koelstra S, Muhl C, Soleymani M, Lee JS, Yazdani A, Ebrahimi T, Pun T, Nijholt A, Patras I (2012) DEAP: a database for emotion analysis; using physiological signals. IEEE Trans Affect Comput 3(1):18–31

    Article  Google Scholar 

  23. Lang P, Bradley M, Cuthbert B (2008) International affective picture system (IAPS): affective ratings of pictures and instruction manual. Tech. Rep. A-8, University of Florida, Gainesville, FL

  24. Lewis M, Haviland-Jones JM, Barrett LF (eds) (2010) Handbook of emotions, 3rd edn. The Guilford Press

  25. Medina LAS, Fred ALN (2010) Genetic algorithm for clustering temporal data—application to the detection of stress from ECG signals. In: Proc 2nd Int’l Conf. on Agents and Artificial Intelligence (ICAART), pp 135–142

  26. Pantic M, Valstar M, Rademaker R, Maat L (2005) Web-based database for facial expression analysis. In: IEEE International Conference on Multimedia and Expo (ICME), pp 6–8

  27. Picard RW (2000) Affective computing, 1st edn. The MIT Press

  28. Rottenberg J, Johnson SL (eds) (2007) Emotion and psychopathology: bridging affective and clinical science. American Psychological Association (APA)

  29. Silva H (2007) Feature selection in pattern recognition systems. MSc thesis, Instituto Superior T\(\acute{\rm e}\)cnico, Universidade T\(\acute{{\rm e}}\)cnica de Lisboa

  30. Silva H, Gamboa H, Fred A (2007) Applicability of lead V2 ECG measurements in biometrics. In: Proc. of the int’l eHealth, Telemedicine and Health ICT forum (Med-e-Tel), pp 177–180

  31. Silva H, Gamboa H, Fred A (2007) One lead ECG based human identification with feature subspace ensembles. In: Proc. of the 5th int’l con. on machine learning and data mining, pp 770–783

  32. Sneddon I, McRorie M, McKeown G, Hanratty J (2012) The belfast induced natural emotion database. IEEE Trans Affect Comput 3(1):32–41

    Article  Google Scholar 

  33. Soleymani M, Lichtenauer J, Pun T, Pantic M (2012) A multimodal database for affect recognition and implicit tagging. IEEE Trans Affect Comput 3(1):42–55

    Article  Google Scholar 

  34. Spreen O, Strauss E (1998) A compendium of neuropsychological tests: administration, norms, and commentary, 2nd edn. Oxford University Press, USA

    Google Scholar 

  35. Tolkmitt FJ, Scherer KR (1986) Effect of experimentally induced stress on vocal parameters. J Exp Psychol Hum Percept Perform 12(3):302–313

    Article  Google Scholar 

  36. Zeng Z, Pantic M, Roisman GI, Huang TS (2009) A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Trans Pattern Anal Mach Intell 31(1):39–58

    Article  Google Scholar 

Download references

Acknowledgements

In the context of the HiMotion project there were several entities and people that helped or provided their support to the development of the project. First of all we express the gratitude to INSTICC in the person of Prof. Joaquim Filipe, by the support to the project by providing funding resources for scholarships and electrophysiology equipment. Part of the data acquisition was performed in the facilities of the Escola Superior de Tecnologia de Setúbal, where part of the subjects that voluntary participated in the projected were recruited. The research was preformed in the Pattern and Image Analysis group of Instituto de Telecomunicações, to which we would also like to thank. Besides the researchers, the HiMotion project had collaboration in diverse extents of Ricardo Gamboa, David Cordeiro, João Almeida, and Filipe Canento; we address a special thanks for their involvement and collaboration to the project. We would also like to thank to the psychologist, Dr. Hans Welling from Student Counseling Center at Instituto Superior Técnico and Dr. Teresa Paiva from the University of Lisbon Medical School which, kindly gave advise in preliminary stages of the design of the experiments. This work was also partially funded by the National Strategic Reference Framework (NSRF-QREN) programme under contract no. 3475 “Affective Mouse”, by PLUX—Wireless Biosignals, S.A., and by the Fundação para a Ciência e Tecnologia (FCT) under the grant SFRH/BD/65248/2009, whose support the authors gratefully acknowledge.

Author information

Authors and Affiliations

  1. CEFITEC - Centro de Física e Investigação Tecnológica, Faculdade de Ciência e Tecnologia, 2829-516, Caparica, Portugal

    Hugo Gamboa

  2. IT - Instituto de Telecomunicações, Instituto Superior Técnico, 1049-001, Lisboa, Portugal

    Hugo Silva & Ana Fred

Authors
  1. Hugo Gamboa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hugo Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ana Fred
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hugo Silva.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gamboa, H., Silva, H. & Fred, A. HiMotion: a new research resource for the study of behavior, cognition, and emotion. Multimed Tools Appl 73, 345–375 (2014). https://doi.org/10.1007/s11042-013-1602-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-013-1602-x

Keywords

Search

Navigation