Skip to main content
SpringerLink
Log in

Cognitive Performance and Physiological Response Analysis

Analysis of the Variation of Physiological Parameters Based on User’s Personality, Sensory Profile, and Morningness–Eveningness Type in a Human–Robot Interaction Scenario

  • Published:
International Journal of Social Robotics Aims and scope Submit manuscript

Abstract

For more socially natural interactions between robots and humans there is a need for the robots to detect and understand the current internal state of the individuals they interact with. In this paper, we investigate the impact of personality, user sensory profile, and morningness–eveningness type of an individual on the variation of different physiological parameters (i.e., blinking, galvanic skin response, facial temperature variation), as well as the performance in a non-verbal word-color Stroop task. The TIAGo humanoid robot displayed two interaction styles that are based on visual and auditory stimuli. The visual and auditory stimuli used by the robot are in accordance with the user sensory profile. We report the results of a within participants study carried out with 24 participants. Our results show that personality, user sensory profile, and morningness–eveningness type have an impact on the variation of the chosen physiological parameters. Furthermore, the interaction style of the robot has an influence on the variation of the physiological parameters.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. www.enrichme.eu

  2. www.pal-robotics.com

  3. www.enrichme.eu

  4. http://wiki.seeedstudio.com/Grove-GSR_Sensor/

References

  1. Abdelrahman Y, Velloso E, Dingler T, Schmidt A, Vetere F (2017) Cognitive heat: exploring the usage of thermal imaging to unobtrusively estimate cognitive load. Proc ACM Interact Mob Wearable Ubiquitous Technol 1(3):33

    Article  Google Scholar 

  2. Adan A (1991) Influence of morningness-eveningness preference in the relationship between body temperature and performance: a diurnal study. Pers Individ Dif 12(11):1159–1169

    Article  Google Scholar 

  3. Adan A, Archer SN, Hidalgo MP, Di Milia L, Natale V, Randler C (2012) Circadian typology: a comprehensive review. Chronobiol Int 29(9):1153–1175

    Article  Google Scholar 

  4. Agrigoroaie R, Tapus A (2017) Contactless physiological data analysis for user quality of life improving by using a humanoid social robot. In: International conference on image analysis and processing, pp 696–706

  5. Agrigoroaie R, Tapus A (2017) Influence of robot’s interaction style on performance in a stroop task. In: International conference on social robotics

  6. Agrigoroaie R, Tapus A (2018) ’oh! i am so sorry!’: understanding user physiological variation while spoiling a game task. In: IROS

  7. Bandura A (1999) Social cognitive theory of personality. In: Pervin LA, John OP (eds) Handbook of personality: theory and research. Elsevier, Amsterdam pp 154–196

  8. Bandura A, Jourden FJ (1991) Self-regulatory mechanisms governing the impact of social comparison on complex decision making. J Pers Soc Psychol 60(6):941

    Article  Google Scholar 

  9. Bellotto N, Fernandez-Carmona M, Cosar S, et al. (2017) Enrichme integration of ambient intelligence and robotics for aal. In: AAAI spring symposium series, AAAI

  10. Bentivoglio AR et al (1997) Analysis of blink rate patterns in normal subjects. Mov Disord 12(6):1028–1034

    Article  Google Scholar 

  11. Brown C, Dunn W (2002) Adult/adolescent sensory profile: self questionnaire. Pearson Clinical, Bengaluru

    Google Scholar 

  12. Brown C, Tollefson N, Dunn W, Cromwell R, Filion D (2001) The adult sensory profile: measuring patterns of sensory processing. Am J Occup Ther 55(1):75–82

    Article  Google Scholar 

  13. Campagne A, Pebayle T, Muzet A (2005) Oculomotor changes due to road events during prolonged monotonous simulated driving. Biol Psychol 68(3):353–368

    Article  Google Scholar 

  14. Cassell J, Bickmore T (2000) External manifestations of trustworthiness in the interface. Commun ACM 43(12):50–56

    Article  Google Scholar 

  15. Charles RL, Nixon J (2019) Measuring mental workload using physiological measures: a systematic review. Appl Ergon 74:221–232

    Article  Google Scholar 

  16. Corr P, Cooper A (2016) The reinforcement sensitivity theory of personality questionnaire: development and validation. Psychol Assess 28(11):1427

    Article  Google Scholar 

  17. Cruz-Maya A, Tapus A (2018) Negotiating with a robot: analysis of regulatory focus behavior. In: Proceedings of the IEEE international conference on robotics and automation

  18. Dunn W (1997) The impact of sensory processing abilities on the daily lives of young children and their families: a conceptual model. Infants Young Child 9:23–35

    Article  Google Scholar 

  19. Ehrlich S, Wykowska A, Ramirez-Amaro K, Cheng G (2014) When to engage in interaction and how? eeg-based enhancement of robot’s ability to sense social signals in hri. In: 2014 14th IEEE-RAS international conference on humanoid robots (humanoids), IEEE, pp 1104–1109

  20. Eysenck H (1967) The biological basis of personality. Taylor & Francis Springfield

  21. Eysenck H (1983) Psychophysiology and personality: extraversion, neuroticism and psychoticism. Physiol Correl Hum behav 3:13–30

    Google Scholar 

  22. Eysenck HJ (1991) Dimensions of personality: 16, 5 or 3?–criteria for a taxonomic paradigm. Pers Individ Dif 12(8):773–790

    Article  Google Scholar 

  23. Eysenck HJ, Eysenck SBG (1975) Manual of the eysenck personality questionnaire (junior and adult). Hodder and Stoughton, London

    Google Scholar 

  24. Fukuda K (2001) Eye blinks: new indices for the detection of deception. Int J Psychophysiol 40(3):239–245

    Article  Google Scholar 

  25. Goldberg L (1990) An alternative ‘description of personality’: the big-five factor structure. J Pers Soc Psychol 59(6):1216

    Article  Google Scholar 

  26. Gray J, McNaughton N (2000) The neuropsychology of anxiety: an enquiry into the functions of the septo-hippocampal system, 2nd edn. Oxford University Press, New York

    Google Scholar 

  27. Hall ET (1990) The hidden dimension. Bantam Doubleday Dell Publishing Group Inc, p 217

  28. Horne J, Ostberg O (1976) A self-assessment questionnaire to determine morningness-eveningness in human circadian rhythms. Int J Chronobiol 4(2):97–110

    Google Scholar 

  29. Ioannou S, Morris P, Mercer H, Baker M, Gallese V, Reddy V (2014) Proximity and gaze influences facial temperature: a thermal infrared imaging study. Front Psychol 5:845

    Article  Google Scholar 

  30. Kahneman D (1973) Attention and effort. Prentice-Hall, New Jersey

    Google Scholar 

  31. Kramer AF (1991) Physiological metrics of mental workload: a review of recent progress. Mult Task Perform pp 279–328

  32. Liu C, et.al. (2008) Online affect detection and robot behavior adaptation for intervention of children with autism. In: IEEE transactions on robotics

  33. MacLeod C (1991) Half a century of research on the stroop effect: an integrative review. Psychol Bull 109(2):163

    Article  Google Scholar 

  34. Mitsunaga N et al (2006) Robot behavior adaptation for hri based on policy gradient reinforcement learning. J Robot Soc Jpn 24(7):820–829

    Article  Google Scholar 

  35. Mower E, Feil-Seifer DJ, Mataric MJ, Narayanan S (2007) Investigating implicit cues for user state estimation in human–robot interaction using physiological measurements. In: RO-MAN 2007-The 16th IEEE international symposium on robot and human interactive communication, IEEE, pp 1125–1130

  36. Nakatsu R (1998) Toward the creation of a new medium for the multimedia era. Proc IEEE 86(5):825–836

    Article  Google Scholar 

  37. Nourbakhsh N, Chen F, Wang Y, Calvo RA (2017) Detecting users cognitive load by galvanic skin response with affective interference. ACM Trans Interact Intell Syst 7(3):12:1–12:20

    Article  Google Scholar 

  38. Nourbakhsh N, Wang Y, Chen F (2013) Gsr and blink features for cognitive load classification. In: IFIP conference on human–computer interaction. Springer, pp 159–166

  39. Nourbakhsh N, Wang Y, Chen F, Calvo RA (2012) Using galvanic skin response for cognitive load measurement in arithmetic and reading tasks. In: Proceedings of the 24th Australian computer–human interaction conference, ACM

  40. Or CK, Duffy VG (2007) Development of a facial skin temperature-based methodology for non-intrusive mental workload measurement. Occup Ergon 7(2):83–94

    Google Scholar 

  41. Paas FG, Van Merriënboer JJ (1993) The efficiency of instructional conditions: an approach to combine mental effort and performance measures. Hum Factors 35(4):737–743

    Article  Google Scholar 

  42. Pages J, Marchionni L, Ferro F (2016) Tiago: the modular robot that adapts to different research needs. In: International workshop on robot modularity, IROS

  43. Randler C (2008) Differences between smokers and nonsmokers in morningness–eveningness. Soc Behav Pers 36(5):673–80

    Article  Google Scholar 

  44. Reeves B, Nass CI (1996) The media equation: how people treat computers, television, and new media like real people and places. Cambridge University Press, Cambridge

    Google Scholar 

  45. Renaud P, Blondin JP (1997) The stress of stroop performance: physiological and emotional responses to color-word interference, task pacing, and pacing speed. Int J Psychophysiol 27(2):87–97

    Article  Google Scholar 

  46. Setz C, Arnrich B, Schumm J, La Marca R, Tröster G, Ehlert U (2010) Discriminating stress from cognitive load using a wearable eda device. IEEE Trans Inf Technol Biomed 14(2):410–417

    Article  Google Scholar 

  47. Shi Y, Ruiz N, Taib R, Choi E, Chen F (2007) Galvanic skin response (gsr) as an index of cognitive load. In: Begole B, Payne S, Churchill E, Amant RS, Gilmore D, Rosson MB (eds) CHI’07 extended abstracts on human factors in computing systems, ACM, NY, pp 2651–2656

  48. Sorostinean M, Ferland F, Tapus A (2015) Reliable stress measurement using face temperature variation with a thermal camera in human–robot interaction. In: 15th IEEE-RAS international conference on humanoid robots

  49. Sprent P, Smeeton NC (2000) Applied nonparametric statistical methods. Chapman and Hall/CRC, Santa Clarita

    Book  Google Scholar 

  50. Stelmack RM (1981) The psychophysiology of extraversion and neuroticism. Springer, Berlin, pp 38–64

    Google Scholar 

  51. Stern JA, Walrath LC, Goldstein R (1984) The endogenous eyeblink. Psychophysiology 21(1):22–33

    Article  Google Scholar 

  52. Stroop J (1935) Studies of interference in serial verbal reactions. J Explor Psychol 18(6):643

    Article  Google Scholar 

  53. Svetlak M et al (2010) Electrodermal complexity during the stroop colour word test. Auton Neurosci 152(1–2):101–107

    Article  Google Scholar 

  54. Tanaka Y, Yamaoka K (1993) Blink activity and task difficulty. Percept Mot Skills 77(1):55–66

    Article  Google Scholar 

  55. Tapus A, Tapus C, Mataric MJ (2007) Hands-off therapist robot behavior adaptation to user personality for post-stroke rehabilitation therapy. In: 2007 IEEE international conference on robotics and automation

  56. Tulen J et al (1989) Characterization of stress reactions to the stroop color word test. Pharmacol Biochem Behav 32(1):9–15

    Article  Google Scholar 

  57. Walters M, et.al. (2005) The influence of subjects’ personality traits on personal spatial zones in a human–robot interaction experiment. In: IEEE international workshop on robot and human interactive communication

  58. Yanco HA, Drury JL, Scholtz J (2004) Beyond usability evaluation: analysis of human–robot interaction at a major robotics competition. Hum Comput Interact 19(1–2):117–149

    Article  Google Scholar 

Download references

Funding

This work was funded and done in the context of the EU H2020 ENRICHME project, Grant Agreement No: 643691. The authors declare that they have no conflict of interest.

Author information

Authors and Affiliations

  1. Autonomous Systems and Robotics Laboratory, U2IS, ENSTA ParisTech, 828 Boulevard des Marechaux, Palaiseau, France

    Roxana Agrigoroaie & Adriana Tapus

Authors
  1. Roxana Agrigoroaie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adriana Tapus
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Roxana Agrigoroaie.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Agrigoroaie, R., Tapus, A. Cognitive Performance and Physiological Response Analysis. Int J of Soc Robotics 12, 47–64 (2020). https://doi.org/10.1007/s12369-019-00532-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12369-019-00532-z

Keywords

Search

Navigation