Skip to main content
SpringerLink
Log in

Analysis of the “D’oh!” Moments. Physiological Markers of Performance in Cognitive Switching Tasks

  • Conference paper
  • First Online:
Technology-Enhanced Learning for a Free, Safe, and Sustainable World (EC-TEL 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12884))

Included in the following conference series:

  • 1651 Accesses

Abstract

The link between the body and mind has fascinated philosophers and scientists for ages. The increasing availability of sensor technologies has enabled the possibility to explore this link even deeper, providing some evidence that certain physiological measurements such as galvanic skin response can have in the performance of different learning activities. In this paper, we explore the link between learners’ performance of cognitive tasks and their physiological state with the use of Multimodal Learning Analytics (MMLA). We used MMLA tools and techniques to collect, annotate, and analyse physiological data from 16 participants wearing an Empatica E4 wristband while engaging in task-switching cognitive exercises. The collected data include temperature, blood volume pulse, heart rate variability, galvanic skin response, and screen recording from each participant while performing the exercises. To examine the link between cognitive performance we applied a preliminary qualitative analysis to galvanic skin response and tested different Artificial Intelligence techniques to differentiate between productive and unproductive performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.empatica.com/research/e4/.

  2. 2.

    https://tsfresh.com/.

References

  1. James, W.: The Principles of Psychology, vol. 1. Cosimo, Inc. (2007)

    Google Scholar 

  2. Marmolejo-Ramos, F., et al.: Your face and moves seem happier when i smile. Exp. Psychol. (2020)

    Google Scholar 

  3. Duncan, J.W., Laird, J.D.: Positive and reverse placebo effects as a function of differences in cues used in self-perception. J. Pers. Soc. Psychol. 39(6), 1024 (1980)

    Article  Google Scholar 

  4. Foglia, L., Wilson, R.A.: Embodied cognition. Wiley Interdisc. Rev. Cogn. Sci. 4(3), 319–325 (2013)

    Article  Google Scholar 

  5. Derakshan, N., Eysenck, M.W.: Anxiety, processing efficiency, and cognitive performance (2009)

    Google Scholar 

  6. Sandi, C.: Stress and cognition. Wiley Interdisc. Rev. Cogn. Sci. 4(3), 245–261 (2013)

    Article  Google Scholar 

  7. Raab, M., Araújo, D.: Embodied cognition with and without mental representations: the case of embodied choices in sports. Front. Psychol. 10, 1825 (2019)

    Article  Google Scholar 

  8. Greller, W., Drachsler, H.: Translating learning into numbers: a generic framework for learning analytics. J. Educ. Technol. Soc. 15(3), 42–57 (2012)

    Google Scholar 

  9. Pardo, A., Kloos, C. D.: Stepping out of the box: towards analytics outside the learning management system. In: Proceedings of the 1st International Conference on Learning Analytics and Knowledge, pp. 163–167 (2011)

    Google Scholar 

  10. Di Mitri, D., Schneider, J., Specht, M., Drachsler, H.: From signals to knowledge: a conceptual model for multimodal learning analytics. J. Comput. Assist. Learn. 34(4), 338–349. Ew developments from attentional control theory. Eur. Psychol. 14(2), 168 (2018)

    Google Scholar 

  11. Giannakos, M.N., Sharma, K., Pappas, I.O., Kostakos, V., Velloso, E.: Multimodal data as a means to understand the learning experience. Int. J. Inf. Manage. 48, 108–119 (2019)

    Article  Google Scholar 

  12. Sharma, K., Pappas, I., Papavlasopoulou, S., Giannakos, M.: Towards automatic and pervasive physiological sensing of collaborative learning. In: Lund, K., Niccolai, G.P., Lavoué, E., Gweo, C.H., Baker, M. (eds.) Thirteenth International Conference on Computer Supported Collaborative Learning (CSCL), pp. 684–687 (2019)

    Google Scholar 

  13. Chanel, G., Bétrancourt, M., Pun, T., Cereghetti, D., Molinari, G.: Assessment of computer-supported collaborative processes using interpersonal physiological and eye-movement coupling. In Proceedings of Humaine Association Conference on Affective Computing and Intelligent Interaction, pp. 116–122 (2013)

    Google Scholar 

  14. Spann, C.A., Schaeffer, J., Siemens, G.: Expanding the scope of learning analytics data: preliminary findings on attention and self-regulation using wearable technology. In: LAK17, pp. 203–207 (2017)

    Google Scholar 

  15. Bleck, M., Le, N.T., Pinkwart, N.: Physiology-aware learning analytics using pedagogical agents (2020)

    Google Scholar 

  16. Pijeira-Díaz, H.J., Drachsler, H., Kirschner, P.A., Järvelä, S.: Profiling sympathetic arousal in a physics course: How active is students? J. Comput. Assist. Learn. 34(4), 397–408 (2018)

    Article  Google Scholar 

  17. Larmuseau, C., Vanneste, P., Cornelis, J., Desmet, P., Depaepe, F.: Combining physiological data and subjective measurements to investigate cognitive load during complex learning. Frontline Learn. Res. 7(2), 57–74 (2019)

    Article  Google Scholar 

  18. Worsley, M., Blikstein, P.: What’s an expert? Using learning analytics to identify emergent markers of expertise through automated speech, sentiment and sketch analysis. In: EDM 2011, pp. 235–240 (2011)

    Google Scholar 

  19. Sharma, K., Niforatos, E., Giannakos, M., Kostakos, E.: Assessing cognitive performance using physiological and facial features: generalizing across contexts. Proc. ACM Interact. Mob. Wearable Ubiquit. Technol. 4(3), 1–41 (2020)

    Google Scholar 

  20. Ryu, K., Myung, R.: Evaluation of mental workload with a combined measure based on physiological indices during a dual task of tracking and mental arithmetic. Int. J. Ind. Ergon. 35(11), 991–1009 (2005)

    Article  Google Scholar 

  21. Sharma, K., Giannakos, M.: Multimodal data capabilities for learning: what can multimodal data tell us about learning? Br. J. Edu. Technol. 51(5), 1450–1484 (2020)

    Article  Google Scholar 

  22. Scott, W.A.: Cognitive complexity and cognitive flexibility. Sociometry 405–414 (1962)

    Google Scholar 

  23. Jacob, R., Parkinson, J.: The potential for school-based interventions that target executive function to improve academic achievement: a review. Rev. Educ. Res. 85(4), 512–552 (2015)

    Article  Google Scholar 

  24. Di Mitri, D., Schneider, J., Specht, M.M., Drachsler, H. J.: Multimodal pipeline: a generic approach for handling multimodal data for supporting learning. In: First workshop on AI-based Multimodal Analytics for Understanding Human Learning in Real-world Educational Contexts (2019)

    Google Scholar 

  25. Chelune, G.J., Baer, R.A.: Developmental norms for the Wisconsin card sorting test. J. Clin. Exp. Neuropsychol. 8(3), 219–228 (1986)

    Article  Google Scholar 

  26. Schneider, J., Di Mitri, D., Limbu, B., Drachsler, H.: Multimodal learning hub: a tool for capturing customizable multimodal learning experiences. In: Pammer-Schindler, V., Pérez-Sanagustín, M., Drachsler, H., Elferink, R., Scheffel, M. (eds.) EC-TEL 2018. LNCS, vol. 11082, pp. 45–58. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-98572-5_4

    Chapter  Google Scholar 

  27. Di Mitri, D., Schneider, J., Klemke, R., Specht, M., Drachsler, H. : Read between the lines: an annotation tool for multimodal data for learning. In: LAK19, pp. 51–60 (2019)

    Google Scholar 

  28. Blasiak, S., Rangwala, H.: A hidden Markov model variant for sequence classification. In: Twenty-Second International Joint Conference on Artificial Intelligence (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Goethe University, Frankfurt am Main, Germany

    Tetiana Buraha

  2. DIPF, Frankfurt am Main, Germany

    Jan Schneider, Daniele Di Mitri & Daniel Schiffner

Authors
  1. Tetiana Buraha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan Schneider
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniele Di Mitri
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daniel Schiffner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jan Schneider .

Editor information

Editors and Affiliations

  1. KU Leuven, Leuven, Belgium

    Tinne De Laet

  2. Open University of the Netherlands, Heerlen, The Netherlands

    Roland Klemke

  3. Universidad Carlos III de Madrid, Leganés, Spain

    Carlos Alario-Hoyos

  4. Pontificia Universidad Católica de Chile, Santiago, Chile

    Isabel Hilliger

  5. Universidad de Valladolid, Valladolid, Spain

    Alejandro Ortega-Arranz

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Buraha, T., Schneider, J., Di Mitri, D., Schiffner, D. (2021). Analysis of the “D’oh!” Moments. Physiological Markers of Performance in Cognitive Switching Tasks. In: De Laet, T., Klemke, R., Alario-Hoyos, C., Hilliger, I., Ortega-Arranz, A. (eds) Technology-Enhanced Learning for a Free, Safe, and Sustainable World. EC-TEL 2021. Lecture Notes in Computer Science(), vol 12884. Springer, Cham. https://doi.org/10.1007/978-3-030-86436-1_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-86436-1_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-86435-4

  • Online ISBN: 978-3-030-86436-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation