Skip to main content
SpringerLink
Log in

Multimodal Approach to Measuring Cognitive Load Using Sternberg Memory and Input Diagrammatic Reasoning Tests

  • Conference paper
  • First Online:
Intelligent Information and Database Systems (ACIIDS 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13758))

Included in the following conference series:

  • 737 Accesses

Abstract

Results of a study of cognitive load using multimodal biometric techniques including electrocardiography, electroencephalography and galvanic skin response are presented in the paper. Thirty student volunteers took part in an experiment conducted on the iMotions integrated biometric platform in a laboratory setting. Two types of tests were employed as research stimuli, namely the Sternberg memory test and input diagrammatic reasoning test. Data were collected using participant activity measures, Single Ease Question (SEQ) and NASA Task Load Index (NASA-TLX) self-report questionnaires, and biometric measurements. In total, 21 metrics were calculated, including two performance, eight subjective, four electrocardiographic, three encephalographic, and four galvanic skin response metrics based on the collected experimental data. The nonparametric Wilcoxon tests were applied to find statistically significant differences between individual metrics for the Sternberg memory tasks and input diagrammatic reasoning tasks for easy and hard difficulty levels. The conducted research allowed to make many interesting observations and showed the usefulness of various measures in the analysis of the cognitive load associated with memory and reasoning tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Sweller, J.: Cognitive load during problem solving: effects on learning. Cogn. Sci. 12(1), 257–285 (1988). https://doi.org/10.1016/0364-0213(88)90023-7

    Article  Google Scholar 

  2. Reddy, G.S.R., et al.: Estimating cognitive load and cybersickness of pilots in VR simulations via unobtrusive physiological sensors. In: Chen, J.Y.C., Fragomeni, G. (eds) Virtual, Augmented and Mixed Reality: Applications in Education, Aviation and Industry. HCII 2022. LNCS, vol. 13318, pp. 251–269, Springer, Cham (2022). https://doi.org/10.1007/978-3-031-06015-1_18

  3. Pillai, P., Balasingam, B., Kim, Y.H., Lee, C., Biondi, F.: Eye-gaze metrics for cognitive load detection on a driving simulator. IEEE/ASME Trans. Mechatron. 27(4), 2134–2141 (2022). https://doi.org/10.1109/TMECH.2022.3175774

    Article  Google Scholar 

  4. Armstrong, N.C., et al.: Cognitive performance of military men and women during prolonged load carriage. BMJ Mil. Health 1–9 (2022). https://doi.org/10.1136/bmjmilitary-2021-002000

  5. Karczewska, B., Kukla, E., Muke, P.Z., Telec, Z., Trawiński, B.: Usability study of mobile applications with cognitive load resulting from environmental factors. In: Nguyen, N.T., Chittayasothorn, S., Niyato, D., Trawiński, B. (eds.) ACIIDS 2021. LNCS (LNAI), vol. 12672, pp. 851–864. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-73280-6_67

    Chapter  Google Scholar 

  6. Zihisire Muke, P., Trawinski, B.: Concept of research into cognitive load in human computer interaction using biometric techniques. In: Proceedings of the PP-RAI 2019 Conference, Wrocław, Poland, pp. 78–83 (2019). http://pp-rai.pwr.edu.pl/PPRAI19_proceedings.pdf. Accessed 01 Sep 2022

  7. iMotions Biometric Research Platform (8.1). iMotions A/S, Copenhagen, Denmark (2021)

    Google Scholar 

  8. Waghela, B.: Study of cognitive load during human-computer interaction using electromyographic and galvanic skin response measurements. Master’s thesis. Wroclaw University of Science and Technology, Wrocław (2021)

    Google Scholar 

  9. Kota, G.P.: Study of cognitive load during human-computer interaction using electrocardiography. Master’s thesis. Wroclaw University of Science and Technology, Wrocław (2021)

    Google Scholar 

  10. Singh, V.: Study of cognitive load during human-computer interaction using electroencephalographic and pupillometric measurements. Master’s thesis. Wroclaw University of Science and Technology, Wrocław (2021)

    Google Scholar 

  11. Zihisire Muke, P., Piwowarczyk, M., Telec, Z., Trawiński, B., Maharani, P.A., Bresso, P.: Impact of the Stroop effect on cognitive load using subjective and psychophysiological measures. In: Nguyen, N.T., Iliadis, L., Maglogiannis, I., Trawiński, B. (eds.) ICCCI 2021. LNCS (LNAI), vol. 12876, pp. 180–196. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-88081-1_14

    Chapter  Google Scholar 

  12. Zihisire Muke, P., Telec, Z., Trawiński, B.: Cognitive load measurement using arithmetic and graphical tasks and galvanic skin response. In: Nguyen, N.T., et al. (eds.) ICCCI 2022. LNCS (LNAI), vol. 13501, pp. 836–850. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16014-1_66

  13. Nourbakhsh, N., Chen, F., Wang, Y., Calvo, R.A.: Detecting users’ cognitive load by galvanic skin response with affective interference. ACM Trans. Interact. Intell. Syst. 7(3), 1–20 (2017). https://doi.org/10.1145/2960413

    Article  Google Scholar 

  14. Young, J.Q., Van Merrienboer, J., Durning, S., Ten Cate, O.: Cognitive load theory: implications for medical education: AMEE Guide No. 86. Med. Teach. 36(5), 371–384 (2014). https://doi.org/10.3109/0142159X.2014.889290

  15. Baddeley, A.: Recent developments in working memory. Curr. Opin. Neurobiol. 8(2), 234–238 (1998)

    Article  MathSciNet  Google Scholar 

  16. Paivio, A.: Dual coding theory and the mental lexicon. Ment. Lex. 5(2), 205–230 (2010). https://doi.org/10.1075/ml.5.2.04pai

    Article  Google Scholar 

  17. Orru, G., Longo, L.: The evolution of cognitive load theory and the measurement of its intrinsic, extraneous and germane loads: a review. In: Longo, L., Leva, M.C. (eds.) H-WORKLOAD 2018. CCIS, vol. 1012, pp. 23–48. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-14273-5_3

    Chapter  Google Scholar 

  18. Greenberg, K., Zheng, R.: Cognitive load theory and its measurement: a study of secondary tasks in relation to working memory. J. Cogn. Psychol. 34(4), 497–515 (2022). https://doi.org/10.1080/20445911.2022.2026052

    Article  Google Scholar 

  19. Berssanette, J.H., de Francisco, A.C.: Cognitive load theory in the context of teaching and learning computer programming: a systematic literature review. IEEE Trans. Educ. 65(3), 440–449 (2021). https://doi.org/10.1109/TE.2021.3127215

    Article  Google Scholar 

  20. Ghanbari, S., Haghani, F., Barekatain, M., Jamali, A.: A systematized review of cognitive load theory in health sciences education and a perspective from cognitive neuroscience. J. Edu. Health Promot. 9, 176 (2020). https://doi.org/10.4103/jehp.jehp_643_19

    Article  Google Scholar 

  21. He, M., Guo, J., Zeng, S.: Cognitive load measurement and impact analysis on performance in dual-task situations. In: ACM International Conference Proceeding Series, pp. 303–307 (2020). https://doi.org/10.1145/3425329.3425388

  22. Gibson, A., et al.: Assessing usability testing for people living with dementia. In: REHAB 2016: Proceedings of the 4th Workshop on ICTs for improving Patients Rehabilitation Research Techniques, pp. 25–31 (2016). https://doi.org/10.1145/3051488.3051492

  23. Paas, F., Tuovinen, J.E., Tabbers, H., Van Gerven, P.W.M.: Cognitive load measurement as a means to advance cognitive load theory. Educ. Psychol. 38(1), 63–71 (2003). https://doi.org/10.1207/S15326985EP3801_8

    Article  Google Scholar 

  24. Chen, F., et al.: Robust Multimodal Cognitive Load Measurement. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-31700-7

  25. Arshad, S., Wang, Y., Chen, F.: Analysing mouse activity for cognitive load detection. In: Proceedings of the 25th Australian Computer-Human Interaction Conference: Augmentation, Application, Innovation, Collaboration, OzCHI 2013, pp. 115–118 (2013). https://doi.org/10.1145/2541016.2541083

  26. Zhu, X., Aryadoust, V.: A synthetic review of cognitive load in distance interpreting: toward an explanatory model. Front. Psychol. 13, 899718 (2022). https://doi.org/10.3389/fpsyg.2022.899718

    Article  Google Scholar 

  27. Tracy, J.P., Albers, M.J.: Measuring cognitive load to test the usability of web sites. SIGDOC 2011: Proceedings of the 29th ACM international conference on Design of communication, pp. 25–32 (2011). https://doi.org/10.1145/2038476.2038481

  28. Klabes, J., Babilon, S., Zandi, B., Khanh, T.Q.: The Sternberg paradigm: correcting encoding latencies in visual and auditory test designs. Vision (Switzerland) 5(2), 1–13 (2021). https://doi.org/10.3390/vision5020021

    Article  Google Scholar 

  29. Wang, Y., Wang, Q., Du, J., Lin, Y.: Quantifying cognitive load in wayfinding information review using EEG. In: Construction Research Congress 2020: Computer Applications, 8–10 March 2020. Tempe, Arizona (2020). https://doi.org/10.1061/9780784482865.06

  30. Yogesh, S., Ratna, S., Anjana, T.: Immediate and long-term effects of meditation on acute stress reactivity, cognitive functions, and intelligence. Altern. Ther. Health Med. 18(6), 46–53 (2012)

    Google Scholar 

  31. Hoffmann, M.H.G.: Cognitive conditions of diagrammatic reasoning. Semiotica 2011(186), 189–212 (2011). https://doi.org/10.1515/semi.2011.052

    Article  Google Scholar 

  32. Byron, M.: How to Pass Diagrammatic Reasoning Tests: Essential Practice for Abstract, Input Type and Spacial Reasoning Tests, pp. 71–133. Kogan Page, London and Philadelphia (2008)

    Google Scholar 

  33. Byron, M.: Ultimate Psychometric Tests: Over 1000 Verbal Numerical Diagrammatic and Personality Tests, pp. 231–237. Kogan Page, London, Philadelphia, and New Delhi (2015)

    Google Scholar 

  34. Ghaderyan, P., Abbasi, A.: A novel cepstral-based technique for automatic cognitive load estimation. Biomed. Signal Process. Control 39, 396–404 (2018). https://doi.org/10.1016/j.bspc.2017.07.020

    Article  Google Scholar 

  35. Tjolleng, A., et al.: Classification of a Driver’s cognitive workload levels using artificial neural network on ECG signals. Appl. Ergon. 59, 326–332 (2017). https://doi.org/10.1016/j.apergo.2016.09.013

    Article  Google Scholar 

  36. Kumar, P., Das, A.K., Halder, S.: Time-domain HRV analysis of ECG signal under different body postures. Procedia Comput. Sci. 167, 1705–1710 (2020). https://doi.org/10.1016/j.procs.2020.03.435

    Article  Google Scholar 

  37. Shaffer, F., Ginsberg, J.P.: An overview of heart rate variability metrics and norms. Front Public Health 5, Article 258 (2017). https://doi.org/10.3389/fpubh.2017.00258

  38. Elahi, M.T.E., Binta Islam, I.: Galvanic skin response signal based cognitive load classification using machine learning classifier. In: 3rd International Conference on Electrical, Computer and Telecommunication Engineering, ICECTE 2019, pp. 33–36 (2019). https://doi.org/10.1109/ICECTE48615.2019.9303564

  39. Abdelrahman, Y., Velloso, E., Dingler, T., Schmidt, A., Vetere, F.: Cognitive heat: exploring the usage of thermal imaging to unobtrusively estimate cognitive load. In: Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 1, Issue 3, September 2017, Article No.: 33, pp. 1–20 (2017). https://doi.org/10.1145/3130898

  40. Keskin, M., Ooms, K., Dogru, A.O., De Maeyer, P.: Exploring the cognitive load of expert and novice map users using EEG and eye tracking. ISPRS Int. J. Geo-Inf. 9(7), 429 (2020). https://doi.org/10.3390/ijgi9070429

    Article  Google Scholar 

  41. Dan, A., Reiner, M.: EEG-based cognitive load of processing events in 3D virtual worlds is lower than processing events in 2D displays. Int. J. Psychophysiol. 122, 75–84 (2017). https://doi.org/10.1016/j.ijpsycho.2016.08.013

    Article  Google Scholar 

  42. Word frequency data (2021). https://www.wordfrequency.info/samples.asp. Accessed 18 Jan 2021

  43. GSR+ User Guide Revision 1.13 Shimmer, Dublin (2018). https://shimmersensing.com/support/wireless-sensor-networks-documentation/

  44. ECG User Guide Revision 1.12. Shimmer, Dublin (2018). https://shimmersensing.com/support/wireless-sensor-networks-documentation/

  45. Enobio Research User Manual. Neuroelectrics, Barcelona (2021). https://www.neuroelectrics.com/resources/manuals/

  46. HRV R-Notebooks: Processing in iMotions and algorithms used. https://help.imotions.com/hc/en-us/articles/360010089679-HRV-R-Notebooks-Processing-in-iMotions-and-algorithms-used (2022). Accessed 4 July 2022

  47. R Notebook (EEG): Power Spectral Density (2022). https://help.imotions.com/hc/en-us/articles/360014861719-R-Notebook-EEG-Power-Spectral-Density. Accessed 4 July 2022

  48. GSR R-Notebooks: Processing in iMotions and algorithms used (Latest Version) (2022). https://help.imotions.com/hc/en-us/articles/360010312220-GSR-R-Notebooks-Processing-in-iMotions-and-algorithms-used-Latest-Version. Accessed 4 July 2022

  49. Ahmad, M.I., Keller, I., Robb, D.A., et al.: A framework to estimate cognitive load using physiological data. Pers. Ubiquit. Comput. 1-15 (2020). https://doi.org/10.1007/s00779-020-01455-7

  50. Cabañero, L., Hervás, R., González, I., Fontecha, J., Mondéjar, T., Bravo, J.: Characterisation of mobile-device tasks by their associated cognitive load through EEG data processing. Future Gener. Comput. Syst. 113, 380–390 (2020). https://doi.org/10.1016/j.future.2020.07.013

    Article  Google Scholar 

  51. Shi, Y., Ruiz, N., Taib, R., Choi, E., Chen, F.: Galvanic skin response (GSR) as an index of cognitive load. In: Conference on Human Factors in Computing Systems - Proceedings, pp. 2651–2656 (2007). https://doi.org/10.1145/1240866.1241057

Download references

Author information

Authors and Affiliations

  1. Department of Applied Informatics, Wrocław University of Science and Technology, Wrocław, Poland

    Patient Zihisire Muke, Zbigniew Telec & Bogdan Trawiński

Authors
  1. Patient Zihisire Muke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zbigniew Telec
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bogdan Trawiński
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bogdan Trawiński .

Editor information

Editors and Affiliations

  1. Wrocław University of Science and Technology, Wrocław, Poland

    Ngoc Thanh Nguyen

  2. Vietnam National University, Ho Chi Minh City, Ho Chi Minh City, Vietnam

    Tien Khoa Tran

  3. Al-Farabi Kazakh National University, Almaty, Kazakhstan

    Ualsher Tukayev

  4. National University of Kaohsiung, Kaohsiung, Taiwan

    Tzung-Pei Hong

  5. Wrocław University of Science and Technology, Wrocław, Poland

    Bogdan Trawiński

  6. University of Newcastle, Newcastle, NSW, Australia

    Edward Szczerbicki

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Muke, P.Z., Telec, Z., Trawiński, B. (2022). Multimodal Approach to Measuring Cognitive Load Using Sternberg Memory and Input Diagrammatic Reasoning Tests. In: Nguyen, N.T., Tran, T.K., Tukayev, U., Hong, TP., Trawiński, B., Szczerbicki, E. (eds) Intelligent Information and Database Systems. ACIIDS 2022. Lecture Notes in Computer Science(), vol 13758. Springer, Cham. https://doi.org/10.1007/978-3-031-21967-2_56

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-21967-2_56

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-21966-5

  • Online ISBN: 978-3-031-21967-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation