Skip to main content
SpringerLink
Log in

Measurement and verification of cognitive load in multimedia presentation using an eye tracker

  • 1175: IoT Multimedia Applications and Services
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The development of interface designs can reduce extraneous processing for users and increase the effectiveness of multimedia presentations. In this study, we investigate cognitive load in multimedia presentations. First, we present a quantitative model to measure cognitive load in terms of information comprehension in which the pupil diameter variation is used as an indicator of cognitive load based on cognitive load theory. We design a verification experiment to measure the pupil diameter using an eye-tracker when different combinations of texts, audio-narrations, and images are presented to subjects. We further allow the subjects take a comprehension test on the presented information and analyze the relationship between cognitive load and the test score using the generalized linear mixed model (GLMM). Moreover, we obtain the subjective cognitive load via a pre-designed questionnaire taken after the experiment and compare these two types of cognitive loads in terms of the mean absolute error (MAE). The experiment results show that there is a gap between the objective cognitive load obtained via the pupillary response and the subjective cognitive load obtained via a questionnaire, and the presentation with an optimized combination of multimedia can enhance information comprehension while reducing cognitive load.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. This study was conducted under the approval of the Ethics Review Committee on Research with Human Subjects of Waseda University, Japan (No.: 2018-092), and all subjects for this experiment signed the informed consent.

  2. Tobii Pro Lab User Manual 1.130, https://www.tobiipro.com/siteassets/tobii-pro/user-manuals/Tobii-Pro-Lab-User-Manual/?v=1.130(LastaccessedonMarch1,2020).

References

  1. Chen S, Epps J, Chen F (2013) An investigation of pupil-based cognitive load measurement with low cost infrared webcam under light reflex interference. In: Proceedings of 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). Osaka, Japan. https://doi.org/10.1109/EMBC.2013.6610222, pp 3202–3205

  2. Clark JM, Paivio A (1991) Dual coding theory and education. Educ Psychol Rev 3(3):149–210. https://doi.org/10.1007/BF01320076

    Article  Google Scholar 

  3. Faraway JJ (2016) Extending the Linear Model with R: Generalized Linear, Mixed Effects and Nonparametric Regression Models. Second Edition. CRC Press

  4. Galy E, Cariou M, Mélan C (2012) What is the relationship between mental workload factors and cognitive load types? Int J Psychophysiol 83 (3):269–275. https://doi.org/10.1016/j.ijpsycho.2011.09.023

    Article  Google Scholar 

  5. Goldinger SD, Papesh MH (2012) Pupil dilation reflects the creation and retrieval of memories. Curr Dir Psychol Sci 21(2):90–95. https://doi.org/10.1177/0963721412436811

    Article  Google Scholar 

  6. Hess EH, Polt JM (1960) Pupil size as related to interest value of visual stimuli. Science 132(3423):349–350. https://doi.org/10.1126/science.132.3423.349

    Article  Google Scholar 

  7. Hess EH, Polt JM (1964) Pupil size in relation to mental activity during simple problem-solving. Science 143(3611):1190–1192. https://doi.org/10.1126/science.143.3611.1190

    Article  Google Scholar 

  8. Hossain G, Yeasin M (2014) Understanding effects of cognitive load from pupillary responses using hilbert analytic phase. In: 2014 IEEE Conference on computer vision and pattern recognition workshops. Columbus, OH, USA. https://doi.org/10.1109/CVPRW.2014.62, pp 381–386

  9. Jercic P, Sennersten C, Lindley C (2020) Modeling cognitive load and physiological arousal through pupil diameter and heart rate. Multimed Tools Appl 79:3145–3159. https://doi.org/10.1007/s11042-018-6518-zhttps://doi.org/10.1007/s11042-018-6518-z

    Article  Google Scholar 

  10. Jiang X, Atkins MS, Tien G, Zheng B, Bednarik R (2014) Pupil dilations during target-pointing respect Fitts’ law. In: Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA ’14). Santa Barbara, California. https://doi.org/10.1145/2578153.2578178, pp 175–182

  11. Johannessen E, Szulewski A, Radulovic N, White M, Braund H, Howes D, Rodenburg D, Davies C (2020) Psychophysiologic measures of cognitive load in physician team leaders during trauma resuscitation. Computers in Human Behavior. https://doi.org/10.1016/j.chb.2020.106393

  12. Klingner J, Tversky B, Hanrahan P (2011) Effects of visual and verbal presentation on cognitive load in vigilance, memory, and arithmetic tasks. Psychophysiol 48(3):323–332. https://doi.org/10.1111/j.1469-8986.2010.01069.xhttps://doi.org/10.1111/j.1469-8986.2010.01069.x

    Article  Google Scholar 

  13. Kumar A, Sangwan SR, Nayyar A (2020) Multimedia social big data: Mining. Multimedia Big Data Computing for IoT Applications 163:289–321. https://doi.org/10.1007/978-981-13-8759-3_11

    Article  Google Scholar 

  14. Leppink J, Paas F, Gog T, Vleuten CPM, Merriënboer JJG (2014) Effects of pairs of problems and examples on task performance and different types of cognitive load. Learn Instr 30:32–42. https://doi.org/10.1016/j.learninstruc.2013.12.001

    Article  Google Scholar 

  15. Liu Y, Zhang C, Hang B, Wang S, Chao HC (2019) An audio attention computational model based on information entropy of two channels and exponential moving average. Hum-centric Comput Inf Sci 9(7). https://doi.org/10.1186/s13673-019-0166-9

  16. Mayer RE (2002) Multimedia learning. Psychol Learn Motiv 41:85–139. https://doi.org/10.1016/S0079-7421(02)80005-6https://doi.org/10.1016/S0079-7421(02)80005-6

    Article  Google Scholar 

  17. Pfleging B, Fekety DK, Schmidt A, Kun AL (2016) A model relating pupil diameter to mental workload and lighting conditions. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI’16). San Jose California, USA. https://doi.org/10.1145/2858036.2858117, pp 5776–5788

  18. Pomplun M, Sunkara S (2003) Pupil dilation as an indicator of cognitive workload in Human-Computer interaction. In: Proceedings of the International Conference on HCI. Crete, Greece, p 273

  19. Rafiqi S, Wangwiwattana C, Kim J, Fernandez E, Nair S, Larson EC (2015) Pupilware: Towards pervasive cognitive load measurement using commodity devices. In: Proceedings of the 8th ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA ’15). Corfu, Greece. Article. https://doi.org/10.1145/2769493.2769506, vol 42, pp 1–8

  20. Reilly J, Kelly A, Kim SH, Jett S, Zuckerman B (2019) The human task-evoked pupillary response function is linear: Implications for baseline response scaling in pupillometry. Behav Res Methods 51(2):865–878. https://doi.org/10.3758/s13428-018-1134-4

    Article  Google Scholar 

  21. Schnotz W, Kürschner C (2007) A reconsideration of cognitive load theory. Educ Psychol Rev 19(4):469–508. https://doi.org/10.1007/s10648-007-9053-4

    Article  Google Scholar 

  22. Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27(3):379–423. https://doi.org/10.1002/j.1538-7305.1948.tb01338.x

    Article  MathSciNet  Google Scholar 

  23. Song H, Lee K, Moon N (2019) User modeling using user preference and user life pattern based on personal bio data and SNS data. J Inf Process Syst 15(3):645–654. https://doi.org/10.3745/JIPS.01.0044

    Google Scholar 

  24. Sweller J (2010) Element interactivity and intrinsic, extraneous, and germane cognitive load. Educ Psychol Rev 22:123–138. https://doi.org/10.1007/s10648-010-9128-5

    Article  Google Scholar 

  25. Sweller J (2011) Cognitive load theory. The psychology of learning and motivation:, Cognition in education 55:37–76. https://doi.org/10.1016/B978-0-12-387691-1.00002-8

    Article  Google Scholar 

  26. Wang L, Guo Y (2019) An evolution model of rumor spreading based on wechat social circle. J Inf Process Syst 15(6):1422–1437. https://doi.org/10.3745/JIPS.04.0152

    Google Scholar 

  27. William J, Murugesh R (2020) Potential eye tracking metrics and indicators to measure cognitive load in Human-Computer interaction research. J Sci Res 64:168–175. https://doi.org/10.37398/JSR.2020.640137https://doi.org/10.37398/JSR.2020.640137

    Google Scholar 

  28. Wu X, Xue C, Zhou F (2015) An Experimental Study on Visual Search Factors of Information Features in a Task Monitoring Interface. Human-Computer Interaction: Users and Contexts 9171:525–536. https://doi.org/10.1007/978-3-319-21006-3_50

    Google Scholar 

  29. Xue Y, Li Z (2019) Research on online learning cognitive load quantitative model based on eye tracking technology. Modern Educ Technol 7:59–65. (in Chinese)

    Google Scholar 

  30. Zagermann J, Pfeil U, Reiterer H (2016) Measuring Cognitive Load using Eye Tracking Technology in Visual Computing. In: Proceedings of the Sixth Workshop on Beyond Time and Errors on Novel Evaluation Methods for Visualization (BELIV ’16). Baltimore. MD. USA. https://doi.org/10.1145/2993901.2993908, pp 78–85

  31. Zeng X (2018) Quantitative Analysis of Pupil Diameter and Pupillary Light Reflex. Soochow University. Master Thesis. (in Chinese)

Download references

Author information

Authors and Affiliations

  1. Graduate School of Human Sciences, Waseda University, Tokorozawa, Japan

    Ruichen Cong & Kiichi Tago

  2. Faculty of Human Sciences, Waseda University, Tokorozawa, Japan

    Qun Jin

Authors
  1. Ruichen Cong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kiichi Tago
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qun Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qun Jin.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cong, R., Tago, K. & Jin, Q. Measurement and verification of cognitive load in multimedia presentation using an eye tracker. Multimed Tools Appl 81, 26821–26835 (2022). https://doi.org/10.1007/s11042-022-13294-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13294-0

Keywords

Search

Navigation