Skip to main content
SpringerLink
Log in
Book cover

Conference on Artificial Intelligence in Medicine in Europe

AIME 2017: Artificial Intelligence in Medicine pp 295–304Cite as

Detecting Mental Fatigue from Eye-Tracking Data Gathered While Watching Video

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10259))

Abstract

Monitoring mental fatigue is of increasing importance for improving cognitive performance and health outcomes. Previous models using eye-tracking data allow inference of fatigue in cognitive tasks, such as driving, but they require us to engage in a specific cognitive task. A model capable of estimating fatigue from eye-tracking data in natural-viewing situations when an individual is not performing cognitive tasks has many potential applications. Here, we collected eye-tracking data from 18 adults as they watched video clips (simulating the situation of watching TV programs) before and after performing cognitive tasks. Using this data, we built a fatigue-detection model including novel feature sets and an automated feature selection method. With eye-tracking data of individuals watching only 30-seconds worth of video, our model could determine whether that person was fatigued with 91.0% accuracy in 10-fold cross-validation (chance 50%). Through a comparison with a model incorporating the feature sets used in previous studies, we showed that our model improved the detection accuracy by up to 13.9% (from 77.1 to 91.0%).

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Alemdar, H., Ersoy, C.: Wireless sensor networks for healthcare: a survey. Comput. Netw. 54(15), 2688–2710 (2010)

    Article  Google Scholar 

  2. Favela, J., Castro, L.A.: Technology and aging. In: García-Peña, C., Gutiérrez-Robledo, L.M., Pérez-Zepeda, M.U. (eds.) Aging Research-Methodological Issues, pp. 121–135. Springer, Cham (2015)

    Chapter  Google Scholar 

  3. Boksem, M.A., Tops, M.: Mental fatigue: costs and benefits. Brain Res. Rev. 59(1), 125–139 (2008)

    Article  Google Scholar 

  4. Avlund, K.: Fatigue in older adults: an early indicator of the aging process? Aging Clin. Exp. Res. 22(2), 100–115 (2010)

    Article  Google Scholar 

  5. Maghout-Juratli, S., Janisse, J., Schwartz, K., Arnetz, B.B.: The causal role of fatigue in the stress-perceived health relationship: a MetroNet study. J. Am. Board Family Med. 23(2), 212–219 (2010)

    Article  Google Scholar 

  6. Hopstaken, J.F., Linden, D., Bakker, A.B., Kompier, M.A.: A multifaceted investigation of the link between mental fatigue and task disengagement. Psychophysiology 52(3), 305–315 (2015)

    Article  Google Scholar 

  7. Schleicher, R., Galley, N., Briest, S., Galley, L.: Blinks and saccades as indicators of fatigue in sleepiness warnings: looking tired? Ergonomics 51(7), 982–1010 (2008)

    Article  Google Scholar 

  8. Di Stasi, L.L., Renner, R., Catena, A., Cañas, J.J., Velichkovsky, B.M., Pannasch, S.: Towards a driver fatigue test based on the saccadic main sequence: a partial validation by subjective report data. Transp. Res. Part C: Emerg. Technol. 21(1), 122–133 (2012)

    Article  Google Scholar 

  9. Dawson, D., Searle, A.K., Paterson, J.L.: Look before you (s)leep: evaluating the use of fatigue detection technologies within a fatigue risk management system for the road transport industry. Sleep Med. Rev. 18(2), 141–152 (2014)

    Article  Google Scholar 

  10. Tseng, P.H., Cameron, I.G., Pari, G., Reynolds, J.N., Munoz, D.P., Itti, L.: High-throughput classification of clinical populations from natural viewing eye movements. J. Neurol. 260(1), 275–284 (2013)

    Article  Google Scholar 

  11. Crabb, D.P., Smith, N.D., Zhu, H.: What’s on TV? Detecting age-related neurodegenerative eye disease using eye movement scanpaths. Frontiers in Aging Neurosci. 6, 312 (2014)

    Article  Google Scholar 

  12. Cook, D.B., O’Connor, P.J., Lange, G., Steffener, J.: Functional neuroimaging correlates of mental fatigue induced by cognition among chronic fatigue syndrome patients and controls. Neuroimage 36(1), 108–122 (2007)

    Article  Google Scholar 

  13. Carmi, R., Itti, L.: The role of memory in guiding attention during natural vision. J. Vis. 6(9), 4 (2006)

    Article  Google Scholar 

  14. Cutting, J.E., DeLong, J.E., Brunick, K.L.: Visual activity in Hollywood film: 1935 to 2005 and beyond. Psychol. Aesthet. Creat. Arts 5(2), 115 (2011)

    Article  Google Scholar 

  15. Bordwell, D.: Intensified continuity visual style in contemporary American film. Film Q. 55(3), 16–28 (2002)

    Article  Google Scholar 

  16. Itti, L., Carmi, R.: Eye-tracking data from human volunteers watching complex video stimuli (2009)

    Google Scholar 

  17. Mital, P.K., Smith, T.J., Hill, R.L., Henderson, J.M.: Clustering of gaze during dynamic scene viewing is predicted by motion. Cogn. Comput. 3(1), 5–24 (2011)

    Article  Google Scholar 

  18. Xu, P., Ehinger, K.A., Zhang, Y., Finkelstein, A., Kulkarni, S.R., Xiao, J.: TurkerGaze: crowdsourcing saliency with webcam based eye tracking. arXiv preprint arXiv:1504.06755 (2015)

  19. Zhang, Y., Wilcockson, T., Kim, K.I., Crawford, T., Gellersen, H., Sawyer, P.: Monitoring dementia with automatic eye movements analysis. In: Czarnowski, I., Caballero, A.M., Howlett, R.J., Jain, L.C. (eds.) Intelligent Decision Technologies 2016. SIST, vol. 57, pp. 299–309. Springer, Cham (2016). doi:10.1007/978-3-319-39627-9_26

    Chapter  Google Scholar 

  20. Tass, P., Rosenblum, M., Weule, J., Kurths, J., Pikovsky, A., Volkmann, J., Schnitzler, A., Freund, H.J.: Detection of n: m phase locking from noisy data: application to magnetoencephalography. Phys. Rev. Lett. 81(15), 3291 (1998)

    Article  Google Scholar 

  21. Treisman, A.M., Gelade, G.: A feature-integration theory of attention. Cogn. Psychol. 12(1), 97–136 (1980)

    Article  Google Scholar 

  22. Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 11, 1254–1259 (1998)

    Article  Google Scholar 

  23. Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: Advances in Neural Information Processing Systems, pp. 545–552 (2006)

    Google Scholar 

  24. Boser, B.E., Guyon, I.M., Vapnik, V.N.: A training algorithm for optimal margin classifiers. In: Proceedings of the Fifth Annual Workshop on Computational Learning Theory, pp. 144–152. ACM (1992)

    Google Scholar 

  25. Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. (TIST) 2(3), 27 (2011)

    Google Scholar 

  26. Guyon, I., Weston, J., Barnhill, S., Vapnik, V.: Gene selection for cancer classification using support vector machines. Mach. Learn. 46(1–3), 389–422 (2002)

    Article  MATH  Google Scholar 

  27. Yan, K., Zhang, D.: Feature selection and analysis on correlated gas sensor data with recursive feature elimination. Sens. Actuators B: Chem. 212, 353–363 (2015)

    Article  Google Scholar 

Download references

Acknowledgments

This research was partially supported by the Japan Science and Technology Agency (JST) under the Strategic Promotion of Innovative Research and Development Program.

Author information

Authors and Affiliations

  1. IBM Research - Tokyo, Tokyo, Japan

    Yasunori Yamada & Masatomo Kobayashi

Authors
  1. Yasunori Yamada
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Masatomo Kobayashi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yasunori Yamada .

Editor information

Editors and Affiliations

  1. Vrije Universiteit Amsterdam, Amsterdam, The Netherlands

    Annette ten Teije

  2. Medical University of Vienna, Vienna, Austria

    Christian Popow

  3. University of Pennsylvania, Philadelphia, Pennsylvania, USA

    John H. Holmes

  4. University of Pavia, Pavia, Italy

    Lucia Sacchi

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Yamada, Y., Kobayashi, M. (2017). Detecting Mental Fatigue from Eye-Tracking Data Gathered While Watching Video. In: ten Teije, A., Popow, C., Holmes, J., Sacchi, L. (eds) Artificial Intelligence in Medicine. AIME 2017. Lecture Notes in Computer Science(), vol 10259. Springer, Cham. https://doi.org/10.1007/978-3-319-59758-4_34

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-59758-4_34

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-59757-7

  • Online ISBN: 978-3-319-59758-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation