Skip to main content
SpringerLink
Log in

Task-independent auditory probes reveal changes in mental workload during simulated quadrotor UAV training

  • Research
  • Published:
Health Information Science and Systems Aims and scope Submit manuscript

Abstract

Objective

The event-related potential (ERP) methods based on laboratory control scenes have been widely used to measure the level of mental workload during operational tasks. In this study, both task difficulty and test time were considered. Auditory probes (ignored task-irrelevant background sounds) were used to explore the changes in mental workload of unmanned aerial vehicle (UAV) operators during task execution and their ERP representations.

Approach

51 students participated in a 10-day training and test of simulated quadrotor UAV. During the experiment, background sound was played to induce ERP according to the requirements of oddball paradigm, and the relationship between mental workload and the amplitudes of N200 and P300 in ERP was explored.

Main results

Our study shows that the mental workload during operational task training is multi-dimensional, and its changes are affected by bottom-up perception and top-down cognition. The N200 component of the ERP evoked by the auditory probe corresponds to the bottom-up perceptual part; while the P300 component corresponds to the top-down cognitive part, which is positively correlated with the improvement of skill level.

Significance

This paper describes the relationship between ERP induced by auditory probes and mental workload from the perspective of multi-resource theory and human information processing. This suggests that the auditory probe can be used to reveal the mental workload during the training of operational tasks, which not only provides a possible reference for measuring the mental workload, but also provides a possibility for identifying the development of the operator’s skill level and evaluating the training effect.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

Not applicable.

Code availability

Not applicable.

References

  1. Hancock P, Parasuraman R. Human factors and safety in the design of intelligent vehicle-highway systems (ivhs). J Safety Res. 1992;23(4):181–98.

    Google Scholar 

  2. Parasuraman R, Rizzo M. Neuroergonomics: the brain at work, vol. 3. New York: Oxford University Press; 2006.

    Google Scholar 

  3. Parasuraman R, Wilson GF. Putting the brain to work: neuroergonomics past, present, and future. Hum Factors. 2008;50(3):468–74.

    Google Scholar 

  4. Ayaz H, Dehais F. Neuroergonomics: the Brain at work and in everyday life. Cambridge: Academic Press; 2018. p. 111–4.

    Google Scholar 

  5. Mehta RK, Parasuraman R. Neuroergonomics: a review of applications to physical and cognitive work. Front Hum Neurosci. 2013;7:889.

    Google Scholar 

  6. Wickens CD, Helton WS, Hollands JG, Banbury S. Engineering psychology and human performance. London: Routledge; 2021. p. 3–4.

    Google Scholar 

  7. Wickens CD. Multiple resources and mental workload. Hum Factors. 2008;50(3):449–55.

    Google Scholar 

  8. Ayaz H, Shewokis PA, Bunce S, Izzetoglu K, Willems B, Onaral B. Optical brain monitoring for operator training and mental workload assessment. Neuroimage. 2012;59(1):36–47.

    Google Scholar 

  9. Haynes JD, Rees G. Decoding mental states from brain activity in humans. Nat Rev Neurosci. 2006;7(7):523–34.

    Google Scholar 

  10. Tharawadeepimuk K, Wongsawat Y. Quantitative eeg in sports: performance level estimation of professional female soccer players. Health Inf Sci Syst. 2021;9(1):1–15.

    Google Scholar 

  11. Tan W, Xu Y, Liu P, Liu C, Li Y, Du Y, Chen C, Wang Y, Zhang Y. A method of vr-eeg scene cognitive rehabilitation training. Health Info Sci Syst. 2021;9(1):1–9.

    Google Scholar 

  12. Kramer AF, Trejo LJ, Humphrey D. Assessment of mental workload with task-irrelevant auditory probes. Biol Psychol. 1995;40(1–2):83–100.

    Google Scholar 

  13. Ke Y, Jiang T, Liu S, Cao Y, Jiao X, Jiang J, Ming D. Cross-task consistency of electroencephalography-based mental workload indicators: comparisons between power spectral density and task-irrelevant auditory event-related potentials. Front Neurosci. 2021;15: 703139.

    Google Scholar 

  14. Ullsperger P, Freude G, Erdmann U. Auditory probe sensitivity to mental workload changes-an event-related potential study. Int J Psychophysiol. 2001;40(3):201–9.

    Google Scholar 

  15. Luck SJ. An introduction to the event-related potential technique. Cambridg: MIT press; 2014. p. 144–6.

    Google Scholar 

  16. Patel SH, Azzam PN. Characterization of n200 and p300: selected studies of the event-related potential. Int J Med Sci. 2005;2(4):147.

    Google Scholar 

  17. Ghani U, Signal N, Niazi IK, Taylor D. A novel approach to validate the efficacy of single task erp paradigms to measure cognitive workload. Int J Psychophysiol. 2020;158:9–15.

    Google Scholar 

  18. Miller MW, Rietschel JC, McDonald CG, Hatfield BD. A novel approach to the physiological measurement of mental workload. Int J Psychophysiol. 2011;80(1):75–8.

    Google Scholar 

  19. Allison BZ, Polich J. Workload assessment of computer gaming using a single-stimulus event-related potential paradigm. Biol Psychol. 2008;77(3):277–83.

    Google Scholar 

  20. Causse M, Fabre E, Giraudet L, Gonzalez M, Peysakhovich V. Eeg/erp as a measure of mental workload in a simple piloting task. Procedia Manufacturing. 2015;3:5230–6.

    Google Scholar 

  21. Sirevaag EJ, Kramer AF, Coles MG, Donchin E. Resource reciprocity: an event-related brain potentials analysis. Acta Physiol (Oxf). 1989;70(1):77–97.

    Google Scholar 

  22. Ghani U, Signal N, Niazi IK, Taylor D. Efficacy of a single-task erp measure to evaluate cognitive workload during a novel exergame. Front Hum Neurosci. 2021. https://doi.org/10.3389/fnhum.2021.742384.

    Article  Google Scholar 

  23. Tang S, Liu C, Zhang Q, Gu H, Li X, Li Z. Mental workload classification based on ignored auditory probes and spatial covariance. J Neural Eng. 2021;18(4):0460–9.

    Google Scholar 

  24. Xu J, Ke Y, Liu S, Song X, Xu C, Zhou G, Ming D. (2020) Task-irrelevant auditory event-related potentials as mental workload indicators: a between-task comparison study. In: 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), IEEE, pp 3216–3219

  25. Papanicolaou AC, Johnstone J. Probe evoked potentials: theory, method and applications. Int J Neurosci. 1984;24(2):107–31.

    Google Scholar 

  26. Dyke FB, Leiker AM, Grand KF, Godwin MM, Thompson AG, Rietschel JC, McDonald CG, Miller MW. The efficacy of auditory probes in indexing cognitive workload is dependent on stimulus complexity. Int J Psychophysiol. 2015;95(1):56–62.

    Google Scholar 

  27. Takeda Y, Inoue K, Kimura M, Sato T, Nagai C. Electrophysiological assessment of driving pleasure and difficulty using a task-irrelevant probe technique. Biol Psychol. 2016;120:137–41.

    Google Scholar 

  28. Rösler F, Heil M, Röder B. Slow negative brain potentials as reflections of specific modular resources of cognition. Biol Psychol. 1997;45(1–3):109–41.

    Google Scholar 

  29. Wickens C, Kramer A, Vanasse L, Donchin E. Performance of concurrent tasks: a psychophysiological analysis of the reciprocity of information-processing resources. Science. 1983;221(4615):1080–2.

    Google Scholar 

  30. Horat SK, Herrmann FR, Favre G, Terzis J, Debatisse D, Merlo MC, Missonnier P. Assessment of mental workload: a new electrophysiological method based on intra-block averaging of erp amplitudes. Neuropsychologia. 2016;82:11–7.

    Google Scholar 

  31. Roy RN, Charbonnier S, Campagne A, Bonnet S. Efficient mental workload estimation using task-independent eeg features. J Neural Eng. 2016;13(2): 026019.

    Google Scholar 

  32. Kramer AF, Parasuraman R. Neuroergonomics: applications of neuroscience to human factors. In: Cacioppo JT, Tassinary LG, Berntson GG, editors. Handbook of psychophysiology. Cambridge: Cambridge University Press; 2007. p. 704–22.

    Google Scholar 

  33. Gu H, Yao Q, Chen H, Ding Z, Zhao X, Liu H, Feng Y, Li C, Li X. The effect of mental schema evolution on mental workload measurement: an eeg study with simulated quadrotor uav operation. J Neural Eng. 2022;19(2): 026058.

    Google Scholar 

  34. Cooper GE, Harper RP. The use of pilot rating in the evaluation of aircraft handling qualities. Washington: National Aeronautics and Space Administration; 1969.

    Google Scholar 

  35. Shahid A, Wilkinson K, Marcu S, Shapiro CM. Karolinska sleepiness scale (kss). In: Shahid A, Wilkinson K, Marcu S, Shapiro C, editors. STOP, THAT and one hundred other sleep scales. New York: Springer; 2011. p. 209–10.

    Google Scholar 

  36. O’Reilly JA. Can intensity modulation of the auditory response explain intensity-decrement mismatch negativity? Neurosci Lett. 2021;764:136199.

    Google Scholar 

  37. Daniel E. Noise and hearing loss: a review. J Sch Health. 2007;77(5):225–31.

    Google Scholar 

  38. Jasper HH. The ten-twenty electrode system of the international federation. Electroencephalogr Clin Neurophysiol. 1958;10:370–5.

    Google Scholar 

  39. Iriarte J, Urrestarazu E, Valencia M, Alegre M, Malanda A, Viteri C, Artieda J. Independent component analysis as a tool to eliminate artifacts in eeg: a quantitative study. J Clin Neurophysiol. 2003;20(4):249–57.

    Google Scholar 

  40. Handy TC. Event-related potentials: a methods handbook. Cambridg: MIT press; 2005.

    Google Scholar 

  41. Delorme A, Makeig S. Eeglab: an open source toolbox for analysis of single-trial eeg dynamics including independent component analysis. J Neurosci Methods. 2004;134(1):9–21.

    Google Scholar 

  42. De Winter JC, Gosling SD, Potter J. Comparing the pearson and spearman correlation coefficients across distributions and sample sizes: a tutorial using simulations and empirical data. Psychol Methods. 2016;21(3):273.

    Google Scholar 

  43. Verwey WB, Shea CH, Wright DL. A cognitive framework for explaining serial processing and sequence execution strategies. Psychon Bull Rev. 2015;22(1):54–77.

    Google Scholar 

  44. Isreal JB, Chesney GL, Wickens CD, Donchin E. P300 and tracking difficulty: evidence for multiple resources in dual-task performance. Psychophysiology. 1980;17(3):259–73.

    Google Scholar 

  45. Debener S, Kranczioch C, Herrmann CS, Engel AK. Auditory novelty oddball allows reliable distinction of top-down and bottom-up processes of attention. Int J Psychophysiol. 2002;46(1):77–84.

    Google Scholar 

  46. Boutyline A, Soter LK. Cultural schemas: what they are, how to find them, and what to do once you’ve caught one. Am Sociol Rev. 2021;86(4):728–58.

    Google Scholar 

  47. Bartlett FC, Bartlett FC. Remembering: a study in experimental and social psychology. New York: Cambridge University Press; 1995.

    Google Scholar 

  48. Abrahamse EL, Jiménez L, Verwey WB, Clegg BA. Representing serial action and perception. Psychon Bull Rev. 2010;17(5):603–23.

    Google Scholar 

  49. Verwey WB, Dronkers WJ. Skill in discrete keying sequences is execution rate specific. Psychol Res. 2019;83(2):235–46.

    Google Scholar 

  50. Van Merrienboer JJ, Schuurman JG, De Croock M, Paas F. Redirecting learners’ attention during training: effects on cognitive load, transfer test performance and training efficiency. Learn Instr. 2002;12(1):11–37.

    Google Scholar 

  51. Dehais F, Duprès A, Blum S, Drougard N, Scannella S, Roy RN, Lotte F. Monitoring pilot’s mental workload using erps and spectral power with a six-dry-electrode eeg system in real flight conditions. Sensors. 2019;19(6):1324.

    Google Scholar 

  52. Roy RN, Bonnet S, Charbonnier S, Campagne A. (2015) Enhancing single-trial mental workload estimation through xdawn spatial filtering. In: 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER), IEEE. pp 360–363

  53. Solís-Marcos I, Kircher K. Event-related potentials as indices of mental workload while using an in-vehicle information system. Cogn Technol Work. 2019;21(1):55–67.

    Google Scholar 

  54. Sokhadze EM, Casanova MF, Casanova EL, Lamina E, Kelly DP, Khachidze I. Event-related potentials (erp) in cognitive neuroscience research and applications. NeuroRegulation. 2017;4(1):14.

    Google Scholar 

Download references

Funding

This research was supported by grants from the National Defence Basic Scientific Research Program of China.

Author information

Author notes

  1. Shaodi Wang, Heng Gu have contributed equally to this work.

Authors and Affiliations

  1. State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, 100875, People’s Republic of China

    Shaodi Wang, Heng Gu, Qunli Yao, Chao Yang, Xiaoli Li & Gaoxiang Ouyang

Authors
  1. Shaodi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Heng Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qunli Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaoli Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Gaoxiang Ouyang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by SW, HG, QY, CY, GO and XL; The rst draft of the manuscript was written by SW and all authors commented on previous versions of the manuscript. All authors read and approved the all manuscript.

Corresponding authors

Correspondence to Xiaoli Li or Gaoxiang Ouyang.

Ethics declarations

Conflict of interest

The authors have no relevant fnancial or non-fnancial interests to disclose.

Research involving human and animals rights

Not applicable.

Consent for publication

All authors have checked the manuscript and have agreed to the submission.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, S., Gu, H., Yao, Q. et al. Task-independent auditory probes reveal changes in mental workload during simulated quadrotor UAV training. Health Inf Sci Syst 11, 12 (2023). https://doi.org/10.1007/s13755-023-00213-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13755-023-00213-2

Keywords

Search

Navigation