Christian Thomay
ABSTRACT
Obtaining quantifiable information about the behavior, perception, and cognition of aircraft pilots in simulator training processes is a key component in realizing Competence-based Training and Assessment (CBTA) principles. In this paper, we present the FlightAnalyser system, a real-time monitoring and assessment framework that is applicable in any flight simulator environment and even extendable to real-world flight. Using mobile eye tracking technology, it provides a 3D visualization of the trainee’s gaze in their environment, thereby allowing for investigation of gaze distribution and metrics based on dwell times on and transitions between areas of interest. Furthermore, the system provides a real-time estimate of cognitive load via pupil dilation, algorithmically correcting for the effect of environmental light intensity. The system has been deployed at a flight training facility, and has been in use by trainees and instructors, with early results allowing for a validation of the basic functionalities. The aim of the FlightAnalyser is to aid the transition away from ‘hours of experience’ in as the competence metric in flight training. It enables trainees as well as instructors to assess and estimate procedural, behavioral and psychophysiological indicators and progress in the workflow to enable deductions about skill levels, and allows for an individual adaption of training plans both run-time and on the scale of long-term training schedules.
- 2023. Pupil Labs. https://pupil-labs.com/Google Scholar
- A. J. Bayuk. 2008. Aviation safety management systems as a template for aligning safety with business strategy in other industries. In American Society of Safety Engineers-The Business of Safety: A Matter of Success Symposium. Baltimore, Maryland.Google Scholar
- Joan Cahill, Tiziana C. Callari, Florian Fortmann, Stefan Suck, Denis Javaux, Andreas Hasselberg, and Sybert Stroeve. 2018. Adaptive Automation and the Third Pilot. In Aircraft Technology. IntechOpen.Google Scholar
- Kurt Colvin, Rahul Dodhia, and R Dismukes. 2005. Is pilots’ visual scanning adequate to avoid mid-air collisions?. In 2005 International Symposium on Aviation Psychology. 141.Google Scholar
- Steven L. Estes, Kevin J. Burns, John R. Helleberg, Kevin M. Long, Matthew E. Pollack, and Jeffrey L. Stein. 2016. Digital Copilot: Cognitive Assistance for Pilots. Association for the Advancement of Artificial Intelligence (2016).Google Scholar
- European Aviation Safety Agency. 2012. Certification Specifications for Aeroplane Flight Simulation Training Devices.Google Scholar
- Latoya Gibbs, Lisa Slevitch, and Isaac Washburn. 2017. Competency-based training in aviation: The impact on flight attendant performance and passenger satisfaction. Journal of Aviation/Aerospace Education & Research 26, 2 (2017), 55–80.Google Scholar
- Benedikt Gollan and Alois Ferscha. 2016. Modeling pupil dilation as online input for estimation of cognitive load in non-laboratory attention-aware systems. COGNITIVE (2016).Google Scholar
- Bert Hoeks and Willem JM Levelt. 1993. Pupillary dilation as a measure of attention: A quantitative system analysis. Behavior Research Methods, Instruments, & Computers 25, 1 (1993), 16–26.Google ScholarCross Ref
- International Civil Aviation Organisation. 2022. Safety Report. Retrieved January 17, 2023, from https://www.icao.int/safety/Documents/ICAO_SR_2022.pdf.Google Scholar
- Suzanne K Kearns, Timothy J Mavin, and Steven Hodge. 2017. Competency-based education in aviation: Exploring alternate training pathways. Routledge.Google Scholar
- Richard Yeun Chee Kin. 2015. The Impact of Safety Management Systems on Safety Performance: Commercial Aviation Operations. (2015), 213.Google Scholar
- Oliver W Klaproth, Marc Halbrügge, Laurens R Krol, Christoph Vernaleken, Thorsten O Zander, and Nele Russwinkel. 2020. A neuroadaptive cognitive model for dealing with uncertainty in tracing pilots’ cognitive state. Topics in cognitive science 12, 3 (2020), 1012–1029.Google Scholar
- Olivier Lefrancois, Nadine Matton, Yves Gourinat, Vsevolod Peysakhovich, and Mickael Causse. 2016. The role of Pilots’ monitoring strategies in flight performance. (2016).Google Scholar
- Wen-Chin Li, Andreas Horn, Zhen Sun, Jingyi Zhang, and Graham Braithwaite. 2020. Augmented visualization cues on primary flight display facilitating pilot’s monitoring performance. International Journal of Human-Computer Studies 135 (2020), 102377.Google ScholarDigital Library
- Jing Liu, Alessandro Gardi, Subramanian Ramasamy, Yixiang Lim, and Roberto Sabatini. 2016. Cognitive pilot-aircraft interface for single-pilot operations. Knowledge-Based Systems 112 (2016), 37–53.Google ScholarDigital Library
- Christophe Lounis, Vsevolod Peysakhovich, and Mickaël Causse. 2019. Flight eye tracking assistant (feta): Proof of concept. In International conference on applied human factors and ergonomics. Springer, 739–751.Google Scholar
- Christophe Lounis, Vsevolod Peysakhovich, and Mickaël Causse. 2020. Lempel-Ziv Complexity of dwell sequences: visual scanning pattern differences between novice and expert aircraft pilots. In Eye-Tracking in Aviation. Proceedings of the 1st International Workshop (ETAVI 2020). ISAE-SUPAERO, Université de Toulouse; Institute of Cartography and …, 61–68.Google Scholar
- Christophe Lounis, Vsevolod Peysakhovich, and Mickaël Causse. 2021. Visual scanning strategies in the cockpit are modulated by pilots’ expertise: A flight simulator study. PLOS ONE 16, 2 (02 2021), 1–25. https://doi.org/10.1371/journal.pone.0247061Google ScholarCross Ref
- Svetlana Ognjanovic, Manfred Thüring, Ryan O Murphy, and Christoph Hölscher. 2019. Display clutter and its effects on visual attention distribution and financial risk judgment. Applied ergonomics 80 (2019), 168–174.Google Scholar
- Manuele Reani, Niels Peek, and Caroline Jay. 2018. An investigation of the effects of n-gram length in scanpath analysis for eye-tracking research. In Proceedings of the 2018 acm symposium on eye tracking research & applications. 1–8.Google ScholarDigital Library
- David Rudi, Peter Kiefer, and Martin Raubal. 2020. The instructor assistant system (iASSYST)-utilizing eye tracking for commercial aviation training purposes. Ergonomics 63, 1 (2020), 61–79.Google ScholarCross Ref
- Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. 2020. SuperGlue: Learning Feature Matching with Graph Neural Networks. In CVPR. https://arxiv.org/abs/1911.11763Google Scholar
- Brook Shiferaw, Luke Downey, and David Crewther. 2019. A review of gaze entropy as a measure of visual scanning efficiency. Neuroscience & Biobehavioral Reviews 96 (2019), 353–366.Google ScholarCross Ref
- Moritz Stolte, Benedikt Gollan, and Ulrich Ansorge. 2020. Tracking visual search demands and memory load through pupil dilation. Journal of Vision 20, 6 (2020), 21–21.Google ScholarCross Ref
- Gal Ziv. 2016. Gaze behavior and visual attention: A review of eye tracking studies in aviation. The International Journal of Aviation Psychology 26, 3-4 (2016), 75–104.Google ScholarCross Ref
Index Terms
- Modeling Behavior, Perception, and Cognition of Pilots in a Real-time Training Assistance Application
Recommendations
Oculus Rift Application for Training Drone Pilots
SIMUTOOLS '17: Proceedings of the 10th EAI International Conference on Simulation Tools and TechniquesThe research described in this paper, focuses on a virtual reality headset system that integrates the Oculus Rift VR headset with a low cost Unmanned Aerial Vehicle (UAV) to allow for drone teleoperation and telepresence using the Robot Operating System ...
Towards a Semantic Knowledge Base for Competency-Based Training of Airline Pilots
AbstractThe acquisition and maintenance of non-technical skills by the pilots are fundamental factors for the prevention of aviation accidents. The aviation authorities are promoting that air crew training be carried out through simulator sessions using ...
Psychophysiological monitoring as part of the training of Hellenic airforce pilots
CIT'11: Proceedings of the 5th WSEAS international conference on Communications and information technologyPsychophysiological monitoring as part of the training of pilots of the Greek Airforce and the implementation of an auditory identification task for error monitoring are discussed in this paper. The proposed tones-matching task (TMT) allows ...
Comments