Abstract
Augmented Reality (AR) is a tool which can be used to improve human-computer interaction in flight operations. The application of AR can facilitate pilots integrating the information from interfaces in the flight deck to analyze various sources of messages simultaneously. There are seventeen subjects aged from 23 to 53 (M = 29.82, SD = 8.93) who have participated in this experiment. Their flight experience ranged from zero flight hours to 3000 flight hours (M = 605.00, SD = 1051.04). Two types of HCI AR design (gesture or voice control checklist) have been compared with traditional paper checklist. The results show that AR gesture control induced the highest perceived workload compared with AR voice checklist and traditional paper checklist. There are lots of complicated cognitive processes and physical movements involved in the AR gesture checklist that induced the highest level of effort and frustration based on NASA-TLX. The AR checklist application has relied on the use of the default HoloLens interactions including cursor movement linked with head movements, Air Tap gesture and Microsoft voice recognition system. The current technological features embedded in the HoloLens device are not certified to be used in the cockpit yet. The improvement in the types of interaction and displays with AR devices could lead to changes in pilot’s perceived workload while interacting with an innovative device. This research demonstrated that AR integrated with voice command has potential of significant benefits to be applied in the flight deck for future flight operation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Dorneich, M.C., Rogers, W., Whitlow, S.D., DeMers, R.: Human performance risks and benefits of adaptive systems on the flight deck. Int. J. Aviat. Psychol. 26(1–2), 15–35 (2016)
Luzik, E., Akmaldinova, A.: Psychological aspects of ensuring flight safety in civil aviation. Aviation 10(1), 25–35 (2006)
Azuma, R.T.: A survey of augmented reality. Teleop. Virtual Environ. 6(4), 355–385 (1997)
Prinzel, L.J, Risser, M.: Head-up displays and attention capture (NASA/TM-2004-213000). NASA Langley Research Center, Hampton, VA, United States (2004)
Barnette, J.F.: Role of head-up display in instrument flight (No. IFC-LR-76-2). Air Force Instrument Flight Center Randolph AFB TX, Dublin, OH, United States (1976)
Newman, R. L.: Improvement of head-up display standards. In: Head-Up Display Design Guide. Appendix, vol. 1. Crew Systems Consultants, Yellow Springs, OH, United States (1987)
Thomas, P., Biswas, P., Langdon, P.: State-of-the-art and future concepts for interaction in aircraft cockpits. In: Antona, M., Stephanidis, C. (eds.) UAHCI 2015. LNCS, vol. 9176, pp. 538–549. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-20681-3_51
Comerford, D., Johnson, W.W.: Potential capabilities in a future, augmented cockpit. Ergon. Design 15(1), 8–13 (2007)
Brandt, S.L., Lachter, J., Battiste, V., Johnson, W.: Pilot situation awareness and its implications for single pilot operations: analysis of a human-in-the-loop study. Procedia Manuf. 3, 3017–3024 (2015)
Tran, T.H., Behrend, F., Fünning, N., Arango, A.: Single pilot operations with AR-glasses using Microsoft HoloLens. In: IEEE/AIAA 37th Digital Avionics Systems Conference, pp. 1–7. IEEE, London (2018)
Kharoufah, H., Murray, J., Baxter, G., Wild, G.: A review of human factors causations in commercial air transport accidents and incidents: from to 2000–2016. Prog. Aerosp. Sci. 99, 1–13 (2018)
Australian Transport Safety Bureau: Loss of control and collision with terrain involving B200 King Air, VH-ZCR at Essendon Airport, Victoria on 21 February 2017 (Investigation No. AO-2017-024). ATSB, Canberra, Australia (2018)
Dismukes, R.K., Berman, B.: Checklists and monitoring in the cockpit: why crucial defenses sometimes fail (NASA/TM-2010-216396, TH-084, ARC-E-DAA-TN1902). NASA Ames Research Center, Moffett Field, CA, United States (2010)
Degani, A., Wiener, E.L.: Human factors of flight-deck checklists: the normal checklist (NCC2-377; RTOP 505-67-41). Ames Research Center, Moffett Field, CA, United States (1991)
Rouse, S., Rouse, W.: Computer-based manuals for procedural information. IEEE Trans. Syst. Man Cybern. 10(8), 506–510 (1980)
Weiser, M.: The computer for the 21st century. Sci. Am. 265(3), 94–104 (1991)
Ishii, H., Ullmer, B.: Tangible bits: towards seamless interfaces between people, bits and atoms. In: Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, pp. 234–241. ACM Press, New York (1997)
Kato, H., Billinghurst, M.: Marker tracking and HMD calibration for a video-based augmented reality conferencing system. In: Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality, pp. 85–94. IEEE, London (1999)
Mine, M.R., Brooks, F.P., Jr., Sequin, C.H.: Moving objects in space: exploiting proprioception in virtual-environment interaction. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, pp. 19–26. ACM Press, New York (1997)
Dipietro, L., Sabatini, A.M., Dario, P.: A survey of glove-based systems and their applications. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 38(4), 461–482 (2008)
Erol, A., Bebis, G., Nicolescu, M., Boyle, R.D., Twombly, X.: Vision-based hand pose estimation: a review. Comput. Vis. Image Underst. 108(1), 52–73 (2007)
Oberweger, M., Lepetit, V.: DeepPrior++: improving fast and accurate 3D hand pose estimation. In: IEEE International Conference on Computer Vision Workshops. IEEE, London (2017)
Stanton, N.A., Plant, K.L., Roberts, A.P., Allison, C.K.: Use of highways in the sky and a virtual pad for landing head up display symbology to enable improved helicopter pilots situation awareness and workload in degraded visual conditions. Ergonomics 62(2), 255–267 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, WC. et al. (2020). Evaluating Pilot’s Perceived Workload on Interacting with Augmented Reality Device in Flight Operations. In: Harris, D., Li, WC. (eds) Engineering Psychology and Cognitive Ergonomics. Cognition and Design. HCII 2020. Lecture Notes in Computer Science(), vol 12187. Springer, Cham. https://doi.org/10.1007/978-3-030-49183-3_26
Download citation
DOI: https://doi.org/10.1007/978-3-030-49183-3_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-49182-6
Online ISBN: 978-3-030-49183-3
eBook Packages: Computer ScienceComputer Science (R0)