Skip to main content
SpringerLink
Log in

Towards eMCO/SiPO: A Human Factors Efficacy, Usability, and Safety Assessment for Direct Voice Input (DVI) Implementation in the Flight Deck

  • Conference paper
  • First Online:
Engineering Psychology and Cognitive Ergonomics (HCII 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14018))

Included in the following conference series:

  • 727 Accesses

Abstract

The recent aviation industry trend towards commercial extended multi-crew operations (eMCO) and single-pilot operations (SiPO) creates the necessity to offload tasks from the future commercial pilot and facilitate their flying duties. The application of DVI should be reconsidered to facilitate pilots’ workload (WL) management on the future flight deck. eMCO/SiPO may not require a complete flight deck redesign at this moment and sound use of existing technology may hold a key role in this transient period. Since there is still not a conclusive accuracy threshold for a DVI to operate in the aviation context, only critical actions were considered in early research. Existing commercially available DVI technology maybe ready to serve some low impact/ non time-critical applications in eMCO/SiPO (e.g., weather at destination and alternate airports update during cruise phase).

The research was conducted in two stages. Initially, a systematic literature review utilised the proven three pilar framework in-depth investigation method. Based on the validation criteria of the viability, operational reliability and functional utility as used by General Dynamics and USAF, the study argued that DVI could be considered reliable and safe for deployment on the flight deck for candidate target applications. Based on the frequency of reference in the reviewed literature and the context relevance, operational reliability could be related to accuracy (or error rate), error types, language used (vocabulary and grammar), aviation environmental conditions, feedback, transparency, adaptivity and transaction time. DVI’s operational viability and reliability are interwoven its contextually applied utility. Based on the frequency of reference in the reviewed literature and thematic analysis, the existing and target applications could form five general categories and five sub-categories. These are a) data entry, with sub-categories FMS entries and frequency tuning, b) system management, sub-categorised into general, display reconfigurations and navigation, c) information retrieval, d) interactive assistant and e) operator monitoring.

A holistic complimentary approach was employed utilizing SMEs, pilots, and human factors experts to perform an efficacy, usability, and safety assessment under the eMCO/SiPO prism. The study narrowed down and prioritize the DVI target applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ruth, J.C., Godwin, A.M., Werkowitz, E.B.: Voice Interactive System Development Program (1982)

    Google Scholar 

  2. Deutsch, S., Pew, R.W.: Single Pilot Commercial Aircraft Operation (2005)

    Google Scholar 

  3. Schmid, D., Stanton, N.A.: Progressing toward airliners’ reduced-crew operations: a systematic literature review. Int. J. Aerosp. Psychol. (2019). https://doi.org/10.1080/24721840.2019.1696196

  4. Dormoy, C., Andre, J.M., Pagani, A.: A human factors’ approach for multimodal collaboration with cognitive computing t. o create a human intelligent machine team: a review. IOP Conf. Ser.: Mater. Sci. Eng. 1024 012105 (2020)

    Google Scholar 

  5. Stanton, N.A., Harris, D.: The future flight deck: modelling dual, single and distributed crewing options. Appl. Ergon. 53, 331–342 (2015). https://doi.org/10.1016/j.apergo.2015.06.019

  6. Saunders, M., Lewis, P., Thornhill, A.: Research Methods for Business Students, 8th edn. Pearson Education Limited (2019)

    Google Scholar 

  7. Steeneken, H.J.M.: Development and performance of a cockpit control system operated by voice: summary report of project DMKLu/AC02/A/9105 (1996)

    Google Scholar 

  8. Schulte, A., Stutz, P.: Evaluation of the cockpit assistant military aircraft CAMA in simulator trials. In: Engineering Psychology and Cognitive Ergonomics,Routledge (1998)

    Google Scholar 

  9. South, A.J.: Voice Recognition in Adverse Aircraft Cockpit Environments (1996)

    Google Scholar 

  10. Wesson, R.B., Pearson, G.M.: Voice-Activated Cockpit for General Aviation (2006)

    Google Scholar 

  11. Gauci, J., Xuereb, M., Muscat, A., Zammit-Mangion, D.: Multi-modal interaction between pilots and avionic systems on-board large commercial aircraft. In: Harris, D. (ed.) EPCE 2017. LNCS (LNAI), vol. 10276, pp. 200–210. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58475-1_15

    Chapter  Google Scholar 

  12. Gosper, S., Trippas, J.R., Richards, H., Allison, F., Sear, C.: Understanding the Utility of Digital Flight Assistants: A Preliminary Analysis (2021)

    Google Scholar 

  13. Cohen, P., Oviatt, S.: The role of voice input for human-machine communication. Natl. Acad. Sci. USA 92, 9921–9927 (1995)

    Article  Google Scholar 

  14. Oviatt, S.: Advances in Robust Multimodal Interface Design (2003)

    Google Scholar 

  15. Hourlier, S., Meehan, J., Ligger, A., Roumes, C.: Relative effectiveness of audio tools for fighter pilots in simulated operational flights: a human factors approach (2005)

    Google Scholar 

  16. Draper, M., Calhoon, G., Ruff, H., Williamson, D., Barry, T.: Manual Versus speech input for unmanned aerial vehicle control station operations (2003)

    Google Scholar 

  17. Cook, M.J., Cranmer, C., Milton, C.-A., Finan, R., Sapeluk, A.: Hidden usability issues in speech interfaces (1996)

    Google Scholar 

  18. Levoulis, S.J., DeLucia, P.R., Kim, S.Y.: Effects of touch, voice, and multimodal input, and task load on multiple-UAV monitoring performance during simulated manned- unmanned teaming in a military helicopter. Hum. Factors 60(8), 1117–1129 (2018)

    Article  Google Scholar 

  19. Beek, B.:Overview of State-of-the-Art, R&D NATO activities, and Possible Applications- Voice Processing Technology (1982)

    Google Scholar 

  20. White, R.W., Parks, D.L., Smith, W.D.: Potential flight applications for voice recognition and synthesis systems (1984)

    Google Scholar 

  21. Weinstein, C.J.: Opportunities for advanced speech processing in military computer-based systems. IEEE 79(11), 1626–1641 (1991). https://doi.org/10.1109/5.118986

    Article  Google Scholar 

  22. Herdman, C.M., Johannsdottir, K.R., Lessard, L., Jarmasz, J., Churchill, L.L.: Attentional benefits and costs associated with integrating a direct voice input (DVI) system into a multi-crew helicopter environment (2001)

    Google Scholar 

  23. Zon, G.D.R., Roerdink, M.I.: Using Voice to Control the Civil Flightdeck (2007)

    Google Scholar 

  24. Ward, K.A.: speech interfaces and pilot performance: a meta-analysis. Int. J. Aviat. Aeronaut. Aerosp. 6(1) (2019). https://doi.org/10.15394/ijaaa.2019.1305

  25. Bell, R., Benett, M., Brown, W.: Direct voice input for the cockpit environment (1982)

    Google Scholar 

  26. Cooke, N.: RAE bedford’s experience of using direct voice input (DVI) in the cockpit (1990)

    Google Scholar 

  27. Chen, F., Jonsson, I.M., Villing, J., Larsson, S.: Application of speech technology in vehicles. In: Chen, F., Jokinen, K. (eds.) Speech Technology. Springer, New York (2010). https://doi.org/10.1007/978-0-387-73819-2_11

  28. Williamson, D.T., Barry, T.P., Ligget, K.K.: Flight Test Results of ITT VRS-1290 in NASA OV-10. Wright-Patterson AFB (1996)

    Google Scholar 

  29. Arthur, J.J., Shelton, K., Prinzel, L.J., Bailey, R.E.: Performance evaluation of speech recognition systems as a next-generation pilot-vehicle interface technology (2016)

    Google Scholar 

  30. Gurluk, H., Helmke, H., Wies, M., Ehr, H., Kleinert, M.: Assistant Based Speech Recognition - Another Pair of Eyes for the Arrival Manager (2015)

    Google Scholar 

  31. L. Wei, L. He, and Y. Liu, “Study of Artificial Intelligence Flight Co-Pilot Speech Recognition Technology (2020). https://doi.org/10.1109/ICCASIT50869.2020.9368869

  32. Trzos, M., Dostl, M., Machkov, P., Eitlerov, J.: Voice control in a real flight deck environment. In: Sojka, P., Horák, A., Kopeček, I., Pala, K. (eds.) TSD 2018. LNCS (LNAI), vol. 11107, pp. 388–402. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00794-2_42

    Chapter  Google Scholar 

  33. D. W. Beeks, “Speech Recognition and Synthesis,” in The Avionics Handbook, C. R. Spitzer, Ed. CRC Press, Taylor & Francis Group (2001)

    Google Scholar 

  34. H. J. M. Steeneken and E. W. Pijpers, “Development and performance of a cockpit control system operated by voice (1996)

    Google Scholar 

  35. G. Calhoon, P. Le Blaye, and H. Welsch, “Pilot-Vehicle Interface (2001)

    Google Scholar 

  36. T. B. Sheridan and R. Parasuraman, “Human-Automation Interaction,” Rev. Hum. Factors Ergon. 1(1) (2005)

    Google Scholar 

  37. G. Katz, C. Ding, and A. Doyle, “NASA Pilot-Engaged Expert Response using IBM Watson Technology Prototype Evaluation of Knowledge Retrieval System,” 2018

    Google Scholar 

  38. F. Simon, A. Brock, and M. Causse, “Designing a virtual cognitive assistant for pilots: a user-centered approach,” 2020

    Google Scholar 

  39. G. Zellou, M. Cohn, and B. Ferenc Segedin, “Age- and Gender-Related Differences in Speech Alignment Toward Humans and Voice-AI,” Front. Commun., 2021. https://doi.org/10.3389/fcomm.2020.600361

  40. A. de Barcelos Silva, M. M. Gomes, C. A. da Costa, and R. da Rosa Righi, “Intelligent personal assistants: A systematic literature review,” Expert Syst. Appl., 147 (2020). https://doi.org/10.1016/j.eswa.2020.113193

  41. A. Badr and A. K. Abdul-Hassan, “A Review on Voice-based Interface for Human-Robot Interaction,” Iraqi J. Electr. Electron. Eng., 2020. https://doi.org/10.37917/ijeee.16.2.10

  42. F. Weng, P. Angkititrakul, E. Shriberg, L. Heck, S. Peters, J. Hansen, “Conversational In-Vehicle Dialog Systems,” IEEE SIGNAL PROCESSING MAGAZINE (2016)

    Google Scholar 

  43. E. Matheson, R. Minto, E. Zampieri, M. Faccio, and G. Rosati, “Human–Robot Collaboration in Manufacturing Applications: A Review,” Robotics 8(100) (2019). https://doi.org/10.3390/robotics8040100

  44. D. Harris, N. A. Stanton, A. Marshall, M. S. Young, J. Demagalski, and P. Salmon, “Using SHERPA to predict design-induced error on the flight deck,” Aerosp. Sci. Technol., 2005. https://doi.org/10.1016/j.ast.2005.04.002

  45. H. P. Greeley, E. Friets, J. P. Wilson, S. Raghavan, J. Picone, and J. Berg, “Detecting Fatigue From Voice Using Speech Recognition,” 2006

    Google Scholar 

  46. C. D. Wickens, J. Lee, Y. Liu, and S. Gordon Becker, Introduction to human factors engineering. 2014

    Google Scholar 

  47. G. Salvendy, Handbook of Human Factors and Ergonomics: Fourth Edition. 2012

    Google Scholar 

  48. U.K. MoD, “JSP 912 Part 2: Directive (Human Factors Integration for Defence Systems),” Jt. Serv. Publ. 912 Part 2, 2015

    Google Scholar 

  49. Spitzer, C.R.: The Avionics Handbook. CRC Press, Taylor & Francis Group (2011)

    Google Scholar 

  50. Endsley, M.R.: Toward a theory of situation awareness in dynamic systems. Hum. Factors 37(1), 32–64 (1995). https://doi.org/10.1518/001872095779049543

    Article  Google Scholar 

  51. Y. Zhang, Y. Sun, and Y. Chen, “A Framework for Ergonomics Design of Transport Category Airplane Cockpit,” 2013

    Google Scholar 

  52. FAA, “Electronic Flight Displays (AC 25–11B),” 2014

    Google Scholar 

  53. FAA, “Human Factors Design Standard (HF-STD-001B),” 2016

    Google Scholar 

  54. A. Perott, N. Schader, J. Leonhardt, and T. Licu, “Human Factors Integration in ATM System Design,” 2020

    Google Scholar 

  55. G. Singer, “Methods for Validating Cockpit Design The best tool for the task,” Royal Institute of Technology, Stockholm, Sweden, 2002

    Google Scholar 

  56. M. Yeh, C. Swider, Y. Jin Jo, C. Donovan, and J. A. Volpe, “Human Factors Considerations in the Design and Evaluation of Flight Deck Displays and Controls Version 2.0,” 2016

    Google Scholar 

  57. N. A. Stanton, P. M. Salmon, G. H. Walker, C. Baber, and D. P. Jenkins, Human factors methods: A practical guide for engineering and design. Ashgate Publishing Ltd, 2005

    Google Scholar 

  58. N. A. Stanton et al., “Predicting design induced pilot error using HET (human error template) - A new formal human error identification method for flight decks,” Aeronautical Journal. 2006. https://doi.org/10.1017/S0001924000001056

Download references

Author information

Authors and Affiliations

  1. School of Aviation and Transportation Technology, Purdue University, West Lafayette, IN, 47907, USA

    Dimitrios Ziakkas

  2. Faculty of Engineering, Environment & Computing, Coventry University, Coventry, CV1 5FB, UK

    Don Harris & Konstantinos Pechlivanis

Authors
  1. Dimitrios Ziakkas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Don Harris
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Konstantinos Pechlivanis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dimitrios Ziakkas .

Editor information

Editors and Affiliations

  1. Coventry University, Coventry, UK

    Don Harris

  2. Cranfield University, Cranfield, UK

    Wen-Chin Li

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ziakkas, D., Harris, D., Pechlivanis, K. (2023). Towards eMCO/SiPO: A Human Factors Efficacy, Usability, and Safety Assessment for Direct Voice Input (DVI) Implementation in the Flight Deck. In: Harris, D., Li, WC. (eds) Engineering Psychology and Cognitive Ergonomics. HCII 2023. Lecture Notes in Computer Science(), vol 14018. Springer, Cham. https://doi.org/10.1007/978-3-031-35389-5_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-35389-5_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-35388-8

  • Online ISBN: 978-3-031-35389-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation