Abstract
Adapting training in real time can be challenging for instructors. Real-time simulation can present rapid sequences of events, making it difficult for an instructor to attribute errors or omissions to specific underling gaps in skills and knowledge. Monitoring multiple students simultaneously imposes additional attentional workload on an instructor. This challenge can be further exacerbated when an instructor’s view of the student is obscured by virtual reality (VR) equipment. To support instructors’ ability to adapt training, Eduworks and USC’s Institute for Creative Technologies are developing machine learning (ML) models that can measure user engagement during training simulations and offer recommendations for restoring lapses in engagement. We have created a system, called the Observational Motivation and Engagement Generalized Appliance (OMEGA), which we tested in the context of a new U.S. Air Force approach to Specialized Undergraduate Pilot Training (SUPT) called Pilot Training Next (PTN). PTN integrates traditional flying sorties with VR-enabled ground-based training devices to achieve training efficiencies, improve readiness, and increase throughput. The virtual environment provides a rich source of raw data that machine learning models can use to associate user activity with user engagement. We created a testbed for data capture to construct the ML models, based on theoretical foundations we developed previously. Our research explores OMEGA’s potential to help alert an instructor pilot (IP) to student distraction by flagging attention and engagement lapses. Our hypothesis is that OMEGA could help an IP adapt learning, and potentially manage multiple students at the same time, with alerts of lapsed attention and recommendations for restoring engagement. To test this hypothesis, we ran pilots through multiple PTN scenarios to create data for training the model. In this paper, we report on work to create machine learning models using three different techniques, and present model performance data using standard machine learning metrics. We discuss the modeling approach used to generate instructor recommendations. Future work will present results from a formative evaluation using instructor pilots. These early findings provide preliminary validation for the use of ML models for learning to detect engagement from the rich data sources characteristic of virtual environments. These findings will be applicable across a broad range of conventional and VR training applications.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Sottilare, R.A., Goodwin, G.A.: Adaptive instructional methods to accelerate learning and enhance learning capacity. In: International Defense & Homeland Security Simulation Workshop of the I3M Conference (2017)
Hu, P.J.H., Hui, W.: Examining the role of learning engagement in technology-mediated learning and its effects on learning effectiveness and satisfaction. Decis. Support Syst. 53(4), 782–792 (2012)
Chi, M.T., Wylie, R.: The ICAP framework: linking cognitive engagement to active learning outcomes. Educ. Psychol. 49(4), 219–243 (2014)
Core, M.G., Georgila, K., Nye, B.D., Auerbach, D., Liu, Z.F., DiNinni, R.: Learning, adaptive support, student traits, and engagement in scenario-based learning. In: I/ITSEC (2016)
Porter, L.W., Lawler, E.E.: Managerial attitudes and performance. R.D. Irwin, Homewood (1968)
Gawel, J.E.: Herzberg’s theory of motivation and Maslow’s hierarchy of needs. Pract. Assess. Res. Eval. 5(11), 3 (1997)
Pintrich, P.R.: Multiple goals, multiple pathways: the role of goal orientation in learning & achievement. J. Educ. Psych. 92(3), 544 (2000)
D’Mello, S., Graesser, A.: Dynamics of affective states during complex learning. Learn. Instr. 22(2), 145–157 (2012)
Baker, R.S., Corbett, A.T., Roll, I., Koedinger, K.R.: Developing a generalizable detector of when students game the system. User Model. User-Adap. Inter. 18(3), 287–314 (2008)
Bell, B., Kelsey, E., Nye, B. Monitoring engagement and motivation across learning environments. In: Proceedings of the 2019 MODSIM World Conference, Norfolk, VA (2019)
Harrivel, A.R., et al.: Prediction of cognitive states during flight simulation using multimodal psychophysiological sensing. In: AIAA Information Systems-AIAA Infotech, p. 1135 (2017)
Wickens, C.D.: Attentional tunneling and task management. In: 2005 International Symposium on Aviation Psychology, p. 812 (2005)
Cummings, M.L., Mastracchio, C., Thornburg, K.M., Mkrtchyan, A.: Boredom and distraction in multiple unmanned vehicle supervisory control. Interact. Comput. 25(1), 34–47 (2013)
Casner, S.M., Schooler, J.W.: Vigilance impossible: diligence, distraction, and daydreaming all lead to failures in a practical monitoring task. Conscious. Cogn. 35, 33–41 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Bell, B., Bennett, W.W., Nye, B., Kelsey, E. (2021). Helping Instructor Pilots Detect and Respond to Engagement Lapses in Simulations. In: Sottilare, R.A., Schwarz, J. (eds) Adaptive Instructional Systems. Adaptation Strategies and Methods. HCII 2021. Lecture Notes in Computer Science(), vol 12793. Springer, Cham. https://doi.org/10.1007/978-3-030-77873-6_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-77873-6_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-77872-9
Online ISBN: 978-3-030-77873-6
eBook Packages: Computer ScienceComputer Science (R0)