Abstract
Inappropriate automation usage is a common cause of incidents in semi-autonomous vehicles. Predicting and understanding the factors influencing this usage is crucial for safety. This study aims to evaluate machine learning models in predicting automation usage from behavioral data; and analyze how workload, environment, performance, and risk influence automation usage for different conditions. An existing dataset from a driving simulator study with 16 participants across four automation conditions (Speed High, Speed Low, Full High, and Full Low) was used. Five machine learning models were trained, using different splitting techniques, to predict automation usage. The input to these models were features related to workload, environment, performance, and risk, pre-processed and optimized to reduce computational time. The best-performing model was used to analyze the impact of each factor on automation usage. Random Forest models consistently demonstrated the highest prediction power, with accuracy exceeding 79% for all conditions, providing a robust foundation for enhancing vehicle safety and optimizing human-automation collaboration. Additionally, factors influencing automation usage ranked: Workload>Environment>Performance>Risk., contrasting with literature on pre-drive intentions to use automation. This study offers insights into real-time prediction of automation usage in semi-autonomous vehicles and quantifies the importance of key factors across different automation conditions. The findings reveal variations in prediction accuracy and factor importance across conditions, providing valuable implications for adaptive automated driving system design. Additionally, the hierarchy of factors influencing automation usage reveals a contrast between real-time decisions and pre-drive intentions, emphasizing the need for adaptive systems in dynamic driving conditions.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability statement
The data that support the findings of this study are available from the DEVCOM Army Research Laboratory but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of DEVCOM Army Research Laboratory. A replication package, with code only, can be found in https://github.com/carlos0983/Automation_Usage_Prediction/.
References
Chiou EK, Lee JD (2021) Trusting automation: designing for responsivity and resilience. Hum Factors 65(1). https://doi.org/10.1177/00187208211009995
Lee J (2008) Review of a Pivotal human factors article: “humans and automation: use, misuse, disuse, abuse”. Hum Factors 50(3):404–410. https://doi.org/10.1518/001872008X288547
Lee J, Moray N (1992) Trust, control strategies and allocation of function in human-machine systems. Ergonomics 35(10):1243–1270. https://doi.org/10.1080/00140139208967392
Dzindolet MT, Peterson SA, Pomranky RA et al (2003) The role of trust in automation reliance. Int J Hum Comput Stud 58(6):697–71. https://doi.org/10.1016/S1071-5819(03)00038-7
Parasuraman R (1997) Humans and automation: Use, misuse, disuse, abuse. Hum Factors 39(2):230–253
Wang S, Li ZI (2019) Exploring the mechanism of crashes with automated vehicles using statistical modeling approaches. PLoS ONE 14(3).https://doi.org/10.1371/journal.pone.0214550
Landgrebe C, Supe J, Schaefer-Kehnert M (2019) The automotive industry can benefit from predictive analytics. ATZproduction worldwide 6(2):40–43. https://doi.org/10.1007/s38312-019-0022-y
Madigan R, Louw T, Dziennus M et al (2016) Acceptance of automated road transport systems (ARTS): An Adaptation of the UTAUT Model. Transp Res Proc 14:2217–222. https://doi.org/10.1016/j.trpro.2016.05.237
Zmud J, Sener IN, Wagner J (2016) Self-driving vehicles: Determinants of adoption and conditions of usage. Transp Res Rec 2565(1):57–6. https://doi.org/10.3141/2565-07
Zhang T, Zeng W, Zhang Y et al (2021) What drives people to use automated vehicles? A meta-analytic review. Accid Anal Prev 159(June):10627. https://doi.org/10.1016/j.aap.2021.106270
Shi X, Wong YD, Chai C et al (2021) An Automated Machine Learning (AutoML) Method of Risk Prediction for Decision-Making of Autonomous Vehicles. IEEE Trans Intell Transp Syst 22(11):7145–7154. https://doi.org/10.1109/TITS.2020.3002419
Mcdonald AD, Lee JD, Madison W et al (2014) Steering in a random forest: ensemble learning for detecting drowsiness-related lane departures. Human Factors 56(5):986–99. https://doi.org/10.1177/0018720813515272
Mcdonald AD, Ferris TK, Wiener TA (2019) Classification of driver distraction: a comprehensive analysis of feature generation, machine learning, and input measures. Hum Factors 62(6):1019–1035. https://doi.org/10.1177/0018720819856454
Gremillion GM, Metcalfe JS, Marathe AR, et al (2016) Analysis of trust in autonomy for convoy operations. In: George T, Dutta AK, Islam MS (eds) Micro- and nanotechnology sensors, systems, and applications VIII, International Society for Optics and Photonics, vol 9836. SPIE, pp 356–365. https://doi.org/10.1117/12.2224009
Guznov S, Lyons J, Nelson A et al (2016) The effects of automation error types on operators’ trust and reliance. Lect Notes Comput Sci (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 9740(1):116–12. https://doi.org/10.1007/978-3-319-39907-2_11
Bustamante C, Rodriguez L, Gremillion GM et al (2022) The impact of automation conditions on reliance dynamics and decision-making. Proc Hum Factors Ergon Soc Annu Meet 66(1):786–790
Madigan R, Louw T, Wilbrink M et al (2017) What influences the decision to use automated public transport? Using UTAUT to understand public acceptance of automated road transport systems. Transp Res F Traffic Psychol Behav 50:55–64. https://doi.org/10.1016/j.trf.2017.07.007
Buckley L, Kaye SA, Pradhan AK (2018) Psychosocial factors associated with intended use of automated vehicles: A simulated driving study. Accid Anal Prev 115:202–20. https://doi.org/10.1016/J.AAP.2018.03.021
Davis FD (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q: Manag Inf Syst 13(3):319–33. https://doi.org/10.2307/249008
Venkatesh V, Morris MG, Davis GB et al (2003) User acceptance of information technology: Toward a unified view. MIS Q: Manag Inf Syst 27(3):425–47. https://doi.org/10.2307/30036540
Lee J, See K (2004) Trust in automation: designing for appropriate reliance. Hum Factors 46(1):50–8. https://doi.org/10.1518/hfes.46.1.50_30392
Azevedo-Sa H, Jayaraman SK, Esterwood CT et al (2021) Real-time estimation of drivers’ trust in automated driving systems. Int J Soc Robot 13(8):1911–1927. https://doi.org/10.1007/s12369-020-00694-1
Deo N, Trivedi MM (2020) Looking at the driver/rider in autonomous vehicles to predict take-over readiness. IEEE Trans Intell Veh 5(1):41–5. https://doi.org/10.1109/TIV.2019.2955364. 1811.06047
Drnec K, Metcalfe JS (2016) Paradigm development for identifying and validating indicators of trust in automation in the operational environment of human automation integration. In: Schmorrow DD, Fidopiastis CM (eds) Foundations of augmented cognition: neuroergonomics and operational neuroscience. Springer International Publishing, Cham, pp 157–167
Neubauer C, Gremillion G, Perelman BS et al (2020) Analysis of facial expressions explain affective state and trust-based decisions during interaction with autonomy. In: Ahram T, Karwowski W, Vergnano A et al (eds) Intelligent human systems integration 2020. Springer International Publishing, Cham, pp 999–1006
Rodriguez L, Bustamante C, Landfair J et al (2021) Dynamics of trust in automation and interactive decision making during driving simulation tasks. Proc Hum Factors Ergon Soc Annu Meet 65(1):786–79. https://doi.org/10.1177/1071181321651288
Rodriguez L, Bustamante C, Gremillion GM et al (2022) Performance and relative risk dynamics during driving simulation tasks under distinct automation conditions. Proc Hum Factors Ergon Soc Annu Meet 66(1):786–790
Van Rossum G, Drake FL (2009) Python 3 Reference Manual. CreateSpace, Scotts Valley, CA
Pedregosa F, Varoquaux G, Gramfort A et al (2011) Scikit-learn: machine learning in python. J Mach Learn Res 12:2825–2830
Harris CR, Millman KJ, van der Walt SJ et al (2020) Array programming with NumPy. Nature 585:357–36. https://doi.org/10.1038/s41586-020-2649-2
Freedman D, Pisani R, Purves R (2007) Statistics (international student edition). Pisani, R Purves, 4th edn WW Norton & Company, New York
Cox DR (1958) The regression analysis of binary sequences. J R Stat Soc Ser B Stat Method 20(2):215–232
Ho TK (1995) Random decision forests. In: Proceedings of 3rd international conference on document analysis and recognition, IEEE, pp 278–282
Ke G, Meng Q, Finley T et al (2017) Lightgbm: A highly efficient gradient boosting decision tree. Adv Neural Inf Process Syst 30:3146–3154
Prokhorenkova LO, Gusev G, Vorobev A et al (2018) Catboost: unbiased boosting with categorical features. In: Bengio S, Wallach HM, Larochelle H, et al (eds) NeurIPS, pp 6639–6649
Chen T, Guestrin C (2016) XGBoost: A scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, USA, KDD ’16, pp 785–794. https://doi.org/10.1145/2939672.2939785
Jex SM (1998) Stress and job performance : theory, research, and implications for managerial practice. Advanced topics in organizational behavior., Sage Publications, Thousand Oaks, Calif
Sitkin SB, Pablo AL (1992) Reconceptualizing the Determinants of Risk Behavior. The Academy of Management Review 17(1): https://doi.org/10.2307/258646
Kotsiantis SB, Zaharakis ID, Pintelas PE (2006) Machine learning: a review of classification and combining techniques. Artif Intell Rev 26:159–19. https://doi.org/10.1007/s10462-007-9052-3
Bergstra J, Bengio Y (2012) Random search for hyper-parameter optimization. J Mach Learn Res 13:281–305
Cover TM, Hart PE (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13(1):21–27. https://doi.org/10.1109/TIT.1967.1053964
Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297
McCulloch WS, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biol 5(4):115–133
Lau N, Fridman L, Borghetti BJ et al (2018) Machine learning and human factors: Status, applications, and future directions. Proc Hum Factors Ergon Soc 1:135–138. https://doi.org/10.1177/1541931218621031
Hannon D, Rantanen E, Sawyer B et al (2019) A human factors engineering education perspective on data science, machine learning and automation. Proc Hum Factors Ergon Soc Annu Meet 63(1):488–49. https://doi.org/10.1177/1071181319631248
Acknowledgements
Research was sponsored by the Army Research Laboratory and was accomplished under Cooperative Agreement Number W911NF-20-2-0252 and the James S. McDonnell Foundation 21st Century Science Initiative in Studying Complex Systems Scholar Award (UHC Scholar Award 220020472). The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Office or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.
Funding
This study was funded by the Army Research Laboratory and was accomplished under Cooperative Agreement Number W911NF-20-2-0252 and the James S. McDonnell Foundation 21st Century Science Initiative in Studying Complex Systems Scholar Award (UHC Scholar Award 220020472).
Author information
Authors and Affiliations
Contributions
Conceptualization: Carlos Bustamante Orellana, Yun Kang, Lucero Rodriguez; Methodology: Carlos Bustamante Orellana, Yun Kang; Formal analysis and investigation: Carlos Bustamante Orellana; Writing - original draft preparation: Carlos Bustamante Orellana; Writing - review and editing: Carlos Bustamante Orellana, Yun Kang, Lucero Rodriguez, Lixiao Huang, Nancy Cooke; Funding acquisition:Yun Kang, Lixiao Huang, Nancy Cooke; Resources: Yun Kang; Supervision: Yun Kang.
Corresponding author
Ethics declarations
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors. The data used in this study was provided by the DEVCOM Army Research Laboratory.
Conflict of Interest
Dr. Yun Kang has received research grants from the Army Research Laboratory under Cooperative Agreement Number W911NF-20-2-0252 and the James S. McDonnell Foundation 21st Century Science Initiative in Studying Complex Systems Scholar Award (UHC Scholar Award 220020472). Dr. Lixiao Huang has received a grant from the Army Research Laboratory under Cooperative Agreement Number W911NF-20-2-0252. Dr. Nancy Cooke has received a grant from the Army Research Laboratory under Cooperative Agreement Number W911NF-20-2-0252. Carlos Bustamante and Lucero Rodriguez declare that they have no conflicts of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix A Feature classification
Appendix B Feature selection results
Appendix C Hyper-parameter tuning
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Bustamante Orellana, C., Rodriguez Rodriguez, L., Huang, L. et al. Machine learning for automation usage prediction: identifying critical factors in driver decision-making. Appl Intell 55, 12 (2025). https://doi.org/10.1007/s10489-024-06052-2
Accepted:
Published:
DOI: https://doi.org/10.1007/s10489-024-06052-2