Skip to main content

Advertisement

Machine learning for automation usage prediction: identifying critical factors in driver decision-making

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Inappropriate automation usage is a common cause of incidents in semi-autonomous vehicles. Predicting and understanding the factors influencing this usage is crucial for safety. This study aims to evaluate machine learning models in predicting automation usage from behavioral data; and analyze how workload, environment, performance, and risk influence automation usage for different conditions. An existing dataset from a driving simulator study with 16 participants across four automation conditions (Speed High, Speed Low, Full High, and Full Low) was used. Five machine learning models were trained, using different splitting techniques, to predict automation usage. The input to these models were features related to workload, environment, performance, and risk, pre-processed and optimized to reduce computational time. The best-performing model was used to analyze the impact of each factor on automation usage. Random Forest models consistently demonstrated the highest prediction power, with accuracy exceeding 79% for all conditions, providing a robust foundation for enhancing vehicle safety and optimizing human-automation collaboration. Additionally, factors influencing automation usage ranked: Workload>Environment>Performance>Risk., contrasting with literature on pre-drive intentions to use automation. This study offers insights into real-time prediction of automation usage in semi-autonomous vehicles and quantifies the importance of key factors across different automation conditions. The findings reveal variations in prediction accuracy and factor importance across conditions, providing valuable implications for adaptive automated driving system design. Additionally, the hierarchy of factors influencing automation usage reveals a contrast between real-time decisions and pre-drive intentions, emphasizing the need for adaptive systems in dynamic driving conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability statement

The data that support the findings of this study are available from the DEVCOM Army Research Laboratory but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of DEVCOM Army Research Laboratory. A replication package, with code only, can be found in https://github.com/carlos0983/Automation_Usage_Prediction/.

References

  1. Chiou EK, Lee JD (2021) Trusting automation: designing for responsivity and resilience. Hum Factors 65(1). https://doi.org/10.1177/00187208211009995

  2. Lee J (2008) Review of a Pivotal human factors article: “humans and automation: use, misuse, disuse, abuse”. Hum Factors 50(3):404–410. https://doi.org/10.1518/001872008X288547

  3. Lee J, Moray N (1992) Trust, control strategies and allocation of function in human-machine systems. Ergonomics 35(10):1243–1270. https://doi.org/10.1080/00140139208967392

    Article  MATH  Google Scholar 

  4. Dzindolet MT, Peterson SA, Pomranky RA et al (2003) The role of trust in automation reliance. Int J Hum Comput Stud 58(6):697–71. https://doi.org/10.1016/S1071-5819(03)00038-7

    Article  Google Scholar 

  5. Parasuraman R (1997) Humans and automation: Use, misuse, disuse, abuse. Hum Factors 39(2):230–253

    Article  MATH  Google Scholar 

  6. Wang S, Li ZI (2019) Exploring the mechanism of crashes with automated vehicles using statistical modeling approaches. PLoS ONE 14(3).https://doi.org/10.1371/journal.pone.0214550

  7. Landgrebe C, Supe J, Schaefer-Kehnert M (2019) The automotive industry can benefit from predictive analytics. ATZproduction worldwide 6(2):40–43. https://doi.org/10.1007/s38312-019-0022-y

    Article  Google Scholar 

  8. Madigan R, Louw T, Dziennus M et al (2016) Acceptance of automated road transport systems (ARTS): An Adaptation of the UTAUT Model. Transp Res Proc 14:2217–222. https://doi.org/10.1016/j.trpro.2016.05.237

    Article  Google Scholar 

  9. Zmud J, Sener IN, Wagner J (2016) Self-driving vehicles: Determinants of adoption and conditions of usage. Transp Res Rec 2565(1):57–6. https://doi.org/10.3141/2565-07

    Article  MATH  Google Scholar 

  10. Zhang T, Zeng W, Zhang Y et al (2021) What drives people to use automated vehicles? A meta-analytic review. Accid Anal Prev 159(June):10627. https://doi.org/10.1016/j.aap.2021.106270

    Article  MATH  Google Scholar 

  11. Shi X, Wong YD, Chai C et al (2021) An Automated Machine Learning (AutoML) Method of Risk Prediction for Decision-Making of Autonomous Vehicles. IEEE Trans Intell Transp Syst 22(11):7145–7154. https://doi.org/10.1109/TITS.2020.3002419

    Article  MATH  Google Scholar 

  12. Mcdonald AD, Lee JD, Madison W et al (2014) Steering in a random forest: ensemble learning for detecting drowsiness-related lane departures. Human Factors 56(5):986–99. https://doi.org/10.1177/0018720813515272

    Article  MATH  Google Scholar 

  13. Mcdonald AD, Ferris TK, Wiener TA (2019) Classification of driver distraction: a comprehensive analysis of feature generation, machine learning, and input measures. Hum Factors 62(6):1019–1035. https://doi.org/10.1177/0018720819856454

    Article  MATH  Google Scholar 

  14. Gremillion GM, Metcalfe JS, Marathe AR, et al (2016) Analysis of trust in autonomy for convoy operations. In: George T, Dutta AK, Islam MS (eds) Micro- and nanotechnology sensors, systems, and applications VIII, International Society for Optics and Photonics, vol 9836. SPIE, pp 356–365. https://doi.org/10.1117/12.2224009

  15. Guznov S, Lyons J, Nelson A et al (2016) The effects of automation error types on operators’ trust and reliance. Lect Notes Comput Sci (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 9740(1):116–12. https://doi.org/10.1007/978-3-319-39907-2_11

    Article  MATH  Google Scholar 

  16. Bustamante C, Rodriguez L, Gremillion GM et al (2022) The impact of automation conditions on reliance dynamics and decision-making. Proc Hum Factors Ergon Soc Annu Meet 66(1):786–790

    MATH  Google Scholar 

  17. Madigan R, Louw T, Wilbrink M et al (2017) What influences the decision to use automated public transport? Using UTAUT to understand public acceptance of automated road transport systems. Transp Res F Traffic Psychol Behav 50:55–64. https://doi.org/10.1016/j.trf.2017.07.007

    Article  Google Scholar 

  18. Buckley L, Kaye SA, Pradhan AK (2018) Psychosocial factors associated with intended use of automated vehicles: A simulated driving study. Accid Anal Prev 115:202–20. https://doi.org/10.1016/J.AAP.2018.03.021

    Article  Google Scholar 

  19. Davis FD (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q: Manag Inf Syst 13(3):319–33. https://doi.org/10.2307/249008

    Article  MATH  Google Scholar 

  20. Venkatesh V, Morris MG, Davis GB et al (2003) User acceptance of information technology: Toward a unified view. MIS Q: Manag Inf Syst 27(3):425–47. https://doi.org/10.2307/30036540

    Article  MATH  Google Scholar 

  21. Lee J, See K (2004) Trust in automation: designing for appropriate reliance. Hum Factors 46(1):50–8. https://doi.org/10.1518/hfes.46.1.50_30392

    Article  MATH  Google Scholar 

  22. Azevedo-Sa H, Jayaraman SK, Esterwood CT et al (2021) Real-time estimation of drivers’ trust in automated driving systems. Int J Soc Robot 13(8):1911–1927. https://doi.org/10.1007/s12369-020-00694-1

    Article  MATH  Google Scholar 

  23. Deo N, Trivedi MM (2020) Looking at the driver/rider in autonomous vehicles to predict take-over readiness. IEEE Trans Intell Veh 5(1):41–5. https://doi.org/10.1109/TIV.2019.2955364. 1811.06047

  24. Drnec K, Metcalfe JS (2016) Paradigm development for identifying and validating indicators of trust in automation in the operational environment of human automation integration. In: Schmorrow DD, Fidopiastis CM (eds) Foundations of augmented cognition: neuroergonomics and operational neuroscience. Springer International Publishing, Cham, pp 157–167

    MATH  Google Scholar 

  25. Neubauer C, Gremillion G, Perelman BS et al (2020) Analysis of facial expressions explain affective state and trust-based decisions during interaction with autonomy. In: Ahram T, Karwowski W, Vergnano A et al (eds) Intelligent human systems integration 2020. Springer International Publishing, Cham, pp 999–1006

    Chapter  MATH  Google Scholar 

  26. Rodriguez L, Bustamante C, Landfair J et al (2021) Dynamics of trust in automation and interactive decision making during driving simulation tasks. Proc Hum Factors Ergon Soc Annu Meet 65(1):786–79. https://doi.org/10.1177/1071181321651288

    Article  MATH  Google Scholar 

  27. Rodriguez L, Bustamante C, Gremillion GM et al (2022) Performance and relative risk dynamics during driving simulation tasks under distinct automation conditions. Proc Hum Factors Ergon Soc Annu Meet 66(1):786–790

    MATH  Google Scholar 

  28. Van Rossum G, Drake FL (2009) Python 3 Reference Manual. CreateSpace, Scotts Valley, CA

    MATH  Google Scholar 

  29. Pedregosa F, Varoquaux G, Gramfort A et al (2011) Scikit-learn: machine learning in python. J Mach Learn Res 12:2825–2830

    MathSciNet  MATH  Google Scholar 

  30. Harris CR, Millman KJ, van der Walt SJ et al (2020) Array programming with NumPy. Nature 585:357–36. https://doi.org/10.1038/s41586-020-2649-2

    Article  MATH  Google Scholar 

  31. Freedman D, Pisani R, Purves R (2007) Statistics (international student edition). Pisani, R Purves, 4th edn WW Norton & Company, New York

  32. Cox DR (1958) The regression analysis of binary sequences. J R Stat Soc Ser B Stat Method 20(2):215–232

    Article  MathSciNet  MATH  Google Scholar 

  33. Ho TK (1995) Random decision forests. In: Proceedings of 3rd international conference on document analysis and recognition, IEEE, pp 278–282

  34. Ke G, Meng Q, Finley T et al (2017) Lightgbm: A highly efficient gradient boosting decision tree. Adv Neural Inf Process Syst 30:3146–3154

    MATH  Google Scholar 

  35. Prokhorenkova LO, Gusev G, Vorobev A et al (2018) Catboost: unbiased boosting with categorical features. In: Bengio S, Wallach HM, Larochelle H, et al (eds) NeurIPS, pp 6639–6649

  36. Chen T, Guestrin C (2016) XGBoost: A scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, USA, KDD ’16, pp 785–794. https://doi.org/10.1145/2939672.2939785

  37. Jex SM (1998) Stress and job performance : theory, research, and implications for managerial practice. Advanced topics in organizational behavior., Sage Publications, Thousand Oaks, Calif

  38. Sitkin SB, Pablo AL (1992) Reconceptualizing the Determinants of Risk Behavior. The Academy of Management Review 17(1): https://doi.org/10.2307/258646

  39. Kotsiantis SB, Zaharakis ID, Pintelas PE (2006) Machine learning: a review of classification and combining techniques. Artif Intell Rev 26:159–19. https://doi.org/10.1007/s10462-007-9052-3

    Article  MATH  Google Scholar 

  40. Bergstra J, Bengio Y (2012) Random search for hyper-parameter optimization. J Mach Learn Res 13:281–305

    MathSciNet  MATH  Google Scholar 

  41. Cover TM, Hart PE (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13(1):21–27. https://doi.org/10.1109/TIT.1967.1053964

    Article  MATH  Google Scholar 

  42. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    Article  MATH  Google Scholar 

  43. McCulloch WS, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biol 5(4):115–133

    MathSciNet  MATH  Google Scholar 

  44. Lau N, Fridman L, Borghetti BJ et al (2018) Machine learning and human factors: Status, applications, and future directions. Proc Hum Factors Ergon Soc 1:135–138. https://doi.org/10.1177/1541931218621031

  45. Hannon D, Rantanen E, Sawyer B et al (2019) A human factors engineering education perspective on data science, machine learning and automation. Proc Hum Factors Ergon Soc Annu Meet 63(1):488–49. https://doi.org/10.1177/1071181319631248

    Article  MATH  Google Scholar 

Download references

Acknowledgements

Research was sponsored by the Army Research Laboratory and was accomplished under Cooperative Agreement Number W911NF-20-2-0252 and the James S. McDonnell Foundation 21st Century Science Initiative in Studying Complex Systems Scholar Award (UHC Scholar Award 220020472). The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Office or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.

Funding

This study was funded by the Army Research Laboratory and was accomplished under Cooperative Agreement Number W911NF-20-2-0252 and the James S. McDonnell Foundation 21st Century Science Initiative in Studying Complex Systems Scholar Award (UHC Scholar Award 220020472).

Author information

Authors and Affiliations

Authors

Contributions

Conceptualization: Carlos Bustamante Orellana, Yun Kang, Lucero Rodriguez; Methodology: Carlos Bustamante Orellana, Yun Kang; Formal analysis and investigation: Carlos Bustamante Orellana; Writing - original draft preparation: Carlos Bustamante Orellana; Writing - review and editing: Carlos Bustamante Orellana, Yun Kang, Lucero Rodriguez, Lixiao Huang, Nancy Cooke; Funding acquisition:Yun Kang, Lixiao Huang, Nancy Cooke; Resources: Yun Kang; Supervision: Yun Kang.

Corresponding author

Correspondence to Yun Kang.

Ethics declarations

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors. The data used in this study was provided by the DEVCOM Army Research Laboratory.

Conflict of Interest

Dr. Yun Kang has received research grants from the Army Research Laboratory under Cooperative Agreement Number W911NF-20-2-0252 and the James S. McDonnell Foundation 21st Century Science Initiative in Studying Complex Systems Scholar Award (UHC Scholar Award 220020472). Dr. Lixiao Huang has received a grant from the Army Research Laboratory under Cooperative Agreement Number W911NF-20-2-0252. Dr. Nancy Cooke has received a grant from the Army Research Laboratory under Cooperative Agreement Number W911NF-20-2-0252. Carlos Bustamante and Lucero Rodriguez declare that they have no conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix A Feature classification

Table 6 Description of features and their category/factor classification

Appendix B Feature selection results

Table 7 Average accuracy and AUC of each classification model when trained with the top 20 and all the features available

Appendix C Hyper-parameter tuning

Table 8 Description of parameters for the different machine learning classifiers used in this study
Table 9 Best parameter values for the machine learning classifiers explored in the study

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bustamante Orellana, C., Rodriguez Rodriguez, L., Huang, L. et al. Machine learning for automation usage prediction: identifying critical factors in driver decision-making. Appl Intell 55, 12 (2025). https://doi.org/10.1007/s10489-024-06052-2

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-06052-2

Keywords