Abstract
Several industries have sought to identify, mitigate, and predict human errors, as human errors contribute to accidents. We present a new methodology, Procedure Deviation Analysis (PDA), which establishes a human-error taxonomy for repair tasks. Unlike other approaches, PDA defines errors as deviations from the task procedure and considers all operator actions that differ from the procedure to classify and quantify procedure execution accuracy. PDA provides a quantitative measure of accuracy that yields insights for training efficacy and procedure design due to its broader consideration of error. PDA was developed to assess accuracy during repair tasks and to provide comparisons between training techniques. Two main findings of the PDA methodology were the establishment of six deviation modes and the associated application rules. To establish the PDA methodology, eighteen subjects participated in a two-session study and received one of two refresher trainings during the second session. The results indicated that neither refresher training prevented significant performance degradation. The total deviation mode occurrence significantly increased from the first to second session (p = 0.001). The average number of steps also increased (p = 0.036), due to subjects repeating steps. The percent accuracy significantly decreased from 87.07% to 61.07% (p < 0.0001), indicating that knowledge loss from the first to second session that was not adequately addressed in the refresher trainings. Furthermore, PDA highlighted potential areas for improvement in the procedures. PDA provided insights into both training efficacy and procedure quality, two of the key tools used to prevent negative outcomes in high-risk environments.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Celik, M., Cebi, S.: Analytical HFACS for investigating human errors in shipping accidents. Accid. Anal. Prev. 41, 66–75 (2009). https://doi.org/10.1016/j.aap.2008.09.004
Shappell, S., Wiegman, D.: Human error analysis of commerical aviation accidents: application of the human factors analysis and classification system’. Aviat. Space Environ. Med. 72, 1006–1016 (2001)
Kirwan, B., Gibson, H., Kennedy, R., Edmunds, J., Cooksley, G., Umbers, I.: Nuclear action reliability assessment (NARA): a data-based HRA tool. In: Spitzer, C., Schmocker, U., Dang, V.N. (eds.) Probabilistic Safety Assessment and Management, pp. 1206–1211. Springer, London (2004). https://doi.org/10.1007/978-0-85729-410-4_195
Bonrath, E.M., Zevin, B., Dedy, N.J., Grantcharov, T.P.: Error rating tool to identify and analyse technical errors and events in laparoscopic surgery. BJS Br. J. Surg. 100, 1080–1088 (2013). https://doi.org/10.1002/bjs.9168
Stanton, N.A., Salmon, P.M.: Human error taxonomies applied to driving: a generic driver error taxonomy and its implications for intelligent transport systems. Saf. Sci. 47, 227–237 (2009). https://doi.org/10.1016/j.ssci.2008.03.006
De Felice, F., Petrillo, A.: Methodological approach for performing human reliability and error analysis in railway transportation system. Int. J. Eng. Technol. 3, 341–353 (2011)
Garrett, J.W., Teizer, J.: Human factors analysis classification system relating to human error awareness taxonomy in construction safety. J. Constr. Eng. Manag. 135, 754–763 (2009). https://doi.org/10.1061/(ASCE)CO.1943-7862.0000034
Cooper, S.E., Wreathall, J., Thompson, C., Drouin, M., Bley, D.: Knowledge-base for the new human reliability analysis method “A Technique for Human Error Analysis (ATHEANA).” Presented at the International Topical Meeting on Probabilistic Safety Assessment Moving Toward Risk Based Regulation, Park City, UT (US) (1996)
Kim, J.W., Jung, W.: A taxonomy of performance influencing factors for human reliability analysis of emergency tasks. J. Loss Prev. Process Ind. 16, 479–495 (2003). https://doi.org/10.1016/S0950-4230(03)00075-5
Akyuz, E., Celik, M.: Utilisation of cognitive map in modelling human error in marine accident analysis and prevention. Saf. Sci. 70, 19–28 (2014)
Williams, J.C., Bell, J.L.: Consolidation of the error producing conditions used in the human error assessment and reduction technique (HEART). In: Safety and Reliability, pp. 26–76. Taylor & Francis (2015)
Marseguerra, M., Zio, E., Librizzi, M.: Quantitative developments in the cognitive reliability and error analysis method (CREAM) for the assessment of human performance. Ann. Nucl. Energy. 33, 894–910 (2006). https://doi.org/10.1016/j.anucene.2006.05.003
Hollnagel, E.: Cognitive Reliability and Error Analysis Method (CREAM). Elsevier (1998)
van Rutte, P.W.J., Nienhuijs, S.W., Jakimowicz, J.J., van Montfort, G.: Identification of technical errors and hazard zones in sleeve gastrectomy using OCHRA. Surg. Endosc. 31(2), 561–566 (2016). https://doi.org/10.1007/s00464-016-4997-4
Rasmussen, J.: Human error mechanisms in complex work environments. Reliab. Eng. Syst. Saf. 22, 155–167 (1988)
Alexander, T.M.: A case based human reliability assessment using HFACS for complex space operations. J. Space Saf. Eng. 6, 53–59 (2019)
Shappell, S.A., Wiegmann, D.A.: The human factors analysis and classification system–HFACS (2000)
Chang, Y.H.J., Mosleh, A.: Cognitive modeling and dynamic probabilistic simulation of operating crew response to complex system accidents. Part 2: IDAC performance influencing factors model. Reliab. Eng. Syst. Saf. 92, 1014–1040 (2007). https://doi.org/10.1016/j.ress.2006.05.010
Shappell, S.A., Wiegmann, D.A.: HFACS analysis of military and civilian aviation accidents: a North American comparison. In: Proceedings of the Annual Meeting of the International Society of Air Safety Investigators. Gold Coast Australia (2004)
Taib, I.A., McIntosh, A.S., Caponecchia, C., Baysari, M.T.: A review of medical error taxonomies: a human factors perspective. Saf. Sci. 49, 607–615 (2011). https://doi.org/10.1016/j.ssci.2010.12.014
Reason, J.: Human error: models and management. BMJ 320, 768–770 (2000)
Tang, B., Hanna, G.B., Joice, P., Cuschieri, A.: Identification and categorization of technical errors by observational clinical human reliability assessment (OCHRA) during laparoscopic cholecystectomy. Arch. Surg. 139, 1215–1220 (2004)
Bates, D., Mächler, M., Bolker, B., Walker, S.: Fitting linear mixed-effects models using lme4. arXiv Preprint arXiv:14065823 (2014)
Acknowledgements
This work was funded by the National Space Biomedical Research Institute (project HFP03801).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
O’Meara, S., Jenks, K., Stevens, C., Mindock, J., Robinson, S. (2021). Methodology to Quantify Accuracy for Procedure Execution Analysis. In: Harris, D., Li, WC. (eds) Engineering Psychology and Cognitive Ergonomics. HCII 2021. Lecture Notes in Computer Science(), vol 12767. Springer, Cham. https://doi.org/10.1007/978-3-030-77932-0_24
Download citation
DOI: https://doi.org/10.1007/978-3-030-77932-0_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-77931-3
Online ISBN: 978-3-030-77932-0
eBook Packages: Computer ScienceComputer Science (R0)