Skip to main content

Advertisement

SpringerLink
Log in

Human performance modeling and its uncertainty factors affecting decision making: a survery

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

This paper introduces the background and connotation of human performance modeling (HPM), HPM models, and the application of artificial intelligence algorithms in HPM. It deeply analyzes the connotation and uncertainty of each model and finally puts forward its military application. The aim is to provide relevant researchers in the field with an in-depth understanding of domain knowledge and related uncertainties and to indicate future research directions. The first part is a general overview of human factors engineering, where the definition, origin, research field, importance, and general problems of HPM are elaborated. The composition of the man–machine system and its corresponding relationship with the observe–orient–decide–act loop are described. The second part reviews the models of perception, cognition, understanding, and decision making. Among them, models of cognition consist of visual search, visual sampling, mental workload, and goals, operators, methods, and selection rules; models of action consist of Hick–Hyman law, Fitts’s law, and manual control theory. The third part is a review of the source and importance of the integrated models and focuses on the principles, composition, and successful application cases of the three models, namely SAINT, IMPRINT, and ACT-R. The fourth part is a review of the application of the algorithms and models in the fields of artificial intelligence, deep learning, and data mining in analyzing multivariate datasets in HPM. In addition, future HPM military applications are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  • Anderson JR, Lebiere CJ (1998) The atomic components of thought. Psychology Press, New York

    Google Scholar 

  • Anderson JR, Bothell D, Byrne MD, Douglass S, Lebiere C, Qin Y (2004) An integrated theory of the mind. Psychol Rev 111:1036–1060

    Google Scholar 

  • Andre AD, Wickens CD (1995) When users want what’s not best for them. Ergon Des 3:10–14

    Google Scholar 

  • Archer S, Headley D, Allender L (2003) Manpower, personnel, and training integration methods and tools. In: Booher HR (ed) Handbook of human systems integration. Wiley, New York, pp 379–431

    Google Scholar 

  • Archer S, Adkins R (1999) IMPRINT User’s Guide prepared for US Army Research Laboratory. Human Research and Engineering Directorate

  • Back MD, Nestler S (2016) Accuracy of judging personality. In: Hall JA, Mast MS, West TV (eds) The social psychology of perceiving others accurately. Cambridge University Press, Cambridge, pp 98–124

    Google Scholar 

  • Best BJ, Lebiere C, Scarpinatto KC (2002) Modeling synthetic opponents in MOUT training simulations using the ACT-R cognitive architecture. In: Proceedings of the 11th conference on computer generated forces and behavioral. Represent

  • Bisantz AM, Kirlik A, Gay P, Phipps DA, Walker N, Fisk AD (2000) Modeling and analysis of a dynamic judgment task using a lens model approach. In: IEEE Transactions on Systems, Man, and Cybernetics, System A, vol 30, pp 605–616, Nov 2000

  • Bisantz AM, Pritchett AR (2003) Measuring the fit between human judgments and automated alerting algorithms: a study of collision detection. Hum Factors 245:266–280

    Google Scholar 

  • Boff KR, Kaufman L, Thomas JP (1994) Handbook of perception and human performance. Cognitive Processes and Performance, vol 2. Wiley, New York

    Google Scholar 

  • Boyd J (2018) A discourse on winning and losing. Maxerll AFB. Air University Press, Alabama, Curtis E. LeMay Center for Doctrine Development and Education

  • Brunswik E (1952) The conceptual framework of psychology. Psychol Bull 49:654–656

    Google Scholar 

  • Byrne MD, Pew RW (2009) A history and primer of human performance modeling. Rev. Hum. Factors Ergon. 5:225–263

    Google Scholar 

  • Cao S, Liu Y (2013) Queueing network-adaptive control of thought rational (QN-ACTR): an integrated cognitive architecture for modelling complex cognitive and multi-task performance. Int J Hum Factors Model Simul 4(1):63–86

    Google Scholar 

  • Cao S, Liu Y (2011) Integrating Queueing Network and ACT-R cognitive architectures. In: Proceedings of the human factors and ergonomics society annual meeting. Sage: Los Angeles, September 2011, vol 55, No 1, pp 836–840

    Google Scholar 

  • Cao S, Liu Y (2011, September) Mental workload modeling in an integrated cognitive architecture. In: Proceedings of the human factors and ergonomics society annual meeting. SAGE Publications, Los Angeles, CA. vol 55, No 1, pp 2083–2087

    Google Scholar 

  • Cao S, Liu Y (2012) QN-ACTR modeling of multitask performance of dynamic and complex cognitive tasks. In: Proceedings of the human factors and ergonomics society annual meeting, Sept 2012. SAGE Publications, Los Angeles, CA, vol 56, No 1, pp 1015–1019

    Google Scholar 

  • Card SK, English WK, Burr BJ (1978) Evaluation of mouse, rate-controlled isometric joystick, step keys, and text keys for text selection on a CRT. Ergonomics 21:601–613

    Google Scholar 

  • Card SK, Moran TP, Newell A (1980) The keystroke-level model for user performance time with interactive systems. Commun ACM 23:396–410

    Google Scholar 

  • Card SK, Moran TP, Newell A (1983) The psychology of human-computer interaction. CRC Press, Boca Raton, pp 139–192

    Google Scholar 

  • Carolan T, Scott-Nash S, Corker K, Kellmeyer D (2000) An application of human performance modeling to the evaluation of advanced user interface features. In: Proceedings of the human factors ergonomics society annual meeting, pp 650–653

    Google Scholar 

  • Cockburn A, Gutwin C, Greenberg S (2007) A predictive model of menu performance. In: Proceedings of the SIGCHI conference on human factors in computing systems, San Jose, CA, pp 627–636

  • Davies DR, Parasuraman R (1982) The psychology of vigilance. Academic Press, New York

    Google Scholar 

  • Dawes RM, Corrigan B (1974) Linear models in decision making. Psychol Bull 81:95–106

    Google Scholar 

  • Deneulin S (2009) An introduction to the human development and capability approach: freedom and agency. International Development Research Centre (IDRC), Ottawa

    Google Scholar 

  • Eisma YB, Cabrall CD, de Winter JC (2018) Visual sampling processes revisited: replicating and extending Senders (1983) using modern eye-tracking equipment. IEEE Trans Hum Mach Syst 48:526–540

    Google Scholar 

  • El Lahib M, Tekli J, Issa YB (2018) Evaluating Fitts’ law on vibrating touch-screen to improve visual data accessibility for blind users. Int J Hum Comput Stud 112:16–27

    Google Scholar 

  • Everitt B, Skrondal A (2010) Standardized Mortality Rate (SMR). The Cambridge Dictionary of Statistics, Cambridge

    Google Scholar 

  • Fisher DL, Coury BG, Tengs TO, Duffy SA (1989) Minimizing the time to search visual displays: the role of highlighting. Hum Factors 31:167–182

    Google Scholar 

  • Fitts PM (1954) The information capacity of the human motor system in controlling the amplitude of movement. J Exp Psychol 47:381–391

    Google Scholar 

  • Fong A, Sibley C, Cole A, Baldwin C, Coyne J (2010) A comparison of artificial neural networks, logistic regressions, and classification trees for modeling mental workload in real-time. In Proceedings of the human factors ergonomics society annual meeting, pp 1709–1712

    Google Scholar 

  • Foyle DC, Andre AD, McCann RS, Wenzel EM, Begault DR, Battiste V (1996) Taxiway Navigation and Situation Awareness (T-NASA) system: problem, design philosophy, and description of an integrated display suite for low-visibility airport surface operations, SAE Technical Paper, Ari. no. 965551

  • Foyle DC, Hooey BL (2007) Human performance modeling in aviation. CRC Press, Boca Raton

    Google Scholar 

  • Gong R, Kieras D (1994) A validation of the GOMS model methodology in the development of a specialized, commercial software application. In: Proceedings of the SIGCHI conference on human factors in computing systems, Boston, MA, USA, pp 351–357

  • Gore BF (2002) Human performance cognitive-behavioral modeling: a benefit for occupational safety. Int J Occup Saf Ergon 8:339–351

    Google Scholar 

  • Gorman JC, Martin MJ, Dunbar TA, Stevens RH, Galloway TL, Amazeen PG, Likens AD (2016) Cross-level effects between neurophysiology and communication during team training. Hum Factors 58:181–199

    Google Scholar 

  • Gray WD, John BE, Atwood ME (1992) The precis of Project Ernestine or an overview of a validation of GOMS. In: Proceedings of the SIGCHI conference on human factors in computing systems Monterey, California, USA, pp 307–312

  • Gray WD, Boehm-Davis DA (2000) Milliseconds matter: an introduction to microstrategies and to their use in describing and predicting interactive behavior. J Exp Psychol Appl 6:322–335

    Google Scholar 

  • Gray WD, John BE, Atwood ME (1993) Project Ernestine: validating a GOMS analysis for predicting and explaining real-world task performance. Hum Comput Interact 8:237–309

    Google Scholar 

  • Green DM, Swets JA (1988) Signal detection theory and psychophysics. Wiley, New York

    Google Scholar 

  • Hammond KR (1955) Probabilistic functioning and the clinical method. Psychol Rev 62:255–262

    Google Scholar 

  • Hansberger JT, Barnette D (2005) Human performance modeling for operational command, control and communication. In: Proceedings of the human factors and ergonomics society annual meeting, pp 1182–1185

    Google Scholar 

  • Hart SG, Staveland LE (1988) Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. Adv Psychol 52:139–183

    Google Scholar 

  • Hassan M, Magee J, MacKenzie IS (2019) A Fitts’ law evaluation of hands-free and hands-on input on a laptop computer. In: International conference on Human–Computer interaction. Springer, Cham, July 2019, pp 234–249

    Google Scholar 

  • Hautamaki BS, Bagnall T, Small R (2005) Human interface evaluation methods for submarine combat systems. Final Report under NAVSEA contract

  • Hawkins HC, Goss S, Cain B, Kerzner L, Sheppard C, Willis RP, Young MJ (2003) Human performance modeling in military simulation: current state of the art and the way ahead (No. TR-TTCP/HUM/02/02): The Technical Cooperation Program (TTCP), Group HUM-human resources and performance. Report of Action Group 19

  • Heeger D (2014) Signal detection theory. Signal, 10:22

  • Hick WE (1952) On the rate of gain of information. Q J Exp Psychol 4:11–26

    Google Scholar 

  • Hooey BL, Foyle DC, Andre AD (2002) A human-centered methodology for the design, evaluation, and integration of cockpit displays. In: Proceedings of the NATO RTO SCI SET symposium, Enhanced synthetic vision system

  • Horrey WJ, Wickens CD, Consalus KP (2006) Modeling drivers’ visual attention allocation while interacting with in-vehicle technologies. J Exp Psychol Appl 12:67–78

    Google Scholar 

  • Hu WL, Meyer JJ, Wang Z, Reid T, Adams DE, Prabnakar S, Chaturvedi AR (2015) Dynamic data driven approach for modeling human error. In: Procedia Computer Science, vol 51, pp 1643–1654, Jan 2051

    Google Scholar 

  • Hyman R (1953) Stimulus information as a determinant of reaction time. J Exp Psychol 45:188–196

    Google Scholar 

  • Itti L, Koch C (2000) A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Res 40:1489–1506

    Google Scholar 

  • Jagacinski RJ, Flach JM (2003) Control theory for humans: quantitative approaches to modeling performance. CRC Press, Boca Raton

    Google Scholar 

  • Janis IL, Mann L (1977) Decision making: a psychological analysis of conflict, choice, and commitment. Free Press, New York

    Google Scholar 

  • John BE, Kieras DE (1996a) Using GOMS for user interface design and evaluation: which technique? ACM Trans Comput Hum Interact 3:287–319

    Google Scholar 

  • John BE, Kieras DE (1996b) The GOMS family of user interface analysis techniques: comparison and contrast. ACM Trans Comput Hum Interact 3:320–351

    Google Scholar 

  • Jonides J (1981) Voluntary versus automatic control over the mind’s eye’s movement. Atten Perform 4:187–203

    Google Scholar 

  • Kahneman D, Tversky A (2013) Prospect theory: an analysis of decision under risk. In: MacLean LC, Ziemba WT (eds) Handbook of the fundamentals of financial decision making: Part I. World Scienfitic, Singapore, pp 99–127

    Google Scholar 

  • Kantowitz BH, Simsek O (2001) Secondary-task measures of driver workload. In: Hancock PA (ed) Stress, workload and fatigue. CRC Press, Boca Raton, pp 395–408

    Google Scholar 

  • Karelaia N, Hogarth RM (2008) Determinants of linear judgment: a meta-analysis of lens model studies. Psychol Bull 134:404–426

    Google Scholar 

  • Karwowski W (2006) International encyclopedia of ergonomics and human factors, vol 3. CRC Press, Boca Raton

    Google Scholar 

  • Karwowski W (2012) A review of human factors challenges of complex adaptive systems: discovering and understanding chaos in human performance. Hum Factors 54:983–995

    Google Scholar 

  • Klein G (2015) Reflections on applications of naturalistic decision making. J Occup Organ Psychol 88:382–386

    Google Scholar 

  • Klein GA (2017) Sources of power: how people make decisions. MIT Press, Cambridge

    Google Scholar 

  • Kramer AF (1991) Physiological metrics of mental workload: a review of recent progress. In: Damos D (ed) Multiple-task performance. Taylor & Francis, New York, pp 279–328

    Google Scholar 

  • Landauer TK, Nachbar DW (1985) Selection from alphabetic and numeric menu trees using a touch screen: breadth, depth, and width. In: ACM SIGCHI Bulletin, ACM, vol 16, No 4, pp 73–78, April 1985

    Google Scholar 

  • Lawton CR, Campbell JE, Miller DP (2005) Human performance modeling for system of systems analytics: soldier fatigue, Sandia National Laboratories, USA. Rep. no. SAND2005-6569, Oct. 1. Online. Available: https://prod-ng.sandia.gov/techlib-noauth/access-control.cgi/2005/056569.pdf

  • Lee B, Oulasvirta A (2016) Modelling error rates in temporal pointing. In Proceedings of the 2016 CHI conference on human factors in computing systems, San Jose, CA, pp 1857–1868

  • Lew R, Dyre BP, Soule T, Ragsdale SA, Werner S (2010) Assessing mental workload from skin conductance and pupillometry using wavelets and genetic programming. In: Proceedings of the human factors and ergonomics society annual meeting, pp 254–258

    Google Scholar 

  • Lim JH, Tsimhoni O, Liu Y (2010) Investigation of driver performance with night vision and pedestrian detection systems—part I: empirical study on visual clutter and glance behavior. IEEE Trans Intell Transp Syst 11:670–677

    Google Scholar 

  • Lin CJ, Wu C, Chaovalitwongsec WA (2014) Integrating behavior modeling with data mining to improve human error prediction in numerical data entry. In: Proceedings of the human factors and ergonomics society annual meeting, pp 854–858

    Google Scholar 

  • Liu Y, Feyen R, Tsimhoni O (2006) Queueing network-model human processor (QN-MHP): a computational architecture for multitask performance in human–machine systems. ACM Trans Comput Hum Interact (TOCHI) 13(1):37–70

    Google Scholar 

  • Mitchell DK, Samms CL, Henthorn T, Wojciechowski JQ (2003) Trade study: a two-versus three-soldier crew for the mounted combat system (MCS) and other future combat system platforms. Army Research Laboratory, ARL-TR-3026. Online. Available: https://apps.dtic.mil/dtic/tr/fulltext/u2/a417992.pdf

  • Morgenstern O, Von Neumann J (1953) Theory of games and economic behavior. Princeton University Press, Princeton

    MATH  Google Scholar 

  • Nasrabadi NM (Oct 2007) Pattern recognition and machine learning. J Electron Imag, vol 16, Art no 049901, Oct 2007

  • Neisser U (1964) Visual search. Sci Am 210:94–103

    Google Scholar 

  • Parasuraman R, Sheridan TB, Wickens CD (2008) Situation awareness, mental workload, and trust in automation: viable, empirically supported cognitive engineering constructs. J Cogn Eng Decis Mak 2:140–160

    Google Scholar 

  • Patterson M, Bond RR, Mulvenna M, Reid C, McMahon F, McGowan P, Cormican H (2016) A web-based human computer interaction audit tool to support collaborative cognitive ergonomics within interaction design. In: Proceedings of the European conference on cognitive ergonomics. ACM, September 2016

  • Payne JW, Bettman JR, Johnson EK (1993) The adaptive decision maker. Cambridge University Press, Cambridge

    Google Scholar 

  • Perlman G (2001) Suggested readings in human-computer interaction (HCI), user interface (UI) development, & human factors (HF)

  • Pew RW, Baron S (1983) Perspectives on human performance modelling. In: Johannsen G, Rijnsdorp JE (eds) Analysis, design and evaluation of man–machine systems. Elsevier, Amsterdam, pp 1–14

    Google Scholar 

  • Poole DL, Mackworth AK, Goebel R (1998) Computational intelligence: a logical approach, vol 1. Oxford University Press, New York

    MATH  Google Scholar 

  • Proctor RW, Van Zandt T (2018) Human factors in simple and complex systems. CRC Press, Boca Raton

    Google Scholar 

  • Rautaray SS, Agrawal A (2015) Vision based hand gesture recognition for human computer interaction: a survey. Artif Intell Rev 43:1–54

    Google Scholar 

  • Rong M, Luo M, Chen Y, Sun C, Wang Y (2014) Human factor quantitative analysis based on OHFAM and Bayesian network. In: Stephanidis C (ed) HCI International 2014—Posters’ Extended Abstracts. HCI 2014. International conference on human-computer interaction. Springer, Cham, pp 533–539

    Google Scholar 

  • Sauro J (2009) Estimating productivity: composite operators for keystroke level modeling. In: Jacko JA (ed) Human–computer interaction. New Trends, HCI 2009. International conference on human–computer interaction. Springer, Berlin, pp 352–361

    Google Scholar 

  • Savage L.J (1961) The foundations of statistics reconsidered. In: Proceedings of the fourth Berkeley symposium on mathematical statistics and probability, contributions to the Theory of Statistics. The Regents of the University of California. vol 1, pp 575–586

  • Sebok A, Wickens C, Clegg B, Sargent R (2014) Using empirical research and computational modeling to predict operator response to unexpected events. In: Proceedings of the human factors and ergonomics society annual meeting, pp 834–838

    Google Scholar 

  • Sebok A, Wickens C, Sargent R (2013) Using meta-analyses results and data gathering to support Human Performance Model development. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp 783–787

  • See JE, Howe SR, Warm JS, Dember WN (1995) Meta-analysis of the sensitivity decrement in vigilance. Psychol Bull 117:230–249

    Google Scholar 

  • Senders JW (1964) The human operator as a monitor and controller of multidegree of freedom systems. IEEE Trans Hum Factors Electron HFE–5:2–5

    Google Scholar 

  • Senders JW (1983) Visual sampling processes. Erlbaum, Mahwah

    Google Scholar 

  • Shannon CE, Weaver W (1949) A mathematical model of communication. University of Illinois Press, Urbana, p 11

    Google Scholar 

  • Siegel AI, Wolf JJ (1969) Man–machine simulation models: psychosocial and performance interaction. Wiley-Interscience, New York

    Google Scholar 

  • Srivastava G, Crottaz-Herbette S, Lau KM, Glover GH, Menon V (2005) ICA-based procedures for removing ballistocardiogram artifacts from EEG data acquired in the MRI scanner. Neuroimage 24:50–60

    Google Scholar 

  • Swets JA (1964) Signal detection and recognition in human observers: contemporary readings. Wiley, New York

    Google Scholar 

  • Swets JA (1996) Signal detection theory and roc analysis in psychology and diagnostics: collected papers. Psychology Press, New York

    MATH  Google Scholar 

  • Szalma JL, Hancock PA, Warm JS, Dember WN, Parsons KS (2006) Training for vigilance: using predictive power to evaluate feedback effectiveness. Hum Factors 48:682–692

    Google Scholar 

  • Tanner WP Jr, Swets JA (1954) A decision-making theory of visual detection. Psychol Rev 61:401–409

    Google Scholar 

  • Tattersall AJ, Hockey GR (1995) Level of operator control and changes in heart rate variability during simulated flight maintenance. Hum Factors 37:682–698

    Google Scholar 

  • Treisman AM, Gelade G (1980) A feature-integration theory of attention. Cogn Psychol 12:97–136

    Google Scholar 

  • Tversky A (1972) Elimination by aspects: a theory of choice. Psychol Rev 79:281–299

    Google Scholar 

  • van Zon NC, Borst C, Pool DM, van Paassen MM (2019) Touchscreens for aircraft navigation tasks: comparing accuracy and throughput of three flight deck interfaces using Fitts’ law. Hum Factors. https://doi.org/10.1177/0018720819862146

    Article  Google Scholar 

  • Wang Y, Ai H, Liang Q, Chang W, He J (2019) How to optimize the input efficiency of keyboard buttons in large smartphone? A comparison of curved keyboard and keyboard area size. In: International conference on human–computer interaction. Springer, Cham, July 2019, pp 85–92

    Google Scholar 

  • Wickens CD, Goh J, Helleberg J, Horrey WJ, Talleur DA (2003) Attentional models of multitask pilot performance using advanced display technology. Hum Factors 45:360–380

    Google Scholar 

  • Wickens CD, McCarley JS, Alexander AL, Thomas LC, Ambinder M, Zheng S (2008) Attention-situation awareness (A-SA) model of pilot error. In: Foyle DC, Hooey BL (eds) Human performance modeling in aviation. CRC Press, Boca Raton, pp 213–239

    Google Scholar 

  • Wickens CD, Gordon SE, Liu Y (2014) An introduction to human factors engineering. Pearson Longman, New York

    Google Scholar 

  • Witus G, Ellis RD (2003) Computational modeling of foveal target detection. Hum Factors 45:47–60

    Google Scholar 

  • Wolfe JM, Horowitz TS (March 2017) Five factors that guide attention in visual search. Nat Hum Behav, vol 11, Art no 0058

  • Wood SD, Kieras DE (2002) Modeling human error for experimentation, training, and error-tolerant design. In: Proceedings of the interservice/industry training, simulation and education conference, pp 1075–1085

  • Wortman DB, Pritsker AA, Seum CS, Seifert DJ, Chubb GP (1974) SAINT: Volume II. User’s manual (AMRL-TR-73-128). Wright–Patterson air force base. Aerospace Medical Research Laboratory, OH

Download references

Funding

This study was funded by National Natural Science Foundation of China (No. 71701205) and National Defence Pre-research Foundation (No. 41412040304).

Author information

Authors and Affiliations

  1. College of Systems Engineering, National University of Defense Technology, Changsha, 410073, Hunan, China

    Ning Li, Jincai Huang & Yanghe Feng

  2. Marine Human Factors Engineering Laboratory, China Institute of Marine Technology and Economy, Beijing, 100081, China

    Ning Li

Authors
  1. Ning Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jincai Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanghe Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanghe Feng.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Communicated by Y. Ni.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, N., Huang, J. & Feng, Y. Human performance modeling and its uncertainty factors affecting decision making: a survery. Soft Comput 24, 2851–2871 (2020). https://doi.org/10.1007/s00500-019-04659-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-019-04659-z

Keywords

Search

Navigation