Skip to main content
SpringerLink
Log in

Improving the scalability of formal human–automation interaction verification analyses that use task-analytic models

  • Original Paper
  • Published:
Innovations in Systems and Software Engineering Aims and scope Submit manuscript

Abstract

The enhanced operator function model with communications (EOFMCs) is a task-analytic modeling formalism used for including human behavior in formal models of larger systems. This allows the contribution of human behavior to the safety of the system to be evaluated with model checking. The previous method for translating the EOFMCs into model checker input language was conceptually straightforward, but extremely statespace inefficient. This limited the applications that could be formally verified using EOFMC. In this paper, we present an alternative approach for formally representing EOFMCs that substantially decreases the model’s statespace size and verification time. This paper motivates this effort, describes how the improvement was achieved, presents benchmarks demonstrating the improvements in statespace size and verification time, discusses the implications of these results, and outlines directions for future improvement.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. Note that because the com operator is so different from the others, it was not included in these tests. Because the com operator behavior only effects action behavior, it behaves in accordance with the previous translator [9]. Thus, the new translator should not affect the way com decompositions are executed. Further, no anomalies were observed in the verification results of the realistic benchmarks reported subsequently. Thus, the evidence suggests that com decompositions are behaving the way they are supposed to.

  2. Note that the formal representation was slightly modified to remove the topmost activities’ Done to Ready transitions. This ensured that the task would not repeat due to a Reset and thus not produce action execution sequences outside of a single execution.

  3. A full listing of all of the models used in these analyses can be found at http://fhsl.eng.buffalo.edu/resources/.

  4. It is important to note the the original translator was involved in rigorous validation testing to ensure that it was behaving in conformance with the formal semantics (see [7]).

  5. Note that more verifications were run beyond those used in the realistic benchmarks discussed in Sect. 6. Deadlock checking was also performed on all of the models. No deadlock states were detected.

  6. Note that the modified translator includes miscommunication generation in the same way as the original translator.

References

  1. Aït-Ameur Y, Baron M (2006) Formal and experimental validation approaches in HCI systems design based on a shared event B model. Int J Softw Tools Technol Transfer 8(6):547–563

    Article  Google Scholar 

  2. Aït-Ameur Y, Baron M, Girard P (2003) Formal validation of HCI user tasks. In: Proceedings of the international conference on software engineering research and practice. CSREA Press, Las Vegas, pp 732–738

  3. Angluin D (1987) Learning regular sets from queries and counterexamples. Inf Comput 75(2):87–106

    Article  MathSciNet  MATH  Google Scholar 

  4. Basnyat S, Palanque P, Schupp B, Wright P (2007) Formal socio-technical barrier modelling for safety-critical interactive systems design. Saf Sci 45(5):545–565

    Article  Google Scholar 

  5. Basnyat S, Palanque PA, Bernhaupt R, Poupart E (2008) Formal modelling of incidents and accidents as a means for enriching training material for satellite control operations. In: Proceedings of the Joint ESREL 2008 and 17th SRA-Europe Conference, Taylor and Francis Group, London, pp CD–ROM

  6. Bass EJ, Bolton ML, Feigh K, Griffith D, Gunter E, Mansky W, Rushby J (2011) Toward a multi-method approach to formalizing human–automation interaction and human–human communications. In: Proceedings of the IEEE international conference on systems, man, and cybernetics. IEEE, Piscataway, pp 1817–1824

  7. Bolton ML (2010) Using task analytic behavior modeling, erroneous human behavior generation, and formal methods to evaluate the role of human–automation interaction in system failure. PhD thesis, University of Virginia, Charlottesville

  8. Bolton ML (2013) Automatic validation and failure diagnosis of human-device interfaces using task analytic models and model checking. Comput Math Organ Theory 19:288–312

    Article  Google Scholar 

  9. Bolton ML (2015) Model checking human–human communication protocols using task models and miscommunication generation. J Aerosp Inf Syst. doi:10.2514/1.I010276

  10. Bolton ML, Bass EJ (2009a) Building a formal model of a human-interactive system: insights into the integration of formal methods and human factors engineering. In: Proceedings of the 1st NASA formal methods symposium. NASA Ames Research Center, Moffett Field, pp 6–15

  11. Bolton ML, Bass EJ (2009b) A method for the formal verification of human interactive systems. In: Proceedings of the 53rd annual meeting of the human factors and ergonomics society. HFES, Santa Monica, pp 764–768

  12. Bolton ML, Bass EJ (2010a) Formally verifying human–automation interaction as part of a system model: limitations and tradeoffs. Innov Syst Softw Eng NASA J 6(3):219–231

    Article  Google Scholar 

  13. Bolton ML, Bass EJ (2010) Using task analytic models to visualize model checker counterexamples. In: Proceedings of the 2010 IEEE international conference on systems, man, and cybernetics. IEEE, Piscataway, pp 2069–2074

  14. Bolton ML, Bass EJ (2012) Using model checking to explore checklist-guided pilot behavior. Int J Aviat Psychol 22(4):343–366

    Article  Google Scholar 

  15. Bolton ML, Bass EJ (2013) Generating erroneous human behavior from strategic knowledge in task models and evaluating its impact on system safety with model checking. IEEE Trans Syst Man Cybern Syst 43(6):1314–1327

  16. Bolton ML, Siminiceanu RI, Bass EJ (2011) A systematic approach to model checking human–automation interaction using task-analytic models. IEEE Trans Syst Man Cybern Part A 41(5):961–976

    Article  Google Scholar 

  17. Bolton ML, Bass EJ, Siminiceanu RI (2012) Using phenotypical erroneous human behavior generation to evaluate human–automation interaction using model checking. Int J Hum Comput Stud 70(11):888–906

    Article  Google Scholar 

  18. Bolton ML, Bass EJ, Siminiceanu RI (2013) Using formal verification to evaluate human–automation interaction in safety critical systems, a review. IEEE Trans Syst Man Cybern Syst 43(3):488–503

    Article  Google Scholar 

  19. Bolton ML, Jimenez N, van Paassen MM, Trujillo M (2014) Automatically generating specification properties from task models for the formal verification of human–automation interaction. IEEE Trans Hum Mach Syst 44(5):561–575

    Article  Google Scholar 

  20. Campos JC (2003) Using task knowledge to guide interactor specifications analysis. In: Proceedings of the 10th international workshop on interactive systems. Design, specification, and verification. Springer, Berlin, pp 171–186

  21. Clarke EM, Grumberg O, Peled DA (1999) Model checking. MIT Press, Cambridge

    Google Scholar 

  22. De Moura L, Owre S, Shankar N (2003) The SAL language manual. Technical report CSL-01-01, Computer Science Laboratory, SRI International, Menlo Park

  23. Degani A (2004) Taming HAL: designing interfaces beyond 2001. Macmillan, New York

    Google Scholar 

  24. Degani A, Heymann M (2002) Formal verification of human–automation interaction. Hum Factors 44(1):28–43

    Article  MATH  Google Scholar 

  25. Degani A, Kirlik A (1995) Modes in human–automation interaction: initial observations about a modeling approach. In: Proceedings of the IEEE international conference on systems, man and cybernetics, vol 4. IEEE, Piscataway, pp 3443–3450

  26. Degani A, Heymann M, Shafto M (1999a) Formal aspects of procedures: the problem of sequential correctness. In: Proceedings of the 43rd annual meeting of the human factors and ergonomics society. HFES, Santa Monica, pp 1113–1117

  27. Degani A, Shafto M, Kirlik A (1999b) Modes in human–machine systems: review, classification, and application. Int J Aviat Psychol 9(2):125–138

    Article  Google Scholar 

  28. Degani A, Gellatly A, Heymann M (2011) HMI aspects of automotive climate control systems. In: Proceeding of the IEEE international conference on systems, man, and cybernetics. IEEE, Piscataway, pp 1795–1800

  29. Emerson EA (1990) Temporal and modal logic. In: van Leeuwen J, Meyer AR, Nivat M, Paterson M, Perrin D (eds) Handbook of theoretical computer science, chapter 16. MIT Press, Cambridge, pp 995–1072

    Google Scholar 

  30. Fields RE (2001) Analysis of erroneous actions in the design of critical systems. PhD thesis, University of York, York

  31. Gunter EL, Yasmeen A, Gunter CA, Nguyen A (2009) Specifying and analyzing workflows for automated identification and data capture. In: Proceedings of the 42nd Hawaii international conference on system sciences. IEEE Computer Society, Los Alatimos, pp 1–11

  32. Harel D (1987) Statecharts: a visual formalism for complex systems. Sci Comput Program 8(3):231–274

    Article  MathSciNet  MATH  Google Scholar 

  33. Hartson HR, Siochi AC, Hix D (1990) The UAN: a user-oriented representation for direct manipulation interface designs. ACM Trans Inf Syst 8(3):181–203

    Article  Google Scholar 

  34. Heymann M, Degani A (2007) Formal analysis and automatic generation of user interfaces: approach, methodology, and an algorithm. Hum Factors 49(2):311–330

    Article  Google Scholar 

  35. Heymann M, Degani A, Barshi I (2007) Generating procedures and recovery sequences: a formal approach. In: Proceedings of the 14th international symposium on aviation psychology. Wright State University, Dayton

  36. John BE (2009) CogTool user guide. Carnegie Mellon University, Pittsburgh

    Google Scholar 

  37. Kirwan B, Ainsworth LK (1992) A guide to task analysis. Taylor and Francis, London

    Book  Google Scholar 

  38. Leveson NG, Turner CS (1993) An investigation of the therac-25 accidents. Computer 26(7):18–41

    Article  Google Scholar 

  39. Li M, Molinaro K, Bolton ML (2015) Learning formal human–machine interface designs from task analytic models. In: Proceedings of the HFES annual meeting. HFES, Santa Monica (in press)

  40. Mitchell CM, Miller RA (1986) A discrete control model of operator function: a methodology for information display design. IEEE Trans Syst Man Cybern Part A Syst Hum 16(3):343–357

    Article  Google Scholar 

  41. Palanque PA, Bastide R, Senges V (1996) Validating interactive system design through the verification of formal task and system models. In: Proceedings of the IFIP TC2/WG2.7 working conference on engineering for human–computer interaction. Chapman and Hall, London, pp 189–212

  42. Paternò F, Santoro C (2001) Integrating model checking and HCI tools to help designers verify user interface properties. In: Proceedings of the 7th international workshop on the design, specification, and verification of interactive systems. Springer, Berlin, pp 135–150

  43. Paternò F, Mancini C, Meniconi S (1997) Concurtasktrees: a diagrammatic notation for specifying task models. In: Proceedings of the IFIP TC13 international conference on human–computer interaction. Chapman and Hall, London, pp 362–369

  44. Paternò F, Santoro C, Tahmassebi S (1998) Formal model for cooperative tasks: concepts and an application for en-route air traffic control. In: Proceedings of the 5th international conference on the design, specification, and verification of interactive systems. Springer, Vienna, pp 71–86

  45. Rushby J (2014) The versatile synchronous observer. In: Iida S, Meseguer J, Ogata K (eds) Specification, algebra, and software: essays dedicated to Kokichi Futatsugi. Springer, Berlin, pp 110–128

  46. Wing JM (1990) A specifier’s introduction to formal methods. Computer 23(9):8, 10–22, 24

    Article  Google Scholar 

Download references

Acknowledgments

The project described was supported by NASA under award NNA10DE79C and the National Science Foundation under Grant No. IIS-1429910.

Author information

Authors and Affiliations

  1. Department of Industrial and Systems Engineering, State University of New York at Buffalo, Amherst, NY, 14260, USA

    Matthew L. Bolton, Xi Zheng, Kylie Molinaro, Adam Houser & Meng Li

Authors
  1. Matthew L. Bolton
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xi Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kylie Molinaro
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Adam Houser
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Meng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matthew L. Bolton.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bolton, M.L., Zheng, X., Molinaro, K. et al. Improving the scalability of formal human–automation interaction verification analyses that use task-analytic models. Innovations Syst Softw Eng 13, 1–17 (2017). https://doi.org/10.1007/s11334-016-0272-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11334-016-0272-z

Keywords

Search

Navigation