Skip to main content
SpringerLink
Log in

How Testable are BDI Agents? An Analysis of Branch Coverage

  • Conference paper
  • First Online:
Engineering Multi-Agent Systems (EMAS 2016)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10093))

Included in the following conference series:

Abstract

Before deploying a software system, it is important to assure that it will function correctly. Traditionally, this assurance is obtained by testing the system with a collection of test cases. However, since agent systems exhibit complex behaviour, it is not clear whether testing is even feasible. In this paper we extend our understanding of the feasibility of testing BDI agent programs by analysing their testability with respect to the all edges test adequacy criterion, and comparing with previous work that considered the all paths criterion. Our findings include that the number of tests required with respect to the all edges criterion is much lower than for the all paths criterion. We also compare BDI program testability with testability of (abstract) procedural programs.

This paper has originally been published in N. Osman and C. Sierra (Eds.), AAMAS 2016 Ws Best Papers, LNAI 10002, 2016.

\(\copyright \) Springer International Publishing AG 2016

N. Osman and C. Sierra (Eds.): AAMAS 2016 Ws Best Papers, LNAI 10002, pp. 90–106, 2016.

DOI: 10.1007/978-3-319-46882-2_6

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    More precisely: “software quality assurance (SQA) is a set of activities that define and assess the adequacy of software processes to provide evidence that establishes confidence that the software processes are appropriate and produce software products of suitable quality for their intended purposes.” (ISO/IEC TR 19759:2015(E), pp. 10–15).

  2. 2.

    We focus on system testing. See [20, Sect. 7] for a discussion of different forms of testing.

  3. 3.

    There is also a body of work on formal methods (primarily model checking) as a means of assurance [3, 6,7,8, 10, 16, 23]. However, despite considerable progress, these are not yet ready to handle realistic programs (e.g. see [8]).

  4. 4.

    To avoid confusion between this paper and the earlier work, I will refer to my earlier work with Stephen Cranefield as “Winikoff & Cranefield” in the remainder of this paper.

  5. 5.

    They also compared BDI programs with procedural programs, and found that BDI programs are harder to test than equivalently sized procedural programs, with respect to the all paths criterion.

  6. 6.

    For the purposes of this paper we ignore other possible plan triggers provided by some AOPLs, such as the addition/removal of belief, and the removal of goals.

  7. 7.

    For the moment we avoid specifying whether \(\mathcal {P}\) is the set of relevant plans or applicable plans. The analysis in the next section considers both cases.

  8. 8.

    Colour is used to assist readability, but is not essential.

  9. 9.

    E.g. \(P_1 = a_1 \triangleright a_2 \triangleright a_3 \triangleright a_4\) and \(P_2 = a_5\).

  10. 10.

    Note that for any internal node, the sum of annotations on incoming edges must equal the sum of annotations on outgoing edges, since all paths begin at S and terminate at E.

References

  1. Bordini, R.H., Dastani, M., Dix, J., El Fallah Seghrouchni, A. (eds.) Multi-Agent Programming: Languages, Platforms and Applications. Springer, Berlin (2005)

    Google Scholar 

  2. Bordini, R.H., Dastani, M., Dix, J., El Fallah Seghrouchni, A. (eds.) Multi-Agent Programming: Languages, Tools and Applications. Springer, Berlin (2009)

    Google Scholar 

  3. Bordini, R.H., Fisher, M., Pardavila, C., Wooldridge, M.: Model checking AgentSpeak. In: Autonomous Agents and Multiagent Systems (AAMAS), pp. 409–416 (2003)

    Google Scholar 

  4. Bratman, M.E.: Intentions, Plans, and Practical Reason. Harvard University Press, Cambridge (1987)

    Google Scholar 

  5. Bratman, M.E., Israel, D.J., Pollack, M.E.: Plans and resource-bounded practical reasoning. Comput. Intell. 4, 349–355 (1988)

    Article  Google Scholar 

  6. Dastani, M., Hindriks, K.V., Meyer, J.-J.C. (eds.): Specification and Verification of Multi-agent Systems. Springer, Heidelberg (2010)

    MATH  Google Scholar 

  7. Dennis, L.A., Fisher, M., Lincoln, N.K., Lisitsa, A., Veres, S.M.: Practical verification of decision-making in agent-based autonomous systems. Automated Software Engineering, 55 pages (2014)

    Google Scholar 

  8. Dennis, L.A., Fisher, M., Webster, M.P., Bordini, R.H.: Model checking agent programming languages. Autom. Softw. Eng. J. 19(1), 3–63 (2012)

    Google Scholar 

  9. Ekinci, E.E., Tiryaki, A.M., Çetin, Ö., Dikenelli, O.: Goal-oriented agent testing revisited. In: Luck, M., Gomez-Sanz, J.J. (eds.) AOSE 2008. LNCS, vol. 5386, pp. 173–186. Springer Berlin Heidelberg, Berlin, Heidelberg (2009). doi:10.1007/978-3-642-01338-6_13

    Chapter  Google Scholar 

  10. Fisher, M., Dennis, L., Webster, M.: Verifying autonomous systems. Commun. ACM 56(9), 84–93 (2013)

    Article  Google Scholar 

  11. Gómez-Sanz, J.J., Botía, J., Serrano, E., Pavón, J.: Testing and debugging of MAS interactions with INGENIAS. In: Luck, M., Gomez-Sanz, J.J. (eds.) AOSE 2008. LNCS, vol. 5386, pp. 199–212. Springer, Heidelberg (2009). doi:10.1007/978-3-642-01338-6_15

    Chapter  Google Scholar 

  12. Jorgensen, P., Testing, S.: A Craftsman’s Approach, 2nd edn. CRC Press, Boca Raton (2002)

    Google Scholar 

  13. Mathur, A.P.: Foundations of Software Testing. Pearson (2008). ISBN 978-81-317-1660-1

    Google Scholar 

  14. Nguyen, C.D., Perini, A., Tonella, P.: Experimental evaluation of ontology-based test generation for multi-agent systems. In: Luck, M., Gomez-Sanz, J.J. (eds.) AOSE 2008. LNCS, vol. 5386, pp. 187–198. Springer, Heidelberg (2009). doi:10.1007/978-3-642-01338-6_14

    Chapter  Google Scholar 

  15. Padgham, L., Zhang, Z., Thangarajah, J., Miller, T.: Model-based test oracle generation for automated unit testing of agent systems. IEEE Trans. Softw. Eng. 39, 1230–1244 (2013)

    Article  Google Scholar 

  16. Raimondi, F., Lomuscio, A.: Automatic verification of multi-agent systems by model checking via ordered binary decision diagrams. J. Appl. Logic 5(2), 235–251 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  17. Rao, A.S.: AgentSpeak(L): BDI agents speak out in a logical computable language. In: Velde, W., Perram, J.W. (eds.) MAAMAW 1996. LNCS, vol. 1038, pp. 42–55. Springer Berlin Heidelberg, Berlin, Heidelberg (1996). doi:10.1007/BFb0031845

    Chapter  Google Scholar 

  18. Rao, A.S., Georgeff, M.P.: Modeling rational agents within a BDI-architecture. In: Allen, J., Fikes, R., Sandewall, E. (eds.) Proceedings of the Second International Conference on Principles of Knowledge Representation and Reasoning, pp. 473–484. Morgan Kaufmann (1991)

    Google Scholar 

  19. Vieira, R., Moreira, Á., Wooldridge, M., Bordini, R.H.: On the formal semantics of speech-act based communication in an agent-oriented programming language. J. Artif. Intell. Res. (JAIR) 29, 221–267 (2007)

    MATH  Google Scholar 

  20. Winikoff, M., Cranefield, S.: On the testability of BDI agent systems. J. Artif. Intell. Res. (JAIR) 51, 71–131 (2014)

    MathSciNet  MATH  Google Scholar 

  21. Winikoff, M., Cranefield, S.: On the testability of BDI agent systems (extended abstract). In: Journal Track of the International Joint Conference on Artificial Intelligence (IJCAI), pp. 4217–4221 (2015)

    Google Scholar 

  22. Winikoff, M., Padgham, L., Harland, J., Thangarajah, J.: Declarative & procedural goals in intelligent agent systems. In: Proceedings of the Eighth International Conference on Principles of Knowledge Representation and Reasoning (KR), Toulouse, France, pp. 470–481. Morgan Kaufmann (2002)

    Google Scholar 

  23. Wooldridge, M., Fisher, M., Huget, M.-P., Parsons, S.: Model checking multi-agent systems with MABLE. In: Autonomous Agents and Multi-Agent Systems (AAMAS), pp. 952–959 (2002)

    Google Scholar 

  24. Zhang, Z., Thangarajah, J., Padgham, L.: Automated unit testing for agent systems. In: Second International Working Conference on Evaluation of Novel Approaches to Software Engineering (ENASE), pp. 10–18 (2007)

    Google Scholar 

  25. Zhu, H., Hall, P.A.V., May, J.H.R.: Software unit test coverage and adequacy. ACM Comput. Surv. 29(4), 366–427 (1997)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Otago, Dunedin, New Zealand

    Michael Winikoff

Authors
  1. Michael Winikoff
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Winikoff .

Editor information

Editors and Affiliations

  1. Università degli Studi di Torino, Turin, Italy

    Matteo Baldoni

  2. TU Clausthal, Clausthal-Zellerfeld, Niedersachsen, Germany

    Jörg P. Müller

  3. Universidade Federal do Rio Grande do Sul, Porto Alegre, Rio Grande do Sul, Brazil

    Ingrid Nunes

  4. University of Texas at Dallas, Richardson, Texas, USA

    Rym Zalila-Wenkstern

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Winikoff, M. (2016). How Testable are BDI Agents? An Analysis of Branch Coverage. In: Baldoni, M., Müller, J., Nunes, I., Zalila-Wenkstern, R. (eds) Engineering Multi-Agent Systems. EMAS 2016. Lecture Notes in Computer Science(), vol 10093. Springer, Cham. https://doi.org/10.1007/978-3-319-50983-9_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-50983-9_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-50982-2

  • Online ISBN: 978-3-319-50983-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation