Abstract
In the MAS evaluation research field there are still few works devoted to evaluating systems’ efficacy, and none of these aimed to measure the adequacy of the MAS in terms of rationality, autonomy, reactivity and environment adaptability. A reliable evaluation method should be general enough to estimate the success of the multi-agent paradigm in different domains, measuring the performances of each single agent and then of the entire MAS. Moreover, it should be able to relate these measures to the environment complexity, that embodies the complexity of the problem solved by the MAS. In this paper a method for evaluating static multi-agent systems is presented and its validation described. The main novelties of the method are that it allows the MAS to be evaluated in the context of the environment in which it will operate, and its adequacy to the environment to be judged from the viewpoints of both the designer, wishful to measure the quality of the designed MAS, and the evaluator, wishful to verify the adequacy of several MASs in a specific context. A validation of the method is described, carried out by evaluating two MASs: the GeCo-Automotive system and a Multi-Agent Tourism Recommender system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Basili, V., Caldiera, G., Rombach, H.: The Goal Question Metric Approach. In: Marciniak, J.J. (ed.) Encyclopedia of Soft. Eng., vol. 2, pp. 528–532. John Wiley & Sons, Inc. (1994)
Di Bitonto, P., Laterza, M., Roselli, T., Rossano, V.: An Evaluation Method for Multi-Agent Systems. In: Jędrzejowicz, P., Nguyen, N.T., Howlet, R.J., Jain, L.C. (eds.) KES-AMSTA 2010. LNCS, vol. 6070, pp. 32–41. Springer, Heidelberg (2010)
Di Bitonto, P., Plantamura, V.L., Roselli, T., Rossano, V.: A taxonomy for cataloging LOs using IEEE educational metadata. In: 7th IEEE International Conference on Advanced Learning Technologies, pp. 139–141. IEEE Press, Los Alamitos (2007)
Casali, A., Von Furth, A., Godo, L., Sierra, C.: A Tourism Recommender Agent: From theory to practice. J. of Inteligencia Artificial 12, 23–38 (2008)
Chidamber, S.R., Kemerer, C.F.: A Metrics Suite for Object Oriented Design. J. IEEE Transactions on Software Engineering 20, 476–493 (1994)
Jang, K.S., Nam, T.E., Wadhwa, B.: On measurement of Objects and Agents, http://www.comp.nus.edu.sg/~bimlesh/ametrics/index.htm
Klügl, F.: Measuring Complexity of Multi-agent Simulations – An Attempt Using Metrics. In: Dastani, M.M., El Fallah Seghrouchni, A., Leite, J., Torroni, P. (eds.) LADS 2007. LNCS (LNAI), vol. 5118, pp. 123–138. Springer, Heidelberg (2008)
Król, D., Zelmozer, M.: Structural Performance Evaluation of Multi-Agent Systems. J. of Universal Computer Science 14, 1154–1178 (2008)
GutiĂ©rez, C., GarcĂa-Magariño, I., GĂłmez-Sanz, J.J.: Evaluation of Multi-Agent System Communication in INGENIAS. In: Cabestany, J., Sandoval, F., Prieto, A., Corchado, J.M. (eds.) IWANN 2009. LNCS, vol. 5517, pp. 619–626. Springer, Heidelberg (2009)
Ben Hmida, F., Lejouad Chaari, W., Tagina, M.: Performance Evaluation of Multiagent Systems: Communication Criterion. In: Nguyen, N.T., Jo, G.-S., Howlett, R.J., Jain, L.C. (eds.) KES-AMSTA 2008. LNCS (LNAI), vol. 4953, pp. 773–782. Springer, Heidelberg (2008)
Lass, R.N., Sultanik, E.A., Regli, W.C.: Metrics for Multi-agent Systems. In: Madhavan, R., Tunstel, E., Messina, E. (eds.) Performance Evaluation and Benchmarking of Intelligent Systems. LNCS, pp. 1–19. Springer, US (2009)
Russell, S., Norvig, P.: Artificial intelligence: A modern approach. Prentice-Hall, Englewood Cliffs (1995)
Wooldridge, M., Jennings, N.R.: Intelligent agents: Theory and practice. The Knowledge Engineering Review 10, 115–152 (1995)
Nwana, H.S.: Software Agents: An Overview. The Knowledge Engineering Review 11, 205–244 (1996)
Wiley, D.A.: Connecting learning objects to instructional design theory: A definition, metaphor and taxonomy. J. of Instructional Use of Learning Objects 2830, 1–35 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Di Bitonto, P., Laterza, M., Roselli, T., Rossano, V. (2012). Evaluation of Multi-Agent Systems: Proposal and Validation of a Metric Plan. In: Nguyen, N.T. (eds) Transactions on Computational Collective Intelligence VII. Lecture Notes in Computer Science, vol 7270. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32066-8_9
Download citation
DOI: https://doi.org/10.1007/978-3-642-32066-8_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-32065-1
Online ISBN: 978-3-642-32066-8
eBook Packages: Computer ScienceComputer Science (R0)